Kagi Search Quick Answer Should Not Use Reddit As Source

LittleTabasco

The quick answer relies on reddit. It says the developers stated that they are not open source because they are a small team and want to make a living.

There is zero evidence the developers have publicly stated this. The source it cites is just some random reddit user that is not associated with the app development in any way. Kagi Quick Answer passes the information as a developer statement.

Other quick search LLMs do not make this mistake as they seem to able to distinguish between a random reddit user and a developer.

If reddit is removed as a source for quick answers, users may be more likely to get quality verifiable information instead an AI hallucination.

MustafaD

Reddit sometimes has very good answers and discussions.
I think it is to check credibility of poster rather than abandon a useful site.

LittleTabasco

How would a quick search LLM determine the credibility of a poster?

It 100% can't be by karma/upvotes/downvotes. The site has the strongest social anchoring effect than any other social media site. The first few upvotes/downvotes on a post/comment significantly determine if the post will get more upvotes/downvotes.

It's a myth that upvotes = quality/correct information.

Most reddit information is parroted information. It would be much better to to go the actual source.

Edit: And my example the post that just made up information had many upvotes. There is no truth to their post, just a vibe with herd mentality upvotes.

RoxyRoxyRoxy

This is a hard one, as some people really like Reddit as a source, and others (obviously) don't. Would like input from community members on this.

fxgn

RoxyRoxyRoxy if the information comes from an untrusted source like Reddit (or any other social media/user-generated content), the model should mention it in text and say "according to a Reddit discussion" instead of stating it as a fact.

I also have this in my system prompt for regular Assistant but it completely ignores it usually

MustafaD

I find Reddit very useful for some sorts of topics, especially ones where technical and pragmatic opinions and experiences are more relevant than hard facts. For example, topics related to gaffer/grip work.
In fact, sometimes, this is the only place where there seems to be info (opinions and discussions).

A very similar case applies to websites like Quora. Surprisingly, sometimes there are absolutely excellent answers there from pertinent professionals (my experience was with Physics topics). There is often a mix of answers, from really bad, to ok, to really insightful. Perhaps AI should NOT pick an answer or weigh at all, but simply point to the discussion if it seems interesting enough.

Reddit is also good for news about disappearing websites/contents, historic issues (in hindsight), etc, particularly on fan and topic pages.

Personally, I do not use Reddit much. But my experience with Google has shown that it is an invaluable resource in topics like Amateur Radio, Grip & Gaffer work, product and service fan discussions (eg: fitness programs), etc.

I generally find these useful for different kinds of info: StackExchange network (programming & dev stuff), Reddit (technician, hobbyist/amateur, and DIY stuff), Quora (Engineering & Physics), specialized forums (for specialized topics like audiophiles, telecine, running Windows 7 these days 😊, etc).

MustafaD

fxgn That's a great idea. It should only pick discussions that seem valuable and fruitful.