Hi all
We are currently running a search quality experiment where we are removing one of our search providers in an effort to measure the impact on search quality and inform future pricing decisions.
Running less search data providers means:
a) faster results
b) less cost to us
Obviously the possible drawback is worse overall search results.
The plan is run this experiment for 24h or until we have enough feedback to revert.
Let us know how this feels.
My most immediate impression is that it seems there are fewer section-level links provide under the top result. I'm often searching for company news. Typically with Kagi, this has meant I search for the company by name. The first result is to the company website home page, followed by six or so section-level links within that site, one of which is usually something like "News," making a quick click to get where I'm wanting to go. I miss having those sub-links a lot. (But maybe it's all in my head, since I don't know exactly what you actually changed ;-)
Vlad I hate to sound like a radical, but that's probably an 8 for me, because I use those links frequently every workday. Since I have finite budget for search and I'm already paying for Neeva, I'm looking at Kagi through the lens of "Is it better than Neeva for my most common use cases? Does it help me do my work faster?" I can probably justify only paying for one or the other.
With respect to the sub-section links, Kagi is 100% better in discovery and presentation of those, not to mention faster in returning the results, which is a big vote in its favor. Obviously Kagi has other amazing aspects I appreciate. But those sub-section links really help me get where I'm going faster, which is gold.
It took me a single query to find out which one was disabled and which one was active. I would 10/10 stop using Kagi as the active one does not work for my searches most of the time. The one in use now provides too broad results for my typical queries and most of the times, irrelevant results.
Searching for people might have become worse. If you search for someone non-famous/non-tech but with some internet footprint, it feels like they're filtered out and no relevant results come up. DDG/Bing seem to come up with much better results -- I don't think this was always the case but maybe I only just noticed it. This is a narrow use case and using other engines is fine.
For something with reproduceable results and where the quality does seem a problem -- search for setup_freespace_2_1.20_v2_(33372).exe on Kagi and DDG (this is a file needed to run an open source project). The file is mentioned within the context of the open source project, distro packages which need it, and related content on DDG. It seems to only come up with the original download of it on Kagi. Again, will have to check tomorrow but similar specific file searches have fared better in the past.
Other recent searches which are missing some relevant results:
front mission 3 similar plot (reddit discussions and some relevant listicles missing)The Industrial Revolution: A Captivating Guide to a Period of Major Industrialization and the Introduction of the Spinning Jenny, the Cotton Gin, Electricity, and Other Inventions (book -- I think there were more reviews/related content before)okx livepeer(top link should be https://www.okx.com/markets/prices/livepeer-lpt)I tried a few other searches and the results I wanted were all still at the top though the total amount that came up seemed reduced in some cases (which is fine in those cases).
Are you able to say which provider was dropped?
Vlad Something like a 6-7 -- I'd be willing to keep paying but only just whereas otherwise I'd be more open to price increases. More specifically, the people search isn't a big concern but the specific file searches come close to really breaking it (that one is a toy search but there's job-specific ones where I really need the better results). Of the other queries, the only one which contributes majorly to the score is okx livepeer -- suggests there will be queries where the data is strangely unreliable.
Not sure if this is directly correlated with this change, but if it is, then probably a 10. If Kagi cannot even bring results about general questions people have, then this would be a major turn-off not only for me but I'm sure many people.
Edit: I've tried "best laptops 2022," and this outputs results but asking the question fully like in the screenshot does not produce anything. Very weird consistency issues. Nevertheless, my opinion still stands as there shouldn't be this problem for a search engine as you would expect to be able to ask almost any 'normal' question and get links
Subjectively, plant and animal searches feel starkly more superficial (using one use case). It's easy to see with a basic search like "Schizachyrium scoparium", which (iirc) previously returned much more material for the specialist concerned with this species's role in wild habitat from academic and government sources rather than a completely boring laundry list of nurseries, botanical gardens, and such just rephrasing the same basic info. Many of these sites would be ones I'd normally Lower in my results ranking, even.
With DuckDuckGo showing me essentially identical first page results for the above search, for me this experiment renders Kagi almost an Abridged version of DuckDuckGo results without even recourse to seeing More Results at the bottom of each page. This seems a massive handicap, particularly if Lenses don't improve quality of results by much (which they do not seem to be doing for me, here).
On a scale of 1-10, I might give 5 or 6. I'm aware that most of my searches each month probably don't need advanced search capabilities (that is, almost any search engine will probably give me what I want a good chunk of the time), and I'm keen on the people behind Kagi, having a UI devoid of clutter and ads, as well as privacy features like the image proxy and the graceful degradation that allows search to run without javascript enabled. But I'll admit being stuck with results like these going forward would dim my enthusiasm for Kagi search since it would no longer be my primary go-to for more specialized information, and as such I'd be much less likely to recommend it to others.
8/10 missing quite a few important results I would have gotten previously
