"Simple" mode for low-cost searches

riddley

While I really appreciate the quality of Kagi's search results for research, sometimes I want to do simple straightforward or trivial searches, where Duck Duck Go would have the specific information I'm looking for in the first few results. It seems wasteful to use up my Kagi quota for these searches but it's inconvenient to jump into DDG or Wikipedia.

I gather that Kagi's biggest expense is downstream searches from Google. Would it be possible to have a "cheap mode" that omits Google or other expensive providers and just provides a basic quality search, but counts less towards my account quota?

Vlad

riddley This is a cool idea. I'd assume it would make sense if we are able to bring it to half a search status. Let's take it bit further.

How would a user enable this? How would this be shown in stats?

Dodo

Yeah, I was also thinking that. Once someone's search quota for the month is complete, they can stop using google and bing api and use Kagi's self-generated results.

riddley

Vlad Do you think it would be possible to have an "unmetered" mode so that these searches were completely unmetered for paid users? Or would they still have to use a fractional amount of quota?

Vlad

riddley The best we can do is half a regular search (by calling just 1 out of two providers for example), but unmetered is impossible as we incur cost for providing it. That does not work for you?

riddley

Vlad If only a "low cost" mode is possible I still think that would be a valuable feature

However, one risk I see is the user accidentally falling into this bad use case: user unintentionally makes a "low cost" search because they have the low cost option turned on, they realise they meant to have made a regular search to start with so they have to run the search again in normal mode, now they have paid 1.5x normal rate for a single search

The other risk I see for Kagi is a user turning on low-cost mode accidentally or without properly understanding what it does, being unhappy with the quality of the results, and then you have a customer support and/or reputational problem.

So I think the UI would have to address both of these

maxime

I also think this would be a useful feature. It feels wasteful making multiple searches in quick succession (e.g. for language / program documentation).

Using bangs to search DDG !ddg <search term>, Google !g <search term> or SO !so <search term> isn't too cumbersome, although it's slightly annoying to think about which bang to use.

Perhaps a similar syntax (e.g. $ <search term>) could be used to trigger a low-cost mode, where results are returned at the lowest cost to Kagi?

Vlad

riddley What @maxime suggested (using '$' to prefix low cost searches) sounds like a good idea ?

riddley

I somehow missed maxime's comment. I would like that approach. Maybe it doesn't even need a GUI option. Although there could be conflicts with searches that naturally started with a "$" e.g. "$200 billion Facebook loss"

TheAllegedGenius

riddley What about "!$" so it looks like a bang? Nobody is going to type "!$" on purpose.

riddley

Another thought on this, if you implement it: I'd like an option for the browser plugins to make location bar searches default to low-cost mode without any prefix.

Vlad

THe problem with this will be discoverability. There are so many things that exist Kagi that don't exist elsewhere but we try to make them intutive in the UI.

riddley

Have a button for "cost-reduced limited search" in the UI, and also have a bang "!$" to trigger it. (Similarly, have the existing GUI interface for lenses and also have a way to trigger them through bangs.)

I don't think it's that hard for for you user base. Currently bangs have no GUI discoverability and power users already love them. And as I've said elsewhere I predict you'll have much larger proportion of power users than the free search engines do.

Personally I believe cost-per-search is going to be your biggest barrier to adoption and you can't afford to ignore anything that could ameliorate it for users.

Vlad

riddley I agree with your assesment it is just not clear how would this help with adoption. The pricing that we publish will be standard and this is a power user feature. A user would already need to accept the pricing and become a user only to discover this as a way to lower their cost. So for adoption purposes it does not seem like it would do as much. (what would help with adoption is a way to reduce cost overall for every search, but we are already close).

This feature really helps reduce the cost for existing users and is useful as such. How would you expose it in the UI?

danny

Previous experiment with lowering the search quality of all queries didn't make people happy.

Instead I suggest adding an advanced setting so each user can decide for themselves what they want as the default. If the default is "lower cost", then include a button to get the full quality result. And maybe !bang for address bar usage: https://kagifeedback.org/d/148-simple-mode-for-low-cost-searches

By doing this Kagi provides the flexibility for users to have some say in controlling their own cost. It also provides a feedback mechanism for what queries need full quality and what queries don't.

Possibly even allow this setting to be configurable per client. E.g. desktop vs mobile have different settings.

And all this would be confusing for non-heavy users, so maybe put it behind an Advanced setting section.

Jett

As suggested on Discord, perhaps a special lens (or bang) could allow a user to search only on Kagi's internal index. We can expect Kagi to grow over time and have a nicely curated (at first) and broader internal index in the future.

This would eventually allow Kagi to reduce search costs by not hitting external indexes at all, especially for some types of searches where it is not necessarily needed, such as:

Queries to official websites: The internal index could initially crawl and index the home pages of important websites to return these results.
Queries to documentation websites: The internal index could initially crawl and index a list of curated websites per domain. We are talking about programming documentation such as popular websites (MDN), languages (Python, Rust, Go, etc.), frameworks (Next, Nuxt, etc.), and libraries (React, Vue, etc.). Other subjects could benefit from this idea, but programming searches for documentation websites might be a fair share of the total searches for power users of Kagi.
Queries to popular websites with stable content: The idea above could be extended to popular websites with "stable" content (i.e., websites of finite size in some sense), such as Wikipedia. A "finite" size means that the crawler does not need to crawl such sites too often for new content (unlike news websites), which can also reduce costs.

The advantage is that a user can keep the internal index lens by default and switch to no lens by clicking the lens toggle button to allow a full search on all indexes if the initial one is unsatisfactory. Perhaps the internal index lens could count for less or not at all for a user's search quota. Maybe the flat membership fee can fully cover the infrastructure costs of crawling those websites and serving those queries.

Of course, there are other issues to consider. How could a user create a custom lens to search both the internal index only and specific sources only?

From a UX perspective, I can think of three approaches to this idea:

Having a broad "internal index" lens (or bang). The disadvantage of this approach is that the user could not easily create a lens for specific topics. Example for an MDN lens: how to create a lens to search only on the internal index and only on the MDN?
An "internal index" toggle next to the lens toggle in the Advanced Search panel, or a drop-down menu that allows the user to choose between internal and full index, along with other drop-down menus such as region, time, etc.
Allow the user to choose between internal index only or full index (including external indexes) when creating a lens.

Vlad

Jett I've merged a couple of threads realizing they all revolve around the same topic.

Besides things I said here
https://kagifeedback.org/d/1207-provide-a-cheap-way-to-query-for-official-sites/9

and here
https://kagifeedback.org/d/1207-provide-a-cheap-way-to-query-for-official-sites/12

Lets first understand the primary motivation of the user. Making a few assumptions: In this case I believe it is to provide a way to reduce cost. The most meaningful cost reduction would happen if a cost for a query would be reduced to zero.

Users already can have a zero cost query if they use a bang, for example !yt to go directly to youtube. Since our Custom bangs feature is introduced user can have zero cost queries for navigating/searching all their favorite websites.

The second use case is to finding 'big sites'.

Kagi could provide an internal index of sites on the web that could be accessed at no cost. This would be an equivalent of querying Kagi's internal index. This is quite doable for us -> the only question is how is this accessed:

is it a bang? Which one?
is it a new special char like $ as originally requested

What I think is not possible (at least short term) is expanding that index to other custom sites, like developer specialised sites. This is because cost of doing that would outweigh any benefit.

So in a nutshell, what is the best way to access the quick Kagi index that will be simple to use, intuitive and sicoverable for new users?

danny

The idea that some queries aren't valuable enough to warrant full quality isn't new: https://kagifeedback.org/d/148-simple-mode-for-low-cost-searches

While the proposed lower cost pricing model might include enough searches for the non-heavy users, there still needs to be a solution for the heavy users. Developers who are heavy users will have a category of search dominated by searching for technical documentation. One way to make this category cheaper for them is to allow lower quality search in certain situations.

For JavaScript developers, maybe sites like MDN, nodejs.org, reactjs.org, etc might be bundled into a JavaScript Lens. This kind of site-specific search should be serviceable by lower cost search query provider. So by allowing Lens to be configured to be lower cost, JavaScript developers can have a way to do site-specific technical documentation search for cheaper.

Eventually when there is enough signal from Kagi users on what sites are more frequently used, then a targeted indexing effort on those sites can bring down the cost for developer users even more.

danny

One category of searches is simply to find the official website of something.
For example, inkscape.org for "inkscape" or pytorch.org for "pytorch".

Allow users to configure a setting such that when there is result from what Kagi knows as official sites to avoid doing a costly query. Just provide a single answer to the official website.

Then provide a button to trigger the full query if user wants.

Kagi could seed this index by getting keywords and the associated website URL from Wikipedia. Then add it as necessary.