"Simple" mode for low-cost searches

Jett

This suggestion may be related to this thread in some way. I gave some ideas about queries to official websites (and others like documentation websites) in a reply.

This suggestion could be broadened. Sometimes a user wants a website but does not know the full URL, especially the TLD (see the Inkscape example). However, many users are accustomed to using Google to get to frequently visited sites like YouTube, Twitter, etc., rather than typing it into the browser bar and using the history (or better, a bookmark).

One idea to solve this costly problem could be to implement a toggle that allows searching only in Kagi's internal index, since a full index search is unnecessary in such cases. Also, a results page with only one entry might be awkward.

Several ideas have been suggested here, such as crawling Wikipedia to get the URLs of official websites. Another option might be to index the most visited sites (like the top 1 million) in the hope that this would cover programming sites. It might also be possible to get a list of all registered domain names, so that at least a lot of home pages could be indexed (this is a problem for subdomains, though). There are probably other ideas, such as indexing DMOZ entries.

carl

After you've visited a website once, their URL will be suggested the next time you type the name in your browser URL/Search bar. At least on Chrome, Safari, Orion and probably every other browser. Is this not good enough?

MildManneredBen

carl The issue with this is that browsers can be configured to disable auto-suggest based on history. Users also sometimes ignore the options presented by the browser and run the search anyway.

My suggestion would be for Kagi to recognise these kinds of queries and launch a flow that guides users into an onboarding of the Bang feature

Vlad

A recent example after we finally pulled the plug on any kind of optimization for cost was for a query 'twitter'.

User (rightfully) submitted a bug report beceause they were getting no news when searching for 'twitter'. In 99% cases when someone types twitter they want to go to twitter.com. But those days Twitter takeover was in the news and they were expecting the news, which we did not server as a part of cost optimization.

So this is is very hard to automate.

Now the original suggestion from the OP is to have a toggle in the settings where the user would opt-into being served lower cost result for 'navigational' results such as this one.

In our experience when we had this on for the entire user base, out costs went down 2-3% mening that these queries are not that frequent. If users were to opt in, the impact would be even lesser (both ways).

I am glad suggestions like these are being proposed and I wanted to share more information so we can come up with the right solution.

Vlad

In general, this is tricky because

Cost of additional crawling and indexing specialized sites may outweigh the nebefit (at least short/mid term)
Such upfront investment would still apply only to a portion of users (not everyone is a developer)
Even among developers, variety of documentation sites is vast (MDN maybe satisfies the needs of 5% of devleopers)

Ultimately I think a search engine should always strive to serve the best possible results it can. The goal should be that those results are so good and so fast, that the user does not need to think about fraction cost of a ₁ cent, because the value they are getting is so much higher (especially if this is used in development context, or any another context, where the results provided ordreds of magnitude more value than that 1 cent).

It is similar to having a light on while reading at night. The fractional cost of the light is outweighted several times by the ability to read your favorite book. I think thiis is what we should strive for Kagi search.

Open to discussion and merging this to the previous thread to have everything in one place.

Vlad

Jett I've merged a couple of threads realizing they all revolve around the same topic.

Besides things I said here
https://kagifeedback.org/d/1207-provide-a-cheap-way-to-query-for-official-sites/9

and here
https://kagifeedback.org/d/1207-provide-a-cheap-way-to-query-for-official-sites/12

Lets first understand the primary motivation of the user. Making a few assumptions: In this case I believe it is to provide a way to reduce cost. The most meaningful cost reduction would happen if a cost for a query would be reduced to zero.

Users already can have a zero cost query if they use a bang, for example !yt to go directly to youtube. Since our Custom bangs feature is introduced user can have zero cost queries for navigating/searching all their favorite websites.

The second use case is to finding 'big sites'.

Kagi could provide an internal index of sites on the web that could be accessed at no cost. This would be an equivalent of querying Kagi's internal index. This is quite doable for us -> the only question is how is this accessed:

is it a bang? Which one?
is it a new special char like $ as originally requested

What I think is not possible (at least short term) is expanding that index to other custom sites, like developer specialised sites. This is because cost of doing that would outweigh any benefit.

So in a nutshell, what is the best way to access the quick Kagi index that will be simple to use, intuitive and sicoverable for new users?

Jett

Vlad It is really hard to find a definitive solution to these problems, and after some thought I came to the conclusion that you are right and that it is probably not worth having Kagi fully index some sites, at least in the short term.

My idea with the internal index was, first, to avoid websites' internal search engines, which are often poorly designed or poorly relevant compared to Kagi's results. Second, it was to reduce the cost for queries like mdn border-radius or wikipedia crystal programming language. But I realized that Kagi already offers a lot of bangs like !mdn, !mdnhtml, !mdncss, !mdnjs, and !w.

Therefore, I think that in the short term, if people are really concerned about their usage, they could probably use bangs, learn about them in some way or another, and put up with websites' internal search engines until a reasonable solution is found.

Regarding the second "use case" about "big websites", or let's say websites' homepages, I think the solution comes from the "go to" suggestions. They probably need some love to make them really good for almost any website, big or small, from any region. With this feature, at least the user is still free to do the full search if they want.

Now a follow up question is whether we want to add a setting so that when a "go to" suggestion pops up, hitting Enter will follow it without having to explicitly select it. Of course, the default behavior if no "go to" suggestion is found would be to perform a full search.

This behavior could also be triggered by a special character or a bang, as you suggested, and have the same default behavior as mentioned above.

riddley

Vlad I have a suggestion: an option to differentiate between searches made from the browser location bar, or from the Kagi search page.

I was thinking about the fact the before Google launched Chrome, we had two text entry fields on the browser shortcut bar: a location bar and a search field. Google started the trend to merge these (obviously to their own benefit, as it made Google the default way of navigating the web).

If we think about breaking out these two functions again: the location bar is for finding and visiting a specific website, the Kagi search box is for text searching/research.

So the differentiation is not "searching big sites" vs "searching all the other sites", it's "getting to one specific site that I don't know the URL of" vs "searching web content".

I feel I often waste a Kagi search when I know what site I want but I've forgotten or don't know the URL. In these cases I don't need more than the most basic and shallow index of the website content: probably just the headers and footer. Often just the URL without any content at all would have been sufficient.

So Kagi would need a broad but extremely shallow internal index of the whole web, as opposed to a narrow but deep index of the major sites. (This whole suggestion hinges on the idea that such a shallow index would be feasible for Kagi to maintain, I have no idea whether this is true...)

A couple of real-life examples: I haven't looked at Kagi feedback for a few months, and I couldn't remember the name of the website. Was it kagisupport, kagifeatures? It wasn't coming up in my history drop-down in the location bar, so I did a search for it.

After I've posted this, I need to visit my local council website. It's called something like "inner west council". But I don't know the exact URL, so I have to search for it. (I just searched on Kagi, it's www.innerwest.nsw.gov.au).

In your example, someone typing "twitter" in the location bar wants to go to twitter.com, but if they type "twitter" in the Kagi search field they want to research news about twitter.

So my suggestion is: in the Kagi search plugins, have a user-selectable option to make location bar searches go to the "basic results" page, which has Kagi's top 10 guesses for the site the user was looking for, and a big "continue your search on Kagi" button. These results are free.

If the user wants to make a regular Kagi search, they can do it in one of three ways: launch the search from the location bar and then click on the "continue" button; navigate to kagi.com and start the search there; use a bang.

Users who are on metered plans and are price sensitive, like me, can turn on that feature. Everyone else can ignore it. You could pro-actively promote it to users who use up their quota.

Vlad

I agree this is not clear.

For reference "go to" discussion is here https://kagifeedback.org/d/1228-go-to-search-suggestions-improvements/

Another relevant thread is on site search: https://kagifeedback.org/d/1231-search-operator-for-site-filtering-via-existing-bang-shortcuts

riddley

riddley Thinking about this a bit more, something else I frequently do from the browser bar is unit and currency conversions

allklier

A variation of this - if you look at SEO traffic reports, people have gotten really lazy with typing and bookmarking domains and there is a huge amount of 'navigational search' traffic. You type 'Amazon' instead of 'Amazon.com', etc. etc.

In some case there really is no full search results page required if there is an obvious domain name resolution.

It could be a light search page, that just shows 'Did you mean Amazon.com or something else' - that page would automatically redirect after 5s (configurable) to amazon.com, unless you clicked 'something else' in which case it goes to a full SERP.

This light weight could help with cost, and could also maybe accounted for as a light query.

I know I can type the .com, but after years of not having to think about it, navigational search is now an expected convenience.

And ideally the solve should be automatic, not another bang, since we're trying to retain trained behavior, not change it.

Vlad

We are going towards the future where we can again offer unlimited searches for ~$10/mo.

Still there can be ways to achieve savings that this thread highlights. Would love if someone could summarize or add the main improvements we can make that would improve the producitivity of users while achieving savings on our end.

douglasg1b

I can only assume the $10/m for unlimited searches is a result of lowering the cost/search, which makes arguments for specific user-reported search patters which may be trivial in cost somewhat dates/inconsequential?

Is it worth discussing habits that would normally be search-wasteful where I use search to go to websites instead of going to them directly (ie, typing in stackoverflow in search instead of going there directly)? If these are search trends you (Kagi) can already identify and pick ways to lower costs, I'm guessing that this discussion may be worth closing then.

Kagibeh

What about preview suggestions? A lot of people probably (ab)use search because they only half remember an URL they would easily recognize if they'd be given the option to click on it in a dropdown.

Also, Google has recently expanded their preview suggestions. If you type "Orlando Bloom" you will see the name with a small pic in the preview options. While with Google, this initiates a Google search, Kagi could take a different approach and send users directly to a popular information source (like Wikipeda). As long as it's clear where you will be heading, this could be a cool feature and a cost-saver all in one.

EDIT: Preview suggestions could also be configurable or simply based on what is set up under "search with..."

Vlad

Kagibeh Cna you share more information about this? Do you mean this feature?

carl

Kagibeh What about preview suggestions? A lot of people probably (ab)use search because they only half remember an URL they would easily recognize if they'd be given the option to click on it in a dropdown.

This is built into every browser. For example on Safari you will always get the most common website as a suggestion when typing into the URL bar, even if you don't have the site in your bookmarks or history.

Kagibeh

Vlad Ok, I realize I didn't do my homework properly - my apologies.

I wasn't even aware that Kagi could do this, because I do 95% of my searches from the browser search bar. In Edge, there are no preview suggestions when you search from the search bar. Just installed Kagi search in Firefox to confirm this: no preview suggestions in the search bar. When I switch to Google as the default search, I am getting those.

In response to carl's post: None of the browsers I mentioned gives me URL suggestions. I type "twitter" and Kagi suggests searching "twitter video downloader". Nobody - neither the browser nor Kagi - suggests visiting twitter.com directly.

carl

Which browsers did you mention? Safari will suggest the most likely web page always, while Chrome and Firefox will suggest from your bookmarks and history. That covers most people, unless the user deliberately configures their browser to disable suggestions.

Kagibeh

carl Carl, you seem to mean well, but your replies to my posts are typically a mix of denying, ignoring and twisting what I wrote. Please don't do this. Unless Vlad needs more info, I honestly don't have more to say about this.

« Previous Page Next Page »