Support of rich / search suggestions

Browsing6853

Firefox just added image preview to the search bar

https://www.mozilla.org/en-US/firefox/122.0/releasenotes/

Kagi should support it too

Vlad

Browsing6853 Any information on the format they require?

Browsing6853

Vlad

You can easily look at that on google.com

curl https://www.google.com/complete/search\?q\=beyon\&client\=gws-wiz

Grooty

In the Firefox 122.0 (Changelog), they added support for search suggestion images:

This works for Google, but not for kagi:

I expected the images to show up, like they do for Google. I don't know if this is a Firefox issue or a Kagi issue as Google seems to be the only search provider where it works (tested DDG and Bing, both didn't work)

Vlad

Grooty There is no standard for this. Firefox just decided to support Google's auto suggestion format without publishing any guidelines for other search engines.

Grooty

Vlad oh okay, that's annoying. Is there a standard for it and/or is this something that could be added through the extension

Vlad

Grooty There is no standard, we'd (or our community) have to do some reverse engineering how Firefox does it and then it would still probably not work in other browsers.

Browsing6853

@Vlad

I took some time reverse engineering the search bar requests from Brave browser and Firefox. Apparently "www.google.com/complete/search" is called differently depending on the client:

google.com: it returns a piece of javascript, it's very easy to inspect but doesn't return calculations (see https://kagifeedback.org/d/2763-results-for-conversions-in-open-search-suggestions) and isn't suitable for a search bar autosuggestions.
firefox: The desktop search bar on Firefox supports calculator and images/descriptions. It's a very simple and clean endpoint (https://www.google.com/complete/search?client=firefox&q=%s&channel=fen).
Chrome based: They have more complex features on the search bar, which they call "omnibox". It includes all that is included on Firefox plus additional features such as currency conversion (extremely useful) and translations (it doesn't work so well).
These advanced features are probably more expensive to chrome and are not triggered directly using a single api call. Kagi shouldn't take these complex calls from google, but I can get some sample of the result for currency conversion if needed.

So to fix/improve Kagi's autocomplete and get both calculator and images working, you can focus on the following examples:

Firefox, calculator, should work for more math formulas, like "1+1)/3": curl https://www.google.com/complete/search\?client\=firefox\&q\=1%2B1%29%2F3\&channel\=fen
Firefox, "beyonce", should show images: curl https://www.google.com/complete/search\?client\=firefox\&q\=beyonce\&channel\=fen
Chrome, calculator: curl -H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36' 'https://www.google.com/complete/search?client=chrome-omni&gs_ri=chrome-ext-ansg&q=1%2B1%29%2F3'
Chrome, images: curl -H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36' 'https://www.google.com/complete/search?client=chrome-omni&gs_ri=chrome-ext-ansg&q=beyonce'. The image urls format are encoded in some format that I haven't tried to decode.

btw, I'm open to work and would love be paid to dive deeper and implement such features

DonReba

Firefox 122.0 introduced a feature, where the search box displays images and descriptions for results in engines that support this. Google is the only engine that does, AFAIK. It would be good, if Kagi supported this, as well.

https://www.mozilla.org/en-US/firefox/122.0/releasenotes/

DonReba

I posted a request for guidance on Mozilla Connect.

wooigak

Kagi should support the new Firefox 122 feature displaying images and descriptions for search suggestions - https://www.mozilla.org/en-US/firefox/122.0/releasenotes/

Problem, it seems that it's not standardized and only supporting Google as today - https://searchfox.org/mozilla-central/rev/f60bb10a5fe6936f9e9f9e8a90d52c18a0ffd818/toolkit/components/search/SearchSuggestionController.sys.mjs

Here the Mozilla Connect Idea "publish a standard for search engine suggestions": https://connect.mozilla.org/t5/ideas/publish-a-standard-for-search-engine-suggestions/idi-p/52059

Use your fingers to type words for a new search, boom, it displays some suggestions when images and descriptions.

On Microsoft Edge, the same feature is available with Bing, but not Google. On Firefox, there are no suggestions displayed with Bing. I didn't try with Google Chrome, cause I don't have it installed.

Vlad

wooigak This is a duplicate issue as far as I can recall, care to find the original so that we can merge?

chris_20017

While I am not keen on the images, I see value in the description. It could be retrieved from Wikidata for individuals, works of art and so on (see e.g. https://m.wikidata.org/w/index.php?search=Beyonc%C3%A9&ns0=1&ns120=1)

(I have deactivated suggestions based on my Google experiences, as it often distracted me)

wooigak

Vlad Are you talking about https://kagifeedback.org/d/2763-results-for-conversions-in-open-search-suggestions/? Maybe it should be renamed.