Use Kagi Assistant to refine search results with AI assisted lenses

whee

Some search terms are overloaded with meaning and the quality of the results is highly dependent on the context of the search. For example "Rust" could mean the programming language, the color, the game, oxidation, etc. Searching for "Rust tips and tricks" currently returns these results (prototype example helpfully provided by Vlad):

Rust Guide: Top 50 Rust Tips and Tricks | GAMERS DECIDE
gamersdecide.com  › articles › rust-guide-top-50-tips-and-tricks
Oct 6, 2018  Top 10 Rust Tips and Tricks To Improve Your Gameplay Rust is an unforgiving game that has a very steep learning curve. In this article I will go over 10 tips and tricks that will help you improve your gameplay and allow you to overcome the odds that are stacked against you.

Rust Tips and Tricks
mudit.blog  › rust-tips-and-tricks
Dec 31, 2021  Rust lang Tips and Tricks · Enum size is bounded by the largest member · Avoid unnecessary clones · Modularizing tests · The dbg! macro · Using _ to ...

Rusttips | #1 Resource for Aim Training, Calculators, and Rust Guides
rusttips.com 
Sep 8, 2023  Equip, Aim, Conquer in Rust. ... Gampine on the other hand helped with a few ideas around to improve and the patience with my coding and not playing.

Rust Tips and Tricks #PartOne. A Comprehensive Guide to... | Medium
medium.com  › @danmugh › rust-tips-and-tricks-partone-5bac0f948b22
May 26, 2023  Rust is a programming language that has gained a reputation for its exceptional qualities in terms of reliability, speed, and overall enjoyment for the programmer. However, it is important to keep in mind that Rust is also recognized as a language with a relatively high level of complexity.

Three Tips for Tackling Rust
http://spin.atomicobject.com  › 2014 › 05 › 09 › tips-learning-rust
May 9, 2014  Rust is a relatively new systems programming language that has been garnering a lot of attention over the past year or two as a compelling alternative to languages like C and C++. Programmers who work “closer to the metal” than I do, will likely appreciate Rust’s memory safety and low run-time overhead (many of Rust’s features and safety guarantees..

When I am in the context of programming, half of the results are noise. I would rather see this:

Rust Tips and Tricks
mudit.blog  › rust-tips-and-tricks
Dec 31, 2021  Rust lang Tips and Tricks · Enum size is bounded by the largest member · Avoid unnecessary clones · Modularizing tests · The dbg! macro · Using _ to ...

Rust Tips and Tricks #PartOne. A Comprehensive Guide to... | Medium
medium.com  › @danmugh › rust-tips-and-tricks-partone-5bac0f948b22
May 26, 2023  Rust is a programming language that has gained a reputation for its exceptional qualities in terms of reliability, speed, and overall enjoyment for the programmer. However, it is important to keep in mind that Rust is also recognized as a language with a relatively high level of complexity.

Three Tips for Tackling Rust
http://spin.atomicobject.com  › 2014 › 05 › 09 › tips-learning-rust
May 9, 2014  Rust is a relatively new systems programming language that has been garnering a lot of attention over the past year or two as a compelling alternative to languages like C and C++. Programmers who work “closer to the metal” than I do, will likely appreciate Rust’s memory

The existing lens implementation doesn't quite solve this problem because they're site-oriented instead of content-oriented. I don't know what sites to add to a lens to get the results I want. I do have a general idea of what I want to see in the results, though, if I'm in a certain context.

A new Kagi Assistant lens type would allow users to specify the context of searches and what kind of results they want to see. For example, a "Work" Assisted lens could be defined as:

What would you like Kagi Assistant to know to provide better results?

I am a software developer. This is my tech stack:

Rust

SQLite

...

Kagi Assistant would use this information to refine the search results when searching with this lens.

warpspin

Not quite the "magical" result of LLMs and self-description, but maybe Kagi could also work a bit more with the wikidata ontology here in general. For the rust example, there's for example:
https://www.wikidata.org/wiki/Q575650 (the language) or
https://www.wikidata.org/wiki/Q543923 (the color)

I imagine, while building up its inverted index during crawling, when it encounters a word like "rust", it could try to match it to one of the wikidata concepts, maybe based on the other words on the page, using word vectors or whatever (yeah, that's the hard part) and then instead of plainly storing a "rust" item in the index, stores a Q543923 or Q575650 item instead.

Then, when you search and it sees, for a word it has multiple underlying concepts stored and it cannot infer from the search automatically which you mean (though I suppose, once you solve the problem of how to map pages to wikidata concepts, you'd solve this one too for many queries at least), it directly offers you a refinement choice on top of the search results: "Do you search for: rust - the programming language, rust - the color"

httpjames

These types of lenses need to be fast. Maybe Kagi should test out quality with Google Chat Bison since it's pretty quick.

Vlad

The idea would be define an AI type lens, which would be used to filter out search results based on user preferences.

We'd love bit more thought put behind UX for this and how would you ideally use this in the workflow?

whee

Initially I was thinking it's another lens configuration option:

And then the lens list adds some kind of indicator to make it clear it's "different", if it's important people know it's more costly:

(Not on the left like this, it looks bad 🙂)

One downside is that you wouldn't be able to combine a built-in lens with this kind of filtering, though.

Vlad

whee It feels like it is almost an entirely different lens mode. It would apply to your normal results where AI would do the re-ranking based on description. This will obviosuly come at a penalty of latency (and cost). How would you imagine using this? Always on? It feels by the time you toggle this you would already find what you are searching for.

Please add more thoughts! 🙂

whee

Vlad Yes, it is like another layer on top of normal results, but I am concerned too many layers could have confusing interactions (e.g., between lenses and AI filtering).

I don't think I would have it on all the time. There are a few "search contexts" I have in mind that are recurring:

At work development: it would define tech stack, types of results (briefer results with example code preferred, etc.)
At home, hobby development: different tech stack, different types of results (long form articles OK)
Gaming: would include platform, subscription details, maybe even game-specific AI lenses

Every "project" in my daily activities is a scenario where I have predictable overall expectations for results. They have similar kinds of noise. I think this would filter the noise out up front, instead of additional searches with more specific criteria in the cases where there's just too many unrelated results.

xeophon

How does Kagi currently rank the results? Does it just take the results from an external provider and re-rank them if the user provdes preferences (Bump/Block etc)?

I am opposed to this idea in general for various reasons:

LLMs are the wrong tool for this. This is a problem for ranking / recommendation systems. A LLM would be too slow, too costly and would introduce new problems (LLMs having too many or too less context as an example)
I think most, if not all described use cases can be solved by writing a better search query. If I program at work, I might write Spring Boot Webserver to get relevant results. The same query (Webserver) but for fun projects, would be FastAPI Webserver. Similarly, if one uses a Mac to game, prepending macOS to queries becomes mandatory, anyways. Rewriting the query will yield better results for those examples, anyways: If Kagi fetches only Windows-related Gaming sites, the LLM cannot re-rank the sites to include any macOS results (because there are none)

beshr

The suggestion here is to allow filtering results (with an optional text input for example) that would allow Kagi results page to be filtered using an LLM instructions. This is sort of like Lenses but with an LLM instruction basically. Maybe it can be part of creating/editing a lens even.

httpjames

Have you guys considered https://cohere.com/rerank?

Vlad

httpjames The issue here is not technology (we know how to do this) but cost and latency if using an LLM for this.

mkaic

Often-times I find myself blocking the same type prototypical SEO-spam websites over and over in my search results — something which I feel an LLM would probably be pretty good at automating! Given Kagi's openness to AI integrations thusfar, do you think there'd ever be a chance we could get (opt-in, off-by-default, obviously) LLM-based filtration of our search results based on our own personal blocklists, or maybe even based on some public database of opted-in user's blocklists? I just feel like there's probably a great opportunity here to take a bunch of volunteered website ranking data and factor it into the search algorithm from the get-go.

Like if I block 10 different SEO-spam auto-generated nothingburger websites, Kagi should be able to automatically detect that I really don't like those sorts of sites, and downrank them after asking me for permission.

I imagine there'd be two levels of this:
1) a less-effective, but more private option that uses zero-shot methods, where an LLM looks at the search results for a given query and compares them against content from pages I've rated as good or bad, and then makes filtration decisions based off of that.
2) a probably-more-effective, but more complex option where opted-in preference data from many Kagi users is pooled and used to finetune an LLM specifically for page ranking based on human preferences.

As far as interfaces go, I feel like a simple toggle in User Preferences to turn either of these two filtration methods on or off would be sufficient.

whee

As an update, Bing has implemented this feature. The UI is pretty easy to use and may be good reference.

After performing a search, you get the ability to filter results by "AI tag":

Which would let me filter out the game Rust vs the language Rust:

(Admittedly this organization of tags isn't completely correct, but it is Bing data…)