Add sonar LLM to assistant

Dra9n

Kagi should add perplexity’s new sonar LLM to the assistant as it gives near instant answers and is built to utilize web search.

Users could use the LLM for faster and more precise answers when searching for general information.

Vlad

Dra9n When you say fast, can you give an idea of what you mean? Have you used the API personally?

RoxyRoxyRoxy

From their pricing page the API seems to be comparable in price to Claude 3.5 Sonnet. Would love to see the benchmark results to see how exactly it measures up but it's claimed to be pretty good and very fast.

pophob

This seems more like a different search tool rather than a raw LLM offering. It seems like their API actually does the web searches iteself. I'd be curious to see if/how it would integrate with Kagi's existing LLM search tools.

Seems interesting.

https://docs.perplexity.ai/guides/pricing#pricing

rxzlion

I'm against including any offering from perplexity.
Their services are unstable and full of bugs.
They themselves say at their FAQ:
"Does Perplexity provide service quality assurances such as service uptime, frequency of failures, and target recovery time in the event of a failure?

We do not guarantee this at the moment."

On top of that:

they use their own search and it doesn't stand up to the standards of kagi.
what they call sonar are just Llama 3.3 70b
sonar-reasoning-pro is just R1 hosted in USA
sonar-reasoning is a distilled R1 based model

Kagi assistant and upcoming products do the same in a better more reliable way.

RoxyRoxyRoxy

Vlad Not him but I found anecdotal evidence of it being quite fast. In the blog post that is linked to they claim 1200 tokens / sec.

rxzlion

RoxyRoxyRoxy

Vlad

Just my 2 cents I have used it and I don't know where the number they post come from.
I'm a heavy user of both kagi assistant and Claude API and I haven't seen any of those speed advantage that perplexity claim.
I can tell you that actually kagi assistant is actually the fastest AI solution I have used to date both in time to first token and streaming.

Vlad

RoxyRoxyRoxy They are refering to Cerebras AI model that serves llama 70b which they use at 1200 tok/sec. that still does not cover time needed to search etc.

RoxyRoxyRoxy

Vlad Ah, guess I fell victim to the marketing then lol, my bad!