Assistant v2 Ignoring Personalized Results

Dustin

I've observed that personalized results/blocked domains are being ignored with various models since the move to Assistant v2. Most frequently/reproducibly, I have seen Grokipedia being used as a reference for queries when it is fully blocked on my settings.

Respect user filtering as prior.

widow5131

Not sure if the same root cause but assistant is getting lots of US centric websites which are not applicable to my queries

beetstabasco

widow5131

The U.S. centric websites may possibly be related to this: https://kagifeedback.org/d/1088-irrelevant-results-appear-related-to-my-ip-address-location

Dustin

I can confirm this is still happening in some regards. I have Facebook blocked in my domain filtering and it's still pulling from there.

Dustin

Can confirm this is still happening. I switched to Kagi's Quick assistant to test this and am seeing fully blocked sites being used as references.

Dustin

Assistant continues to ignore personalized results, surfacing results from blocked sites. It also consistently ignores my region and location settings, giving me US results in Canada and putting currency in USD and so on.

Dustin

Can confirm this continues. I see Assistant pulling information from blocked sources such as Reddit.

Dustin

Can confirm this continues. Quick Assistant did not reference any of my raised and pinned news sources when querying recent local news.

z64

@Dustin Heya, can you please share some example threads where this occurs?

And if you're handy with devtools, capturing the details (headers, request body, ..) of the outgoing prompt request would be a big help too, so we can verify the expected parameters are sent. Be sure to censor any values such as your kagi_session cookie.

Thank you!

Dustin

z64 thanks for looking into it. I will try to provide some examples for you.

Dustin

Here are two previous examples.

https://assistant.kagi.com/share/5b3ce0ef-0bea-4531-983a-6a451ac8ebff

https://assistant.kagi.com/share/9ba22a63-c1f6-4bf0-b117-5c8b403cb579

In these two examples, I see Reddit and Facebook being used as references, which I have set to 'block'. Additionally, x.com is being referenced which I have set to 'lower', But in that case, I'm slightly less concerned because it's not set to fully block.

I will attempt to recreate this with the devtools enabled and then share that as well.

z64

Thanks! The company is at a get-together this week so I don't have someone I can grab to look on the backend at just this moment, but could you confirm for me that in your view of these threads, that you have these boxes that indicate that a search was actually performed? Basically just trying to rule out that the assistant didn't just completely make up the links/citations.

Lastly, just a quick sanity check, this toggle is enabled for you?

Dustin

Hey, yeah. I can confirm personalized results is toggled.

For the "Range of Thread protocol Matter + Thread IKEA devices, etc" query, I will show you the following as well:

And when I look at the references and click through the links, they all appear to be real and relevant so it seems to be doing what it's supposed to.

Dustin

Ok, after a bit of effort, I managed to get a fresh generic query to produce results from a blocked source (Reddit):
https://assistant.kagi.com/share/5badc9e5-a8be-4c31-830e-f759736f09d2

curl 'https://assistant.kagi.com/api/v2/conversations' \ -X POST \ -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:155.0) Gecko/20100101 Firefox/155.0' \ -H 'Accept: */*' \ -H 'Accept-Language: en-CA,en-US;q=0.9,en;q=0.8' \ -H 'Accept-Encoding: gzip, deflate, br, zstd' \ -H 'Referer: https://assistant.kagi.com/' \ -H 'Content-Type: application/json' \ -H 'Origin: https://assistant.kagi.com' \ -H 'Alt-Used: assistant.kagi.com' \ -H 'Connection: keep-alive' \ -H 'Cookie: turnstile-session= \ -H 'Sec-Fetch-Dest: empty' \ -H 'Sec-Fetch-Mode: cors' \ -H 'Sec-Fetch-Site: same-origin' \ -H 'Priority: u=0' \ -H 'TE: trailers' \ --data-raw $'{"content":"So what are people sayin\' about Kagi?","model":"ki_quick","enable_search":true,"lens_id":null,"personalization":true}'

So it seems like here it explicitly looks at Reddit to get a response to my query, which means it's not referencing the personalizations/filtering before it goes out to gather information perhaps?

z64

Thanks @Dustin ! I've identified the issue and we should have a fix out within the next 24hrs.

Sorry for the delay, as we split Assistant from the Search team some wires have been a bit crossed and such / some things fell through the floorboards that opened up. Appreciate your patience.

Dustin

z64 Glad to hear! Thanks so much for looking at it