When I do a specific search on the different AI machines through Kagi, I get almost the same answer regardless of the selected machine. If I perform the same search directly on the page of each chatbot, the answers are totally different from each other and different from those received via Kagi? Is this normal?
I would love to be wrong and that the problem is on my side.
I've found that with the Kagi Assistant, your answers are usually relying more on the top web search results than the model's internal knowledge.
When you ask using ChatGPT directly, it's relying more its training data to answer unless you use the search mode. This means it is pulling from a broader pool of knowledge for each answer, but is more likely to hallucinate on specifics or give an outdated answer. That's also why there's more variation when you ask the same question again.
When you ask GPT-4o in the Assistant, it's focused on using information from the top search results. This means the quality of the response is going to depend on the quality of the search results. In your case, it looks like the government's critics were underrepresented in the top results so you got a less critical answer. And because the search results will be the same each time you ask the same question in the same way, the responses will be more or less the same.
Thank you very much for your quick and detailed reply. I think you are right, I didn't think about the possibility that changing AI machine in the same conversation would not have (as much) influence on the result produced. I will repeat the search by opening a new conversation in each case, so the context should change and ‘new’ answers should be produced each time.
Again, thank you very much for your help.
This is something I found out recently too. I set up Msty (app) with a variety of LLM APIs and the web search just doesn't match what I can get from ChatGPT.com or Kagi's Assistant. So on top of training data and favoring web search results, you have web search quality differences (based on my experience). Kagi's Assistant is amazing because of how they implement web search. Since I use LLMs as a search engine, web search quality is huge for me most of the time.
Also, just a side note, an acquaintance and I have Chat GPT Plus subscriptions. We found out yesterday that despite asking the SAME QUESTION to the same model - we both got different answers (one right, one wrong). :/
Adding to classicsc point, we also have system instructions we use for the different LLMs which could also help explaining that behaviour.
Also, just a side note, an acquaintance and I have Chat GPT Plus subscriptions. We found out yesterday that despite asking the SAME QUESTION to the same model - we both got different answers (one right, one wrong). :/
There's inherent randomness in the LLMs response generation process--this is usually referred to as "stochastic processes" if you want to learn more about it 
