adisibio GPT-4o mini faster, cheaper, and higher rated than Claude 3.0 Haiku. Quick answers would return faster and with higher quality results.
mullinggroove adisibio Did you do those tests yourself? LLM benchmarks are a joke at this point, and higher performance could just indicate the model was better trained on the benchmark questions. Rephrasing these questions can severely impair performance, showing this effect. There is no way around testing the model for Kagi's specific usecase and comparing performance to the current model.
oktoberpaard mullinggroove I guess that's up to the Kagi team to test and decide. However, for the free version of the code assistant GPT-3.5 is used, so for that one I would be very surprised if GPT-4o mini isn't superior.
