Twice today (and several more times in recent memory), Assistant did not search the web or cite any sources in response to a query, even when I had the "Web Search" toggle enabled. The resulting answers are inadequately grounded in publicly-available knowledge.
Two example conversations from today:
GrapheneOS eSIM Compatibility
This one's a bit long but it represents the experience well. For the first several conversation turns, we see one or more web searches for each response. But then here:
That response doesn't do a search or cite any of the source material. Only when I ask does it search the web, although then it incorrectly says Verizon is the carrier with issues, when the linked GOS forum threads specify Cape, not Verizon.
Granted, this is a long conversation and goofy things happen when the context window fills up. The next one is super short.
Fashion Genre for Casual Glamour
I ask a question about current fashion (which I expect may not be represented in the model's internal 'knowledge'). The initial response is improvisational -- it does contain "rich mom" but it's buried several paragraphs down. Only on the second response does it search the web for the best answer, and cite where it came from.
My expectation is that Assistant respects the user's preference to search the web. This seemed to be the case for months, but perhaps isn't anymore. With the toggle on, it's as if the model is able to search, but not required, and sometimes it won't.
I will occasionally switch off Web Search for a faster response when I expect a good answer to be well-represented in the model itself (or when I don't expect the answer to be on the web at all), but usually I leave it on.
Assistant is a strong product that I'm happy to pay for, in part because it grounds its statements of fact in source material from Kagi's high-quality search results. The new reluctance to do a search seems explainable as a cost-cutting measure, but I hope that isn't the case.
