Universal Summarizer gets some info wrong

kaghat

Steps to reproduce:
1- Summarize this article: https://www.lemonde.fr/les-decodeurs/article/2023/04/28/cabas-moins-remplis-premiers-prix-plebiscites-poisson-sacrifie-comment-l-inflation-s-illustre-au-quotidien_6171402_4355770.html

2- Summary says:
"The rise in prices has also affected the fish market, with consumers reducing their purchases of fresh and frozen fish by 13.7% and 7%, respectively."
(It says something similar in the french summary, which is also wrong)

Expected behavior:
If you deepl translate the article, you'll see this:
"L’entreprise d’étude de marché a ainsi calculé que les achats en poissonnerie avaient reculé de 13,7 % en volume au premier trimestre, quand la viande accuse une baisse de 7 %."

translated to:
"The market research company calculated that fish purchases were down 13.7% in volume in the first quarter, while meat was down 7%."

which is accurate.
I'm assuming this paragraph is what the summarizer uses.

Expected: summarize that meat is down 7%, not frozen fish.

Debug info:
Browser: librewolf
OS: fedora
kagi region setting: france
kagi server: Connected to: US-WEST

Vlad

kaghat Thanks for reporting, it is probably limitation of the smaller model we use. Have you tried summarizing this with Muriel model (available through API)?

kaghat

Thanks for the reply.
So the article has not changed on that precise point, but the summary on the website has.
It now does not refer to that "7% meat" part.
This is what I just got now:

Inflation has become a daily reality for French households, with prices rising by around 15% for food products in the past year. This has led to a decrease in food consumption, with households opting to buy less, switch to cheaper brands, or buy smaller portions. The use of store-brand products has increased, with sales of these products rising by 13.3% in the first quarter of 2022. In contrast, sales of national brands have decreased by 4.2%. The rise in prices has also led to a decrease in the consumption of fish, with sales of fish products falling by 13.7% in the first quarter of 2022. Inflation has been driven by a combination of factors, including the pandemic, supply chain disruptions, and rising energy prices. The inflation rate has remained around 6% for the past year, the highest level since the 1980s. The French government has called for negotiations between producers and distributors to address rising prices.

As you predicted, the Muriel model is better at this:

[...] Fish purchases have declined by 13.7% in volume in the first quarter, while meat purchases have dropped by 7%. [...]

Vlad

kaghat Yes it is cost vs quality. we recommend using Muriel for mission critical loads.