Wrong fact in summarization by universal summarizer

phagi

I summarized the Postgres 17 release news: https://www.postgresql.org/about/news/postgresql-17-released-2936/

And got:

PostgreSQL 17 has been released, with significant performance and feature improvements.

The vacuum process in PostgreSQL 17 is more efficient, using up to 20x less memory.

Write throughput has improved by up to 2x for high concurrency workloads.

Query execution has been optimized, including improvements for IN clauses using B-tree indexes.

PostgreSQL 17 adds support for the SQL/JSON standard, including the JSON_TABLE command.

MERGE statements now support a RETURNING clause and updating views.

Bulk loading and exporting data has seen up to 2x performance improvements.

Logical replication has been enhanced, simplifying major version upgrades.

New monitoring and analysis features have been added, including tracking vacuum progress.

PostgreSQL is the world's most advanced open source database, with a mature feature set exceeding top proprietary databases.

The point I'm taking issue with is that this source info:

PostgreSQL was the first relational database to add JSON support (2012), and PostgreSQL 17 adds to its implementation of the SQL/JSON standard. JSON_TABLE is now available in PostgreSQL 17

, which clearly states that Postgres supported JSON for a long time and now adds a feature, is turned into

PostgreSQL 17 adds support for the SQL/JSON standard, including the JSON_TABLE command

, as if it hasn't supported JSON at all yet.

Basically by confusing the "adds to" with an "adds", it totally changes the meaning.

I wasn't sure if it makes sense to report a bug for an individual summarization, and LLM output should always be taken with a grain of salt anyway, but OTOH a summary is usually expected to be more accurate as it's based on existing input and not thin air, so maybe the system prompt or something else can improve this.

I expected the summarizer to only summarize the given facts, and not misinterpret or misrepresent them.

Luis

phagi thank you for taking the time to report. As @gladiator2339 mentioned, it's crucial to review the outputs of language models, especially when they present unexpected or counterintuitive information...

As this technology matures we expect new models, and/or techniques will make these issues less prevalent. I'll mark this ticket as donesince there isn't much we can do for now.

We do appreciate you taking the time to report -- please continue to report any concerns!

gladiator2339

From docs:

From an information retrieval point of view, relevant to our context of a search engine, we should acknowledge the two main limitations of the current generation of AI.

Large language models (LLMs) should not be blindly trusted to provide factual information accurately. They have a significant risk of generating incorrect information or fabricating details (confabulating). This can easily mislead people who are not approaching LLMs pragmatically. (This is a product of auto-regressive nature of these models where the output is predicted one token at a time, and once it strays away from the "correct" path, for which the probablity grows exponentially with the length of the output, it is "doomed" to the end of output, without the ability to plan ahead or correct itself).

Seems like the problem is the LLM model itself, not something that the system prompt can do anything about it.