Duplicated GitHub discussion entry

STM

Here's the search in question: https://kagi.com/search?q=zfs+metadata+device&r=de&sh=4Azjr7CqMuGHS9D6ix3DCg

I've searched for "zfs metadata device" and it came up with two (effectively) identical entries for GitHub discussions:

The first link is this: https://github.com/openzfs/zfs/discussions/14542
The second link is this: https://github.com/openzfs/zfs/discussions/14542?sort=top

Notice how only the sorting option of the individual comments in the discussion is different.

From my understanding this is a bug and should be fixed.

Only https://github.com/openzfs/zfs/discussions/14542 should be listed in my opinion.

Vlad

STM Not easy for a machine to understand this is a bug. The urls are different and the web page will be rendered differently. Not an easy problem to solve.

STM

I don't know anything about Kagi's internals, so pardon my ignorance, but I'd assume adding a simple rule for Github to ignore https://github.com/${X}/${Y}/discussions/${Z}?sort={STRING}, when https://github.com/${X}/${Y}/discussions/${Z} is already present in the results, should be an achievable task.

Edit: Or rather always use https://github.com/${X}/${Y}/discussions/${Z} instead when any occurence of https://github.com/${X}/${Y}/discussions/${Z}?sort={STRING} is in the results and then deduplicate. Because no one really needs a preset sorting option in the search results.

Vlad

STM The way we see this is hoilistically. We'd rather fix it in systemic way than have a databse of rules for every site on the web where this is a problem (github is not the only one).

KagiForMe

Vlad
I don't think there is a perfect way to fix this holistically.

Some URL parameters matter, so we can't just strip all of them.
Yet stuff like ?sort=top (GitHub) or ?top=month (reddit) introduces redundancy.
Just removing those parameters will break a few sites (especially smaller ones probably?)

All in all, I see two options:

Make a database of rules for big sites (GitHub, Reddit, ...)
or
Make a rule that strips those parameters and make a database of exceptions.