Link directly to a news source when a news aggregator site would normally be in the search result

dorkbutt

I'm currently drawn to Kagi over DuckDuckGo because of DuckDuckGo's reliance on Bing causing its news section to be flooded with its MSN sort of proxy site instead of direct links to sources. And some of that has leaked over into the results of Kagi, the prime culprit I'd point to is Yahoo!News, which seems more like an aggregator than actual source. It aggravates me, yes, but it also inconveniences, and outright disables some, users from looking deeper into a particular journalistic enterprise as they are subjected to a UI of another website. Is there a way for search results that carry these middleware sites to instead crawl a bit deeper to link the original source?

Yahoo!News proxy article:
https://news.yahoo.com/trump-reveals-homelessness-policy-tent-212222760.html

Original article:
https://www.independent.co.uk/news/world/americas/us-politics/donald-trump-homelessness-policy-tent-cities-b2322102.html

Vlad

dorkbutt Should be happening lot less with our recent update. Can you confirm?

dorkbutt

Vlad I'd say that the results roughly improved. There still remains aggregator sites like HeadTopics (which doesn't relate at all to what I searched). Also while it is a contentious topic, the lack of mainstream news outlets is concerning. It unnerves me that as you scroll past the initial few searches, the immensely problematic propaganda site Breitbart sits next to The New Yorker, a comparably more reputable source. I believe this might stem from the "We are down-ranking sites with a lot of ads and trackers more aggressively" change you made with the web search, as most of the prominent news sources possess plenty of trackers.

Vlad

dorkbutt We do our best. We do use ads/trackers as a signal and note that NewYorker has more ads/trackers (20) than Breitbart (13). This is why we gave users the power to block sites they dont want to see, which you can use on News results page too.

cbirdsong

I was also looking for a way to block the MSN proxy articles. Does Kagi take the <link rel="canonical"> into account? It points to the correct article for the one I ran into today:

<link rel="canonical" href="https://www.theverge.com/23848754/samba-de-amigo-party-central-review-nintendo-switch">

Vlad

cbirdsong Can you show how to reproduce results from msn?

cbirdsong

Vlad I either can't reproduce or can't remember whatever search surfaced that link yesterday, but if you search for a news-y thing with site:msn.com then you see a bunch of them, which all* have canonical links to the actual source:

https://kagi.com/search?q=the+verge+site%3Amsn.com&dr=2

https://kagi.com/search?q=iphone+event+site%3Amsn.com&dr=2

These don't seem to show up in the "News" filter, though:

https://kagi.com/news?q=iphone+event+site%3Amsn.com

*well, all the ones I clicked through and inspected.

Vlad

cbirdsong Thanks I was looking for confirmation that msn.com naturally surfaces in Kagi which is what we would not want.

cbirdsong

This may need to be a separate issue, but the same thing is happening with Yahoo News' syndicated articles inside the News filter:

Search: https://kagi.com/news?q=The+Toxic+Avenger
Yahoo result: https://www.yahoo.com/entertainment/peter-dinklage-dons-tutu-elijah-150000738.html?guccounter=1
Original article that Yahoo result's canonical points to: https://ew.com/movies/the-toxic-avenger-photos-elijah-wood-peter-dinklage-kevin-bacon/