Malware website filtering

yokoffing

Vlad Where would this warning be? On the bottom of the results?

Yes, precisely.

KagiForMe

Vlad yokoffing

I disagree, I think the warning should be on top.

Arguments:

More often than not, I do not scroll down to the bottom of the results page. This way, I am not getting informed on the removal of results. Contrasting that, on top I can see the information at a glance.
Putting the info at the top does a better job of user communication. Removal of Malicious Domains is actually a feature. Let the average user be aware of this as well, it is a selling point!

Further comments:

Maybe add a link to the list used (in a "?" tooltip).
Give us the option to disable the filter globally.

Vlad

KagiForMe Care to show a mockup of your proposal?

KagiForMe

Vlad
The "?" should be a symbol with the tooltip attached. Maybe a yellow warning triangle in front.

carl

KagiForMe

I think the bottom of the page is the right place for the message. With a warning on top, normal people will associate Kagi with internet threats and be frightened to use it.

KagiForMe

carl
As a power user (and most of Kagi's users are power users, who spread the word to other power users) would feel very uncomfortable with the fact that search results might be filtered without seeing it at a glance. I do not want to scroll down every time to the end of the page to see this.

It feels belittling/condescending to me as a user. Should you consider making "at the bottom" the default, then give us the possibility to disable it entirely and show a warnign symbol next to the search result if it is disabled and the result is malicious. Preferred option would be a toggle "at the bottom/at the top", but I understand how that could be maintainance debt.

I do understand your point and you are probably right. But part of being an ethical search engine is to be as transparent about censoring results as possible. @carl, I feel like our opinions are not mutually exclusive and could be easily reconciled with a toggle setting.

carl

KagiForMe

I think it is a very rare occasion when a user wants to have malware infected sites in their results, but maybe a setting somewhere to let users opt-in to have these results would be a solution.

90% of power users will probably never want to see any malware sites in their results, so I don't think it is fair to have this warning on top, which can become obnoxious depending on what kind of things the user searches for.

Just for curiosity: In what situations do you want to see malware sites among your search results or know that they've been filtered out?

KagiForMe

carl

Just for curiosity: In what situations do you want to see malware sites among your search results or know that they've been filtered out?

I was just sitting here for 5 minutes, thinking of use cases that are not super constructed and actually apply to me. Honestly: I would rarely ever wish to see malicious domains. It is more of a fundamental transparency aspect that is worrying me.

[...] I don't think it is fair to have this warning on top, which can become obnoxious depending on what kind of things the user searches for.

You are right on this one. I didn't think about the fact that searching for "download pdf reader" and "sexy custume" and such this message would be shown every time. By now, I am somewhat convinced to put it at the bottom.

[A] setting somewhere to let users opt-in to have these results would be a solution.

I am still advocating very much for this. I trust Kagi, but the list in question is a third party ressource that can go malicious at any time (E.g. see https://www.wired.com/story/developer-altered-open-source-software-to-wipe-files-in-russia/). I trust the list now, but would want a backup if they do pull stuff like this.

@carl, thank you for your time and effort, you have convinced me! 🙂

lagomorph

Steps to reproduce:

The best example is a kagi search for "foldimate".

Foldimate was a company that was building a laundry folding robot in the early 2010s who went bankrupt 2021.

On a kagi search for this, no articles regarding the companys bankruptcy appears - instead, an article saying the product "finally works now" from the Verge, which is woefully out of date, and a secondary link to a scam website "selling" the device, in spite of the fact that the device has never made it to market. The Wikipedia summary on the side does little as well to clarify that the company no longer exists, other then speaking in the past tense.

Obviously this is a very niche search - but I'm surprised on clicking the shield icon that there is no way to report a website as spam, malicious, etc - this would be a great feature.

In fact, the only "red" shield on the page is for Mashable, who has too many trackers. The search listing also includes 3 more links to scam stores, which are not marked at all.

Expected behavior:

I don't necessarily think that scam links should be fully purged from listings, but I do think that either the shield, or an additional scam warning, should appear for reported sketchy links.

Image/Video:

yokoffing

Yes, we should have a way to report malicious links.

lagomorph I don't necessarily think that scam links should be fully purged from listings

I think they should be removed entirely from listing. A site that is obviously a scam or known to lead directly to viruses, malware, phishing, etc. should be removed.

Vlad

lagomorph

This is tricky to execute on.

Suppose we had a "flag as malware" system.. Then that means now we have to have dedicated people reviewing this all the time. We can not just automatically de-index something because someone flagged it as malware.

And suddenly it becomes non-scalalable, manual, operation that nobody want do so as the rules are not exactly defined and there is a lot of gray area what defines a malicious website.

To someone a major news site is malicious because all the tracking they do, and for someone else because they do not agree with its politics.

It is a very complicated problem.

You can imagine people flagging sites of their competitors.

You can imagine people flagging random websites they don't like.

And if we chose to ignore that, then why you have a system in the first place?

lagomorph

As a comparison, QWANT has exteremly similar search rankings but has stripped out all the scam links.

Edit, this is no longer true - QWANT now also actively pushes these scam sites to top rankings.

9rqnq

Vlad Does Kagi employ any mechanism to delist from index or block known phishing or malware websites? There are many threats that are relatively easy to be avoided with automatic filters. For example NextDNS use many filters to block such sites, like Google Safe Browsing, IDN Homograph Attacks, or block newly registered domains. All this settings are also optional. They also have their own AI model to filter suspicious sites, and I believe something like this should be possible to create in Kagi.

Personally, security is much more important to me than privacy. And I see privacy as a subset of security. Someone who doesn't have my personal data will have a much tougher time attacking me (through hacking or scams). But it is not impossible. Search engines are our window to information on the Internet. That's why they should not only be private but also secure. Elimination of advertisements is, of course, an important step towards more security, but it is not only necessary. And right now, I see only one SE that is capable of implementing such measures, and it is Kagi.

ak42

Vlad
about which blocklist to use, why not just allow users to select their own regex based blocklists in settings ? with compatibility with adblocker blocklist formats.
and have the browser check regularly for updates and store the lists in the local storage or something like that ?

Vlad

9rqnq I think this thread raises a question about somewhat different topic, a mechanism to report a new site as spam.

We could use one of the existing malware site lists, although I think known spam sites already on those lists appear rarely in Kagi to begin with.

dmvscam

I'm a new kagi user. Today I searched for "dmv opt out" and you can see the results below. Unthinkingly I clicked through the first result and submitted my information. A few minutes later, in horror, I realized this is a scam website. The FTC won a case against them in 2022, I don't know why the site is still up, but it certainly shouldn't be the top result on kagi. Now I expect to receive a flood of scam mailings and calls to my personal address and phone number 🙁 Please delist dmv.com, and all affiliated OnPoint scam websites, from your index.

FTC case: https://www.ftc.gov/news-events/news/press-releases/2022/04/ftc-win-at-trial-against-on-point-global-makes-102-million-in-refunds-for-consumers

OnPoint Global LLC list of scam websites: https://www.dmv.com/sites-list

Vlad

dmvscam What is the list where dmv.com is marked as spam?

Pat

Occasionally I stumble upon a malicious search result. An example is the third result of this search:

It's one of those sites that redirects you several times until you land on a, "You've been selected as a winner!" sorts of scams.

It would be nice if there was a built in way to help others by reporting malicious sites

Vlad

Pat Any idea how would we do this inside the crystal orb popup? (it is a bit cramped already) Also what is the expectation what we would with these sites to prevent potential abuse? Is the "Block" feature not enough?

Too many questions but things are never simple.

Pat

Vlad An idea I had - please forgive my poor design skills

Notice small report link beneath website ranking adjustment:

This would change the crystal ball (I'm not so sure if the large title bar makes sense here):

I suppose different reasons could be:

Phishing
Malware
Scam
Illegal content

The heuristic to actually hide results after reports is out of my wheelhouse, but I might suggest looking at uBlock's ban list in the meantime as it originally caught the malicious link in the OP.

Another idea is not hiding results but displaying a warning after some threshold.