Add uBlacklist support, advanced blocking rules, regex matching...

pessoa

uBlacklist is a browser extension that can blocks specific sites from appearing in search results of some search engine. In Kagi search, we are able to manually adjust website ranking to block certain website. But this is inefficient considering sometimes I have to block almost 80% of results in the first page.

I hope I can import those up-to-date community-driven search results blacklists, but the develop of uBlacklist has little motivation to add support for other search engines. Would you add implementing Kagi search support for ublacklist to your roadmap? It would be better if "Personalized Results" support subscribing to uBlacklist blacklists.

BattleHawk

I think so, too.

BattleHawk

There is a lot of garbage in the search results, so I want you to be able to import uBlacklist filters.

LKD70

It looks like uBlackList uses newline seporated regex matches, maybe this can be added as an input method for the bulk domain block under personalized results?
As for now I think it only accepts domain formats, not regex formats.

Vlad

Can someone show actual difference between ublacklist format and what we support to know what we need to add?

MIB

Vlad
I don't think that kagi currently supports (maybe I'm wrong) blocking domains by Match pattern or regular expressions which ublacklist use for blocking syntax. I think kagi expects plain domains for blocking/raising.

For example, if I want to block all Pinterest (they have a lot of domain extensions) I can use /.pinterest.\../ with ublacklist and I don't need to manually add all Pinterest domains.

for ordinary domain block, it can be done using *://*.example.com/* with subdomains included or *://example.com/*without them

I maintain a good list and would like to transfer it to kagi. I think Regex is standard and is worth be added to kagi anyway.

basix

It seems like this should be done from uBlacklist's side, no? The maintainer may not want to implement support for Kagi but they are willing to accept an implementation PR from community.

z64

@MIB

For example, if I want to block all Pinterest (they have a lot of domain extensions) I can use /.pinterest.../ with ublacklist and I don't need to manually add all Pinterest domains.

For reference, the way our current system works is:

If the domain you (block/lower/raise/pin) does not have any subdomains, then all subdomains from that site are affected. So if you insert a block for pinterest.com, all the rest would be blocked by default.
If the domain you (block/lower/raise/pin) does have a subdomain, the rule only affects that subdomain.

MIB

z64

I think you probably misquoted this part of my reply with the other one I talked about sub-domains. This quote I was talking about domain extensions (TLDs) not sub-domains. Something like pinterest having multiple pintrest.*** not sub-domains ***.pinterest.com that I know kagi will currently would do.

Also I think that is not a healthy default, at least there maybe some cases when you want to block(pin/raise/lower) specific sub-domain/s but not ther others. For example I might want to raise gist.github.com but not github.com or any other *.github.com.

rsuwa

@Vlad
It is becoming popular to subscribe to lists of sites to block that are published and maintained in a format for ublacklist. Below is an example of a ublacklist subscription list. Could you please support the ability to import lists published in this format?
https://iorate.github.io/ublacklist/subscriptions

sle4zy84nq

I can't use a search engine without uBlacklist anymore. I've been spoiled. There's just way too many garbage sites I don't want to see. I'd like to use Kagi, but I can't until a uBlacklist-like feature exists.

If you decide to support a uBlacklist-like feature, please consider this scenario:
I block a lot of sites; over 100,000. Some of my search results yield 0 results on the first page. Without a userscript, I have to browse to the second or even third page to see any results. The first few pages are filled with garbage I've filtered out. Loading more results automatically with a userscript works, but I'm consistently flagged as a bot and have to complete captcha after captcha.

If I could import my uBlacklist lists and non-filtered results were nicely compiled on the first page, I'd start using Kagi.

Vlad

sle4zy84nq Can you give a few examples of how Kagi results could be better if you had the block list available? In general we are told that the results are pretty good out of the box.

sle4zy84nq

It's not about being "pretty good", and that's subjective anyway. It's about filtering out what you don't want to see. What if I don't want to see results from twitter.com? Some might consider this a good result, others never want to see it.

I search all day while troubleshooting. There are countless AI-generated trash sites. Before uBlacklist, I had to scroll through all these trash sites. Now, if I think a website is garbage, it goes on the blacklist; never to be seen again 🙂

It's so nice to click one button and to never see that domain in search results again. I update a master list with block rules. All my pc's and even my phone update from this list. It's so nice.

Vlad

sle4zy84nq I guess I did not understand this:

I can't use a search engine without uBlacklist anymore

To my knowledge no search engine supports uBlacklist directly. The closest that exists is Kagi's blocked domains.

What prevents you from using uBlacklist on Kagi, same as on other search engines?

sle4zy84nq

Vlad
From the OP:

...the develop of uBlacklist has little motivation to add support for other search engines.

Vlad

sle4zy84nq OK I understand the problem is uBlacklist not supporting Kagi. No search engine supports uBlacklist and the ask is for us to do so.

We can not expand our backend now to support something like 100,000 long blacklist. The cost of processing this per for every query would be non-trivial as well as there would be a latency penalty.

I understand uBlackist extension is open-source - would you be open to submitting a patch to support Kagi's front-end?

BattleHawk

Vlad I don't want the backend to handle the blacklist. As a client-side function, it is enough to be able to read the filter of uBklacklist, check for updates, reflect this, and manage the filter.

Moreover, as a condition to implement this function, wouldn't the contents of the filter be used for machine learning in the garbage domain, which will lead to kagi's enhancement?

User evaluation can strengthen the sales talk of not displaying garbage in search results.
It doesn't matter if you provide the whole thing. -> https://github.com/Chamiu/Search-Block-Parasiticide

The current implementation of search results that are known to be garbage must be re-evaluated one by one is troublesome.

KagiForMe

GitHub Issue: https://github.com/iorate/ublacklist/issues/223