1) Find a bunch of high-quality block lists on the internet which have been painstakingly curated my their maintainers for many years
2) Combine it all into one big list. Tell everyone that you will quickly whitelist any domains if they are causing breakage.
3) Once enough people start using your list, get an advertiser to pay you to silently remove their domains. If anyone notices, just say it was to fix breakage on some obscure site.
I’m not saying that Energized or StevenBlack are doing step 3, but please realize that there are issues with using lists like these. Even if they aren’t getting paid, they might still have some undesirable whitelisted domains. They also deprive the original block list maintainers of views (meaning they might be less inclined to continue maintaining them). You also won’t receive updates from the lists as quickly because of the middle man.
If you are using Pi—hole, OPNsense, or any other tool which can run multiple block lists simultaneously, I recommend taking a look at https://firebog.net for a list of original-source block lists.
Consolidated block lists like yours are still important for people who are using the traditional etc/hosts file. I believe the Pihole project would like to focus more on the software and less on the block lists - so they include your list, which has a good track record of vetting sources and responding to issues. I also appreciate that your project has produced its own original block lists (which happen to be included on the Energized list and firebog.net).
I just wish more people would use the original source when possible.
I dislike this Energized list, partly because I had a bad experience using one of their non-primary lists which wasn’t well maintained, and partly because their website (https://energized.pro/) makes it sound like a commercial product.
(I'm wasn't implying the EnergizedProtection is sketchy...)
The essential problem is, without active curation, source lists can't quickly react to emerging threats.
Without active curation, source lists become add-only buckets where domains land, and are rarely removed, long after they are abandoned.
The soft option is to simply cumulate domains. We've seen candidate block lists grow from 350,000 domains to 750,000 domains. These are just endless one-way buckets. Use that as your Android phone's hosts file, or you Windows PC hosts file, and you're going to have a bad time.
At the edges, there are myriad grey areas. We can all agree that YouTube serving ads is painful. But what about Youtube retaining your viewing history, which some people find very useful and handy. Over the years we've often had discussions such as this. This is what it means to actively curate.
I'm no expert, but the most obvious answer is "you MITM yourself".
If there is a userbase for a list, they have to trust the list to not filter out domains that shouldn't be filtered. I have a hard time thinking how this could lead to hidden repercussions, other than some security flaw that is only exploitable when some subset of requests go through.
Perhaps they can point a particular host to a malicious IP rather than "0.0.0.0". In a list of several hundred thousand domains, you wouldn't be able to notice this manually.
ex., make Bank of America resolve to a phishing site rather than the real BoA IP.
Pi-Hole and others might check for this though, I don't know.
Back before security fatigue set in, I would filter the lists through my own sanity checks for this sort of thing (never found a single one amiss after years)
then point my clients at my vetted lists.
I am under the impression that the blocklist programs (such as ublock origin or pi-hole) do not have an option to redirect to anything other than the void. I can only see downsides to allowing this.
Actually ublock-origin has some options to replace Javascripts with custom (presumably less intrusive) scripts. Although I don't think 3rd party lists can do this.
Anyway the repository in this post also provides host files, which most definitely can redirect you to malicious IPs.
I'm also using NextDNS and one thing that's a huge boon for me is that the default free tier covers my use case insanely well. Given the statistics for the last 3 months I seem to consistently fly under the free tier limit but if I ever do hit it, it will just default back to a regular DNS. A very user-friendly approach and I hope they keep it as they grow.
Not gonna claim to know the situation of the folks you're replying to. And Im not gonna pretend these organizations operate for free - if you can reasonably afford it? Supporting them is a great patriotism / praxis / etc for internet denizens.
But I will say this - advertising and tracking has a long, storied history of being a malware infection route. The great boon for all of us from a free-tier DNS-filter service is the additional layer of virus and information protection.
Protecting each other, even inexperienced, or low-budget users? Is the best thing we can do to slow the propagation of malware and institutional information leeches. This in turn protects even the servers of potentially ill-informed or budget-constricted server admins.
We are dealing with internet epidemiology. Free-tier "covid masks" / DNS filters preserve more health than simply for the users actively participating. We have to be in this together, or we will watch each other sink.
Thank you for your time, and sorry for the longwinded Lefty- "Dwight Schrute"-ing. But this entire disclaimer felt necessary to me.
Check your Nextdns.io logs and see what gets blocked. Then add the blocked domain (or the whole root domain) to the Whitelist. The whitelist always overrides any blocklist entry.
On Apple devices it’s as easy as toggling a switch to temporarily disable it. Not perfect but easy enough. Still trying to figure out a similar solution on Linux...
The issue for Cloudflare is whitelisting the DDoS protection script included on each page. It's under a lot of URLs so it would have to be content-based.
Looks like my lists are intended to be included, but it was linking to the raw Github source instead of the hosted Github pages version. I went through a major refactor 21 days ago that moved my sources lists around a bit - but preserved the links that are supplied all over the README and the Github hosted pages. So, not only is the project linking to the wrong place, but my list has been broken in it for 21 days now without notice.
Its fine that people love creating these massive all-in-one lists. But I recommend just using the sources directly. That way, if a list gives you trouble, you know who to open a ticket with, or just disable that specific list if its too aggressive for your tastes.
I use this with my Pi-Hole. Works very well. Along with a few other lists the Pi-Hole blocks about 30% of requests with almost no changes on the user end.
Fundamentally D.N.S is a naming system but each site has a separate naming system via user names.
Something like this should also be applicable for social networks as well. I found this for twitter - https://blocktogether.org/ not sure if it is possible for others like facebook.
Is there any tools out there that I can use to generate my own aggregated lists from a set of other blocklists?
Ideally it leverages things like GitHub Actions (or another CI tool) + GH Pages/GH releases/Netlify to relief the burden of having to host it myself.
The reason for this is so that I can use NetGuard, which allows for only 1 blocklist. Currently I'm flipping between Blokada and DNS66 because they allow for multiple lists.
I've been using blockslists from a couple github repositories for a long time, heck probably since they were found on regular web pages.
They work pretty good, but can be a little cumbersome to turn off or to enable certain domains from time to time (such as when a site has so many ads it breaks the site). But the increases safety and speed while surfing is well worth it.
1) Find a bunch of high-quality block lists on the internet which have been painstakingly curated my their maintainers for many years
2) Combine it all into one big list. Tell everyone that you will quickly whitelist any domains if they are causing breakage.
3) Once enough people start using your list, get an advertiser to pay you to silently remove their domains. If anyone notices, just say it was to fix breakage on some obscure site.
I’m not saying that Energized or StevenBlack are doing step 3, but please realize that there are issues with using lists like these. Even if they aren’t getting paid, they might still have some undesirable whitelisted domains. They also deprive the original block list maintainers of views (meaning they might be less inclined to continue maintaining them). You also won’t receive updates from the lists as quickly because of the middle man.
If you are using Pi—hole, OPNsense, or any other tool which can run multiple block lists simultaneously, I recommend taking a look at https://firebog.net for a list of original-source block lists.