britannic / blacklist Goto Github PK

Blacklist and Adware Blocking for the Ubiquiti EdgeMax Router

License: Other

Go 88.33% Shell 7.91% Makefile 3.75%

edgeos-dnsmasq-blacklist paypal-money-transfer dnsmasq blacklist paypal edgemax blacklist-configuration blacklist-sources sponsorship routers

blacklist's People

Contributors

Stargazers

Watchers

blacklist's Issues

Commit failed upon install

When I install this on my ER-X SFP I am getting this at the end of the install:

post-install: INFO[078]20:34:30.295: set service dns forwarding blacklist hosts source sysctl.org url http://sysctl.org/cameleon/hosts
post-install: INFO[079]20:34:30.511: set system task-scheduler task update_blacklists executable path /config/scripts/update-dnsmasq-cronjob.sh
post-install: INFO[07a]20:34:30.688: set system task-scheduler task update_blacklists executable arguments 10800
post-install: INFO[07b]20:34:30.921: set system task-scheduler task update_blacklists interval 1d
At least one interface must be configured for DNS forwarding parameter 'listen-on' or 'except-interface' but not both

[ system task-scheduler ]
Restarting periodic command scheduler: cronStopping periodic command scheduler: cron.
Starting periodic command scheduler: cron.

Commit failed
post-install: ERRO[07c]20:34:33.328: commit failed!
Saving configuration to '/config/config.boot'...
Done
post-install: INFO[07d]20:34:34.709: save
post-install: INFO[07e]20:34:34.896: end
post-install: INFO[07f]20:34:35.069: chgrp -R vyattacfg /opt/vyatta/config

This makes the install fail.

I get this message either way I install it.
Through apt-get or through dpkg

Chromecast

I can't seem to get this working on Chromecast. I have created a custom list. It works on desktop but doesn't seem to be reflected on Chromecast.

apt fails to download sources

I followed the setup instructions on the github page and am seeing the deb repo 404:

$ sudo apt-get update && sudo apt-get install edgeos-dnsmasq-blacklist
Ign https://raw.githubusercontent.com britannic Release.gpg
Ign https://raw.githubusercontent.com britannic Release
Err https://raw.githubusercontent.com britannic/main mips Packages
The requested URL returned error: 404
Ign https://raw.githubusercontent.com britannic/main Translation-en
W: Failed to fetch https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/dists/britannic/main/binary-mips/Packages The requested URL returned error: 404

E: Some index files failed to download. They have been ignored, or old ones used instead.
admin@Router:~$ sudo apt-get update && sudo apt-get install edgeos-dnsmasq-blacklist
Ign https://raw.githubusercontent.com britannic Release.gpg
Ign https://raw.githubusercontent.com britannic Release
Err https://raw.githubusercontent.com britannic/main mips Packages
The requested URL returned error: 404
Ign https://raw.githubusercontent.com britannic/main Translation-en
W: Failed to fetch https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/dists/britannic/main/binary-mips/Packages The requested URL returned error: 404

E: Some index files failed to download. They have been ignored, or old ones used instead.
I went back to downloading and installing the deb file manually but there may be an issue with either documentation or the published repo file.

I went back to downloading and installing the deb file manually but there may be an issue with either documentation or the published repo file.

See corresponding UBNT forum post

Debian repo is inaccessible

When fetching the debian repo, I get a 400 error.

githubSteveBlack host file instructions, double quote error on ER4 v2.0.9-hotfix.6.

In the instructions to add back the githubSteveBlack host source, I get an error on an ER4 v2.0.9-hotfix.6.

Found in the README.md.

https://github.com/britannic/blacklist#release-1248-april-9-2023

In the configuration menu, one command fails.

# set service dns forwarding blacklist hosts source githubSteveBlack description '"Blacklists adware and malware websites"'
Cannot use the double quote (") character in a value string
Value validation failed
Set failed

Just omitting the double quote characters, the command is successful.

# set service dns forwarding blacklist hosts source githubSteveBlack description 'Blacklists adware and malware websites'
[edit]

Thank you for the software, slightly smoother life.(results may vary)

Hi,
I have installed blacklists on my router and I am not clear about two aspects:
how can I check if the service is active and working?
I saw the advertising on YouTube continues to be there (less frequent than before) how can I check?

Many Thanks

GPG signing key fails to install

Console message is as follows:
Is the certificate out of date?
ubnt@ubnt:~$ sudo curl -L https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public.key | sudo apt-key add -
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: http://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
of Certificate Authority (CA) public keys (CA certs). If the default
bundle file isn't adequate, you can specify an alternate file
using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
the bundle, the certificate verification probably failed due to a
problem with the certificate (it might be expired, or the name might
not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
the -k (or --insecure) option.

Minor suggestion: emphasise difference between gateway and controller for "make permanent" instructions

Thanks very much for this: initial installation was a breeze, and I'm really looking forward to seeing how well it works over the next while.

One minor suggestion I have is with regard to the "How can I keep my USG configuration" FAQ entry:

https://github.com/britannic/blacklist#how-do-can-keep-my-usg-configuration-after-an-upgrade-provision-or-reboot

.. I think it might help those coming to this for the first time if you add a bullet point noting that the config.gateway.json file is to be obtained from the gateway, but stored on the controller. This is of course what the Ubiquiti instructions say, but it's an easy detail to miss if you're just skim-reading, since all of the other work needed to install is just done on the gateway.

Server Certificate Verification Failed

Just updated to v1.10.10 of the ER firmware and tried to install adblocking via apt-get method but I'm getting the following:

Ign https://raw.githubusercontent.com wheezy Release.gpg
Ign https://raw.githubusercontent.com wheezy Release
Err https://raw.githubusercontent.com wheezy/main mips Packages
server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
Ign https://raw.githubusercontent.com wheezy/main Translation-en
W: Failed to fetch https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/dists/wheezy/main/binary-mips/Packages server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none

E: Some index files failed to download. They have been ignored, or old ones used instead.

I was able to manually download ind install the .deb file, but only by passing a -k argument to the curl download, then letting dpkg do its thing.
I haven't really tested anything else at this point. Just informing of the issues...

Thanks for all the hard work!

Compatibility Typo

EdgeMAX versions: v1.9.7+hotfix.4-v2.0.9-hotfix.2

replace hotfix.2 with hotfix.6

Things still blocked even after uninstall

First I tried the cli commands to disable and then a complete uninstall.
Cleared browser cache, tried different browsers on different computers.
No Ad block extensions are installed.

But, Ads are still being blocked. I even restarted my router.

I needed to do some testing but I can't do that if I'm unable to disable it.

ERL-3
Linux ubnt 3.10.107-UBNT #1 SMP Mon Oct 1 11:07:12 UTC 2018 mips64 GNU/Linux

Thanks !

Cannot allocate memory when using a large number of hosts

First of all, thank you for creating this application. It is really awesome to be able to use blocklists without any extra hardware.

I have recently been having issues with an USG. The application runs out of memory before completion.
I was using v1.2.4.2, and I tried upgrading to v1.2.4.5, but it didn't help. I tried rebooting and re-running the update task, but it ran out of memory again.

I don't know much about Go development, but I tried to do some debugging.

I was using the following command in each test case: time /config/scripts/update-dnsmasq -v
And while the script was running, I was watching memory usage in another session: watch free -h

RAM before running the script (in each test case)

             total       used       free     shared    buffers     cached
Mem:          483M       305M       177M         0B        24M       115M
-/+ buffers/cache:       165M       317M
Swap:           0B         0B         0B

Test case for v1.2.4.5

NOTI[02f]22:05:01.208: Total entries found: 418129
NOTI[030]22:05:01.209: Total entries extracted 362205
NOTI[031]22:05:01.210: Total entries dropped 55924
ERRO[032]22:05:01.213: ReloadDNS(): error: fork/exec /bin/bash: cannot allocate memory

The least amount of free memory after Total entries dropped, right before the error:

             total       used       free     shared    buffers     cached
Mem:          483M       437M        46M         0B        22M       108M
-/+ buffers/cache:       305M       178M
Swap:           0B         0B         0B

Time:

real	1m45.933s
user	1m55.940s
sys	0m19.680s

Test case: GC 20%

I've added 2 lines in main.go in the latest master branch:

   6   │     "runtime/debug"
...
 113   │     debug.SetGCPercent(20)

I've built with make mips and SCP'd the executable to the USG.

It took a longer time to finish, but it never ran out of memory this way.

Sample run with GC 20%:

NOTI[02f]21:53:22.825: Total entries found: 418129
NOTI[030]21:53:22.852: Total entries extracted 362209
NOTI[031]21:53:22.854: Total entries dropped 55920

The least amount of free memory after Total entries dropped:

             total       used       free     shared    buffers     cached
Mem:          483M       429M        54M         0B        24M       114M
-/+ buffers/cache:       290M       192M
Swap:           0B         0B         0B

Memory freed by GC:

             total       used       free     shared    buffers     cached
Mem:          483M       331M       152M         0B        24M       114M
-/+ buffers/cache:       193M       290M
Swap:           0B         0B         0B

Memory before completion:

             total       used       free     shared    buffers     cached
Mem:          483M       377M       106M         0B        24M       114M
-/+ buffers/cache:       238M       245M
Swap:           0B         0B         0B

Time:

real	3m20.654s
user	3m39.210s
sys	0m31.110s

Remove ads from youtube app on TV

Hi ,

Thank you for your wonderful tool , It has really helped me to get rid of annoying ads in my PCs and other devices.
I do wonder if there is a way to get rid of ADs from youtube TV ap .

Regards

Add support for ERPro-8 / ER-8

It would be nice to have support for ERPro-8 / ER-8

Adding a Host or Domain cause ad blocking stop working

Every time I try to use this command "set service dns forwarding blacklist include ****" or " set service dns forwarding blacklist domains include *****" to blacklist a website that have p0rn ads. The adblocking stop working when I reset the device, not sure what is wrong so I had to reinstall the .deb package to make it working again. I tried even hard reset the device and configure it from scratch same problem
My device is a ER-X with 1.10.8

No internet access, aborting blacklist update!

I installed this thing today and it doesn't want to run.

It says:

NOTI[001]17:35:01.819: Starting blacklist update...
CRIT[002]17:35:04.821: No internet access, aborting blacklist update!

Commit failed

I checked source code and seems like it checks wherever https://www.google.com is available. And it does indeed available from my router. Here's curl output:

$ curl -vvv https://www.google.com
* Rebuilt URL to: https://www.google.com/
*   Trying 172.217.168.228...
* TCP_NODELAY set
* Connected to www.google.com (172.217.168.228) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-ECDSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=Mountain View; O=Google LLC; CN=www.google.com
*  start date: Jul  7 08:10:21 2020 GMT
*  expire date: Sep 29 08:10:21 2020 GMT
*  subjectAltName: host "www.google.com" matched cert's "www.google.com"
*  issuer: C=US; O=Google Trust Services; CN=GTS CA 1O1
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x56802eb8)
> GET / HTTP/1.1
> Host: www.google.com
> User-Agent: curl/7.52.1
> Accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 200
< date: Sun, 26 Jul 2020 14:38:48 GMT
< expires: -1
< cache-control: private, max-age=0
< content-type: text/html; charset=ISO-8859-1
< p3p: CP="This is not a P3P policy! See g.co/p3phelp for more info."
< server: gws
< x-xss-protection: 0
< x-frame-options: SAMEORIGIN
< set-cookie: 1P_JAR=2020-07-26-14; expires=Tue, 25-Aug-2020 14:38:48 GMT; path=/; domain=.google.com; Secure
< set-cookie: NID=204=JMPs6P7nmTwzfIVDxOsEAPYoaoUjZTS_AdTtUrpogsAv124z-vxOF3xYn-eQzN-oGgiNxuywR67iYoxubXSHqYmvVBuueNjsX07ZG5qSR4cqoUvGrap8B4-VMxPrfmbcX29U7IvQfvlZX6yNW4B2DkFHwTk5KncpXn7ObOitYb0; expires=Mon, 25-Jan-2021 14:38:48 GMT; path=/; domain=.google.com; HttpOnly
< alt-svc: h3-29=":443"; ma=2592000,h3-27=":443"; ma=2592000,h3-T050=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
< accept-ranges: none
< vary: Accept-Encoding
<
<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="uk"><head><meta content="text/html; charset=UTF-8" http-equiv="Content-Type"><meta content="/images/branding/googleg/1x/googleg_standard_color_128dp.png" itemprop="image"><title>Google</title><script nonce="/lWi4QVFfvu11uHY7tp53Q==">(function(){window.google={kEI:'eJUdX6vZEsuBkwW77674DA',kEXPI:'0,18167,183956,3,4,32,1151585,5662,731,223,5104,207,3204,10,1226,364,1499,611,206,383,246,5,304,1050,351,297,1046,1936,470,294,20,3,1051,90,193,117,72,84,49,412,112,135,174,272,3,1120585,1197717,303245,26305,13677,4855,32692,15247,867,19397,9287,9188,8384,4858,1362,284,9006,3023,4740,5,11028,2,1811,4998,7931,5297,2054,920,873,1217,9405,14527,4518,2777,919,2277,8,2796,1593,1279,2212,530,149,1103,840,517,1522,4258,312,1137,2,2063,606,2025,544,1231,520,1947,2229,93,328,1284,16,2927,2246,1820,1780,3227,1990,855,9,6066,6286,4455,641,2450,2458,1226,1742,4929,108,3407,908,2,941,2614,2397,1027,6443,218,10,612,1337,1098,3,346,230,970,865,373,3546,705,149,189,3313,2488,2252,1992,1997,83,1010,651,4,1528,17,1277,1010,1242,1,264,874,405,42,763,4,650,381,20,177,2310,1697,9,43,975,441,1425,459,116,759,468,212,3,599,3189,568,1973,86,3,1493,1787,1418,3,5,69,2615,2811,939,2,812,690,1968,2,243,627,509,2525,148,291,218,194,4,24,888,277,212,4,71,299,390,431,30,463,35,410,134,589,123,34,275,2,334,1479,113,2,151,504,205,2,215,63,174,44,7,318,566,845,650,221,29,142,308,219,589,8,650,997,763,1258,665,4,2,237,125,5784360,8801916,549,333,444,1,2,80,1,900,896,1,9,2,2551,1,748,141,59,736,563,1,4265,1,1,2,1017,9,305,3299,129,2,20,65,7,25,64,33,3502099,20458661',kBL:'aaAI'};google.sn='webhp';google.kHL='uk';})();(function(){google.lc=[];google.li=0;google.getEI=function(a){for(var c;a&&(!a.getAttribute||!(c=a.getAttribute("eid")));)a=a.parentNode;return c||google.kEI};google.getLEI=function(a){for(var c=null;a&&(!a.getAttribute||!(c=a.getAttribute("leid")));)a=a.parentNode;return c};google.ml=function(){return null};google.time=function(){return Date.now()};google.log=function(a,c,b,d,g){if(b=google.logUrl(a,c,b,d,g)){a=new Image;var e=google.lc,f=google.li;e[f]=a;a.onerror=a.onload=a.onabort=function(){delete e[f]};google.vel&&google.vel.lu&&google.vel.lu(b);a.src=b;google.li=f+1}};google.logUrl=function(a,c,b,d,g){var e="",f=google.ls||"";b||-1!=c.search("&ei=")||(e="&ei="+google.getEI(d),-1==c.search("&lei=")&&(d=google.getLEI(d))&&(e+="&lei="+d));d="";!b&&google.cshid&&-1==c.search("&cshid=")&&"slh"!=a&&(d="&cshid="+google.cshid);b=b||"/"+(g||"gen_204")+"?atyp=i&ct="+a+"&cad="+c+e+f+"&zx="+google.time()+d;/^http:/i.test(b)&&"https:"==window.location.protocol&&(google.ml(Error("a"),!1,{src:b,glmm:1}),b="");return b};}).call(this);(function(){google.y={};google.x=function(a,b){if(a)var c=a.id;else{do c=Math.random();while(google.y[c])}google.y[c]=[a,b];return!1};google.lm=[];google.plm=function(a){google.lm.push.apply(google.lm,a)};google.lq=[];google.load=function(a,b,c){google.lq.push([[a],b,c])};google.loadAll=function(a,b){google.lq.push([a,b])};}).call(this);google.f={};(function(){
document.documentElement.addEventListener("submit",function(b){var a;if(a=b.target){var c=a.getAttribute("data-submitfalse");a="1"==c||"q"==c&&!a.elements.q.value?!0:!1}else a=!1;a&&(b.preventDefault(),b.stopPropagation())},!0);document.documentElement.addEventListener("click",function(b){var a;a:{for(a=b.target;a&&a!=document.documentElement;a=a.parentElement)if("A"==a.tagName){a="1"==a.getAttribute("data-nohref");break a}a=!1}a&&b.preventDefault()},!0);}).call(this);
var a=window.location,b=a.href.indexOf("#");if(0<=b){var c=a.href.substring(b+1);/(^|&)q=/.test(c)&&-1==c.indexOf("#")&&a.replace("/search?"+c.replace(/(^|&)fp=[^&]*/g,"")+"&cad=h")};</script><style>#gbar,#guser{font-size:13px;padding-top:1px !important;}#gbar{height:22px}#guser{padding-bottom:7px !important;text-align:right}.gbh,.gbd{border-top:1px solid #c9d7f1;font-size:1px}.gbh{height:0;position:absolute;top:24px;width:100%}@media all{.gb1{height:22px;margin-right:.5em;vertical-align:top}#gbar{float:left}}a.gb1,a.gb4{text-decoration:underline !important}a.gb1,a.gb4{color:#00c !important}.gbi .gb4{color:#dd8e27 !important}.gbf .gb4{color:#900 !important}
</style><style>body,td,a,p,.h{font-family:arial,sans-serif}body{margin:0;overflow-y:scroll}#gog{padding:3px 8px 0}td{line-height:.8em}.gac_m td{line-height:17px}form{margin-bottom:20px}.h{color:#36c}.q{color:#00c}em{font-weight:bold;font-style:normal}.lst{height:25px;width:496px}.gsfi,.lst{font:18px arial,sans-serif}.gsfs{font:17px arial,sans-serif}.ds{display:inline-box;display:inline-block;margin:3px 0 4px;margin-left:4px}input{font-family:inherit}body{background:#fff;color:#000}a{color:#11c;text-decoration:none}a:hover,a:active{text-decoration:underline}.fl a{color:#36c}a:visited{color:#551a8b}.sblc{padding-top:5px}.sblc a{display:block;margin:2px 0;margin-left:13px;font-size:11px}.lsbb{background:#eee;border:solid 1px;border-color:#ccc #999 #999 #ccc;height:30px}.lsbb{display:block}#fll a{display:inline-block;margin:0 12px}.lsb{background:url(/images/nav_logo229.png) 0 -261px repeat-x;border:none;color:#000;cursor:pointer;height:30px;margin:0;outline:0;font:15px arial,sans-serif;vertical-align:top}.lsb:active{background:#ccc}.lst:focus{outline:none}.tiah{width:458px}</style><script nonce="/lWi4QVFfvu11uHY7tp53Q=="></script></head><body bgcolor="#fff"><script nonce="/lWi4QVFfvu11uHY7tp53Q==">(function(){var src='/images/nav_logo229.png';var iesg=false;document.body.onload = function(){window.n && window.n();if (document.images){new Image().src=src;}
if (!iesg){document.f&&document.f.q.focus();document.gbqf&&document.gbqf.q.focus();}
}
})();</script><div id="mngb"><div id=gbar><nobr><b class=gb1>&#1055;&#1086;&#1096;&#1091;&#1082;</b> <a class=gb1 href="https://www.google.com.ua/imghp?hl=uk&tab=wi">&#1047;&#1086;&#1073;&#1088;&#1072;&#1078;&#1077;&#1085;&#1085;&#1103;</a> <a class=gb1 href="https://maps.google.com.ua/maps?hl=uk&tab=wl">&#1050;&#1072;&#1088;&#1090;&#1080;</a> <a class=gb1 href="https://www.youtube.com/?gl=UA&tab=w1">YouTube</a> <a class=gb1 href="https://news.google.com.ua/nwshp?hl=uk&tab=wn">&#1053;&#1086;&#1074;&#1080;&#1085;&#1080;</a> <a class=gb1 href="https://mail.google.com/mail/?tab=wm">Gmail</a> <a class=gb1 href="https://drive.google.com/?tab=wo">&#1044;&#1080;&#1089;&#1082;</a> <a class=gb1 href="https://www.google.com/calendar?tab=wc">&#1050;&#1072;&#1083;&#1077;&#1085;&#1076;&#1072;&#1088;</a> <a class=gb1 style="text-decoration:none" href="https://www.google.com.ua/intl/uk/about/products?tab=wh"><u>&#1041;&#1110;&#1083;&#1100;&#1096;&#1077;</u> &raquo;</a></nobr></div><div id=guser width=100%><nobr><span id=gbn class=gbi></span><span id=gbf class=gbf></span><span id=gbe></span><a href="http://www.google.com.ua/history/optout?hl=uk" class=gb4>&#1030;&#1089;&#1090;&#1086;&#1088;&#1110;&#1103; &#1074;&#1077;&#1073;-&#1087;&#1086;&#1096;&#1091;&#1082;&#1091;</a> | <a  href="/preferences?hl=uk" class=gb4>&#1053;&#1072;&#1083;&#1072;&#1096;&#1090;&#1091;&#1074;&#1072;&#1085;&#1085;&#1103;</a> | <a target=_top id=gb_70 href="https://accounts.google.com/ServiceLogin?hl=uk&passive=true&continue=https://www.google.com/" class=gb4>&#1059;&#1074;&#1110;&#1081;&#1090;&#1080;</a></nobr></div><div class=gbh style=left:0></div><div class=gbh style=right:0></div></div><center><br clear="all" id="lgpd"><div id="lga"><img alt="Google" height="92" src="/images/branding/googlelogo/1x/googlelogo_white_background_color_272x92dp.png" style="padding:28px 0 14px" width="272" id="hplogo"><br><br></div><form action="/search" name="f"><table cellpadding="0" cellspacing="0"><tr valign="top"><td width="25%">&nbsp;</td><td align="center" nowrap=""><input name="ie" value="ISO-8859-1" type="hidden"><input value="uk" name="hl" type="hidden"><input name="source" type="hidden" value="hp"><input name="biw" type="hidden"><input name="bih" type="hidden"><div class="ds" style="height:32px;margin:4px 0"><div style="position:relative;zoom:1"><input class="lst tiah" style="margin:0;padding:5px 8px 0 6px;vertical-align:top;color:#000;padding-right:38px" autocomplete="off" value="" title="&#1055;&#1086;&#1096;&#1091;&#1082; Google" maxlength="2048" name="q" size="57"><img src="/textinputassistant/tia.png" style="position:absolute;cursor:pointer;right:5px;top:4px;z-index:300" data-script-url="/textinputassistant/11/uk_tia.js" id="tsuid1" alt="" height="23" width="27"><script nonce="/lWi4QVFfvu11uHY7tp53Q==">(function(){var id='tsuid1';document.getElementById(id).onclick = function(){var s = document.createElement('script');s.src = this.getAttribute('data-script-url');(document.getElementById('xjsc')||document.body).appendChild(s);};})();</script></div></div><br style="line-height:0"><span class="ds"><span class="lsbb"><input class="lsb" value="&#1055;&#1086;&#1096;&#1091;&#1082; Google" name="btnG" type="submit"></span></span><span class="ds"><span class="lsbb"><input class="lsb" id="tsuid2" value="&#1052;&#1077;&#1085;&#1110; &#1087;&#1086;&#1097;&#1072;&#1089;&#1090;&#1080;&#1090;&#1100;" name="btnI" type="submit"><script nonce="/lWi4QVFfvu11uHY7tp53Q==">(function(){var id='tsuid2';document.getElementById(id).onclick = function(){if (this.form.q.value){this.checked = 1;if (this.form.iflsig)this.form.iflsig.disabled = false;}
else top.location='/doodles/';};})();</script><input value="AINFCbYAAAAAXx2jiCeJzAMo6jeBSHs8LueZBBy3otrf" name="iflsig" type="hidden"></span></span></td><td class="fl sblc" align="left" nowrap="" width="25%"><a href="/advanced_search?hl=uk&amp;authuser=0">&#1056;&#1086;&#1079;&#1096;&#1080;&#1088;&#1077;&#1085;&#1080;&#1081; &#1087;&#1086;&#1096;&#1091;&#1082;</a></td></tr></table><input id="gbv" name="gbv" type="hidden" value="1"><script nonce="/lWi4QVFfvu11uHY7tp53Q==">(function(){var a,b="1";if(document&&document.getElementById)if("undefined"!=typeof XMLHttpRequest)b="2";else if("undefined"!=typeof ActiveXObject){var c,d,e=["MSXML2.XMLHTTP.6.0","MSXML2.XMLHTTP.3.0","MSXML2.XMLHTTP","Microsoft.XMLHTTP"];for(c=0;d=e[c++];)try{new ActiveXObject(d),b="2"}catch(h){}}a=b;if("2"==a&&-1==location.search.indexOf("&gbv=2")){var f=google.gbvu,g=document.getElementById("gbv");g&&(g.value=a);f&&window.setTimeout(function(){location.href=f},0)};}).call(this);</script></form><div id="gac_scont"></div><div style="font-size:83%;min-height:3.5em"><br><div id="gws-output-pages-elements-homepage_additional_languages__als"><style>#gws-output-pages-elements-homepage_additional_languages__als{font-size:small;margin-bottom:24px}#SIvCob{display:inline-block;line-height:28px;}#SIvCob a{padding:0 3px;}.H6sW5{display:inline-block;margin:0 2px;white-space:nowrap}.z4hgWe{display:inline-block;margin:0 2px}</style><div id="SIvCob">&#1052;&#1086;&#1074;&#1072; Google:  <a href="https://www.google.com/setprefs?sig=0_IBsa2kBLszcilzfqVvFCIA9gOaQ%3D&amp;hl=ru&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwir9evPkuvqAhXLwKQKHbu3C88Q2ZgBCAU">&#1088;&#1091;&#1089;&#1089;&#1082;&#1080;&#1081;</a>  </div></div></div><span id="footer"><div style="font-size:10pt"><div style="margin:19px auto;text-align:center" id="fll"><a href="http://www.google.com.ua/intl/uk/ads/">&#1056;&#1077;&#1082;&#1083;&#1072;&#1084;&#1072;</a><a href="http://www.google.com.ua/intl/uk/services/">&#1056;&#1110;&#1096;&#1077;&#1085;&#1085;&#1103; &#1076;&#1083;&#1103; &#1073;&#1110;&#1079;&#1085;&#1077;&#1089;&#1091;</a><a href="/intl/uk/about.html">&#1059;&#1089;&#1077; &#1087;&#1088;&#1086; Google</a><a href="https://www.google.com/setprefdomain?prefdom=UA&amp;prev=https://www.google.com.ua/&amp;sig=K_eoXbg-InyBfVaGDrwwyzOxBg4XM%3D">Google.com.ua</a></div></div><p style="font-size:8pt;color:#767676">&copy; 2020</p></span></center><script nonce="/lWi4QVFfvu11uHY7tp53Q==">(function(){window.google.cdo={height:0,width:0};(function(){var a=window.innerWidth,b=window.innerHeight;if(!a||!b){var c=window.document,d="CSS1Compat"==c.compatMode?c.documentElement:c.body;a=d.clientWidth;b=d.clientHeight}a&&b&&(a!=google.cdo.width||b!=google.cdo.height)&&google.log("","","/client_204?&atyp=i&biw="+a+"&bih="+b+"&ei="+google.kEI);}).call(this);})();(function(){var u='/xjs/_/js/k\x3dxjs.hp.en.XsBchCh7bLs.O/m\x3dsb_he,d/am\x3dAE-wOQ/d\x3d1/rs\x3dACT90oGpUq2u9G5OXekOjNEpL2rNntM8Ng';
setTimeout(function(){var b=document;var a="SCRIPT";"application/xhtml+xml"===b.contentType&&(a=a.toLowerCase());a=b.createElement(a);a.src=u;google.timers&&google.timers.load&&google.tick&&google.tick("load","xjsls");document.body.appendChild(a)},0);})();(function(){window.google.xjsu='/xjs/_/js/k\x3dxjs.hp.en.XsBchCh7bLs.O/m\x3dsb_he,d/am\x3dAE-wOQ/d\x3d1/rs\x3dACT90oGpUq2u9G5OXekOjNEpL2rNntM8Ng';})();function _DumpException(e){throw e;}
function _F_installCss(c){}
(function(){google.jl={dw:false,em:[],emw:false,lls:'default',pdt:0,snet:true,uwp:true};})();(function(){var pmc='{\x22d\x22:{},\x22sb_he\x22:{\x22agen\x22:false,\x22cgen\x22:false,\x22client\x22:\x22heirloom-hp\x22,\x22dh\x22:true,\x22dhqt\x22:true,\x22ds\x22:\x22\x22,\x22ffql\x22:\x22en\x22,\x22fl\x22:true,\x22host\x22:\x22google.com\x22,\x22isbh\x22:28,\x22jsonp\x22:true,\x22msgs\x22:{\x22cibl\x22:\x22&#1054;&#1095;&#1080;&#1089;&#1090;&#1080;&#1090;&#1080; &#1087;&#1086;&#1096;&#1091;&#1082;&#1086;&#1074;&#1080;&#1081; &#1079;&#1072;&#1087;&#1080;&#1090;\x22,\x22dym\x22:\x22&#1052;&#1086;&#1078;&#1083;&#1080;&#1074;&#1086;, &#1074;&#1080; &#1084;&#1072;&#1083;&#1080; &#1085;&#1072; &#1091;&#1074;&#1072;&#1079;&#1110;:\x22,\x22lcky\x22:\x22&#1052;&#1077;&#1085;&#1110; &#1087;&#1086;&#1097;&#1072;&#1089;&#1090;&#1080;&#1090;&#1100;\x22,\x22lml\x22:\x22&#1044;&#1086;&#1082;&#1083;&#1072;&#1076;&#1085;&#1110;&#1096;&#1077;\x22,\x22oskt\x22:\x22&#1058;&#1088;&#1072;&#1085;&#1089;&#1083;&#1110;&#1090;&#1077;&#1088;&#1072;&#1094;&#1110;&#1103;\x22,\x22psrc\x22:\x22&#1062;&#1077;&#1081; &#1087;&#1086;&#1096;&#1091;&#1082; &#1074;&#1080;&#1076;&#1072;&#1083;&#1077;&#1085;&#1086; &#1079; &#1074;&#1072;&#1096;&#1086;&#1111; \\u003Ca href\x3d\\\x22/history\\\x22\\u003E&#1074;&#1077;&#1073;-&#1110;&#1089;&#1090;&#1086;&#1088;&#1110;&#1111;\\u003C/a\\u003E\x22,\x22psrl\x22:\x22&#1042;&#1080;&#1076;&#1072;&#1083;&#1080;&#1090;&#1080;\x22,\x22sbit\x22:\x22&#1055;&#1086;&#1096;&#1091;&#1082; &#1079;&#1072; &#1079;&#1086;&#1073;&#1088;&#1072;&#1078;&#1077;&#1085;&#1085;&#1103;&#1084;\x22,\x22srch\x22:\x22&#1055;&#1086;&#1096;&#1091;&#1082; Google\x22},\x22ovr\x22:{},\x22pq\x22:\x22\x* Curl_http_done: called premature == 0
* Connection #0 to host www.google.com left intact
22,\x22refpd\x22:true,\x22rfs\x22:[],\x22sbpl\x22:16,\x22sbpr\x22:16,\x22scd\x22:10,\x22stok\x22:\x22wfMIUFI6m4fKX-P_cYuvyDA5cg8\x22,\x22uhde\x22:false}}';google.pmc=JSON.parse(pmc);})();</script>        </body>

Re-order/number release list

https://github.com/britannic/blacklist/releases

Maybe delete v1.2.5 or manually change it to v1.2.4.65 so it flows better or at the very least move 1.2.4.8 so it appears at the top?

Can't exclude a domain - 1.1.5

Following the docs to exclude a domain.
configure set service dns forwarding blacklist domains exclude list-manage.com commit;save;exit

Domain still resolves to 127.0.0.1
Are redirects logged anywhere?
Where are excludes written to?

Possible to Include TLDs

Is it possible to include TLDs as well? I've tried creating a new domain file with some problematic TLDs and it seems that they get dropped. (They are never downloaded/created to the /etc/dnsmasq.d/ folder.) The hosts file I created and played with works without any problems.

Installation error

sudo curl -L https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public.key | sudo apt-key add -

ubnt@EdgeRouter-4:~$ sudo curl -L https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public.key
curl: (60) SSL certificate problem: self signed certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.

I'm not sure is it add successfully, so that i tried to download it manually but show below error information.

ubnt@EdgeRouter-4:~$ sudo curl -L -k https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public.key

    <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
    <html>
        <head>
            <title>Error: 404 Not Found</title>
            <style type="text/css">
              html {background-color: #eee; font-family: sans;}
              body {background-color: #fff; border: 1px solid #ddd;
                    padding: 15px; margin: 15px;}
              pre {background-color: #eee; border: 1px solid #ddd; padding: 5px;}
            </style>
        </head>
        <body>
            <h1>Error: 404 Not Found</h1>
            <p>Sorry, the requested URL <tt>&#039;https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public.key&#039;</tt>
               caused an error:</p>
            <pre>Not found: &#039;/britannic/debian-repo/master/blacklist/public.key&#039;</pre>
        </body>
    </html>

So how can I ensure "public.key" had been add successfully by the CLI you provide?

My approach is use web browser to download keyfile, so that upload by SCP then use "sudo apt-key add public.key"

ubnt@EdgeRouter-4:~$ sudo apt-key add public.key
OK

But it looks has some error msg:

<FEATURE REQUEST> dns-redirect-host

Would it be possible to have blocked requests redirected to a URL or DNS entry, such as blocked.mysite.com? This way should a user click a google sponsored link (for example) instead of the page turning white because the traffic was sinkholed, an internal web server could display a page letting them know what happened.

Typo on Changelog

Add obsolete source OSID Mobile Blocklist

I think you want to remove the "obsolete" :-)

Does it block when apps or the OS bypass the router DNS?

Does it block something like this nslookup doubleclick.net 9.9.9.9?
Does it block for IPv6 requests?

bug + github page mismatch ?

hello!

first, thank you so much for this work. it really rocks! I've sent to your paypal a 🍺 for that good work 😉

i wanted to give you a couple of heads up in care you are not aware.

i could reproduce this issue opened some time ago, i've added to that thread some information. if you want my help to fix that bug just tell me.
by https://britannic.github.io/blacklist/ it seems that there is a newer release but it does not appear in https://github.com/britannic/blacklist/releases

thanks!

`/etc/init.d/dnsmasq restart` failed when trying to whitelist

I found notion.so was blacklisted:

/etc/dnsmasq.d/hosts.openphish.blacklist.conf:address=/www.notion.so/0.0.0.0

So I followed the readme, added the following to /etc/dnsmasq.conf:

address=/www.notion.so/#

But then the following command fails:

$ sudo /etc/init.d/dnsmasq restart
[....] Restarting dnsmasq (via systemctl): dnsmasq.serviceJob for dnsmasq.service failed because the control process exited with error code.
See "systemctl status dnsmasq.service" and "journalctl -xe" for details.
 failed!

sudo journalctl -xe does not show anything.

Is this the right way to whitelist? Thank you!

ReloadDNS(): error: fork/exec /bin/bash: cannot allocate memory

commit

[ service dns forwarding blacklist ]
NOTI[001]17:59:48.086: Starting blacklist update...
INFO[002]17:59:48.090: Removing stale blacklists...
INFO[003]17:59:48.162: blacklisted-subdomains: downloaded: 15
INFO[004]17:59:48.165: blacklisted-subdomains: extracted: 15
INFO[005]17:59:48.170: blacklisted-subdomains: dropped: 0
INFO[006]17:59:48.199: blacklisted-servers: downloaded: 2
INFO[007]17:59:48.203: blacklisted-servers: extracted: 2
INFO[008]17:59:48.206: blacklisted-servers: dropped: 0
INFO[009]17:59:48.302: global-whitelisted-domains: downloaded: 99
INFO[00a]17:59:48.304: global-whitelisted-domains: extracted: 99
INFO[00b]17:59:48.306: global-whitelisted-domains: dropped: 0
INFO[00c]17:59:48.355: whitelisted-servers: downloaded: 1
INFO[00d]17:59:48.358: whitelisted-servers: extracted: 1
INFO[00e]17:59:48.361: whitelisted-servers: dropped: 0
INFO[00f]17:59:48.365: Downloading domains source zeus
INFO[010]17:59:48.375: Downloading domains source NoBitCoin
INFO[011]17:59:48.378: Downloading domains source SmartTVblacklist
INFO[012]17:59:48.380: Downloading domains source malwaredomains.com
INFO[013]17:59:48.382: Downloading domains source notrackingDomains
INFO[014]17:59:48.385: Downloading domains source simple_tracking
INFO[015]17:59:56.374: simple_tracking: downloaded: 34
INFO[016]17:59:56.405: simple_tracking: extracted: 34
INFO[017]17:59:56.427: simple_tracking: dropped: 0
INFO[018]17:59:56.652: SmartTVblacklist: downloaded: 198
INFO[019]17:59:56.688: SmartTVblacklist: extracted: 175
INFO[01a]17:59:56.740: SmartTVblacklist: dropped: 23
INFO[01b]17:59:57.454: NoBitCoin: downloaded: 689
INFO[01c]17:59:57.473: NoBitCoin: extracted: 677
INFO[01d]17:59:57.496: NoBitCoin: dropped: 12
INFO[01e]18:00:14.422: malwaredomains.com: downloaded: 26885
INFO[01f]18:00:14.441: malwaredomains.com: extracted: 26881
INFO[020]18:00:14.448: malwaredomains.com: dropped: 4
INFO[021]18:00:42.422: Downloading hosts source githubSteveBlack
INFO[022]18:00:42.424: Downloading hosts source YoutubeAdBlockList
INFO[023]18:00:42.425: Downloading hosts source notracking
INFO[024]18:00:42.426: Downloading hosts source YoutubeBlockList
INFO[025]18:00:42.429: Downloading hosts source sysctl.org
INFO[026]18:00:42.434: Downloading hosts source openphish
INFO[027]18:00:44.465: YoutubeBlockList: downloaded: 69
INFO[028]18:00:44.488: YoutubeBlockList: extracted: 69
INFO[029]18:00:44.523: YoutubeBlockList: dropped: 0
INFO[02a]18:00:49.713: openphish: downloaded: 2558
INFO[02b]18:00:49.716: openphish: extracted: 1136
INFO[02c]18:00:49.744: openphish: dropped: 1422
INFO[02d]18:01:03.362: YoutubeAdBlockList: downloaded: 8882
INFO[02e]18:01:03.376: YoutubeAdBlockList: extracted: 8848
INFO[02f]18:01:03.378: YoutubeAdBlockList: dropped: 34
INFO[030]18:01:13.825: sysctl.org: downloaded: 20567
INFO[031]18:01:13.846: sysctl.org: extracted: 17576
INFO[032]18:01:13.847: sysctl.org: dropped: 2991
INFO[033]18:01:28.178: notracking: downloaded: 39080
INFO[034]18:01:28.180: notracking: extracted: 22628
INFO[035]18:01:28.183: notracking: dropped: 16452
INFO[036]18:01:29.625: githubSteveBlack: downloaded: 42353
INFO[037]18:01:29.689: githubSteveBlack: extracted: 27137
INFO[038]18:01:29.694: githubSteveBlack: dropped: 15216
NOTI[039]18:01:32.742: Total entries found: 141432
NOTI[03a]18:01:32.745: Total entries extracted 105278
NOTI[03b]18:01:32.748: Total entries dropped 36154
ERRO[03c]18:01:32.753: ReloadDNS():
error: fork/exec /bin/bash: cannot allocate memory

Everything worked well out of the box.
I've only added a couple of extra sources.
After adding https://raw.githubusercontent.com/Perflyst/PiHoleBlocklist/master/SmartTV.txt i started getting an error while trying to commit

'The specified configuration node is not valid' when setting global domain

I went to globally include a domain and got an error.
# set service dns forwarding blacklist include zefirka.net
The specified configuration node is not valid
Set failed

If I changed the command to # set service dns forwarding blacklist include domains zefirka.net then it worked. Any ideas why it wouldn't work globally?

Commit local blacklist file without Internet access?

CLI commands:

configure
set service dns forwarding blacklist hosts source myhosts file /config/user-data/antiad.txt
commit

Logs:

NOTI[001]21:51:22.197: Starting blacklist update...
CRIT[002]21:51:25.199: No internet access, aborting blacklist update!

dlb.oisd.nl certificate issue

Hi!

I was wondering if there might be a workaround for the following error. When I run update-dnsmasq, I get the following output:

ubnt@edgerouter:~$ sudo /config/scripts/update-dnsmasq -v
NOTI[001]17:43:03.451: Starting blacklist update...
INFO[002]17:43:03.466: Checking for stale blacklists...
INFO[003]17:43:03.501: blacklisted-subdomains: downloaded: 15
INFO[004]17:43:03.503: blacklisted-subdomains: extracted: 15
INFO[005]17:43:03.504: blacklisted-subdomains: dropped: 0
INFO[006]17:43:03.519: blacklisted-servers: downloaded: 2
INFO[007]17:43:03.521: blacklisted-servers: extracted: 2
INFO[008]17:43:03.525: blacklisted-servers: dropped: 0
INFO[009]17:43:03.564: global-whitelisted-domains: downloaded: 99
INFO[00a]17:43:03.565: global-whitelisted-domains: extracted: 99
INFO[00b]17:43:03.566: global-whitelisted-domains: dropped: 0
INFO[00c]17:43:03.592: whitelisted-servers: downloaded: 1
INFO[00d]17:43:03.593: whitelisted-servers: extracted: 1
INFO[00e]17:43:03.594: whitelisted-servers: dropped: 0
INFO[010]17:43:03.598: Downloading domains source NoBitCoin
INFO[00f]17:43:03.598: Downloading domains source simple_tracking
INFO[011]17:43:03.598: Downloading domains source OISD
WARN[012]17:43:04.653: Unable to get response for https://dbl.oisd.nl/basic/
INFO[013]17:43:04.682: simple_tracking: downloaded: 34
INFO[014]17:43:04.683: simple_tracking: extracted: 34
INFO[015]17:43:04.684: simple_tracking: dropped: 0
INFO[016]17:43:04.787: NoBitCoin: downloaded: 720
INFO[017]17:43:04.788: NoBitCoin: extracted: 714
INFO[018]17:43:04.789: NoBitCoin: dropped: 6
ERRO[019]17:43:04.803: Get "https://dbl.oisd.nl/basic/": x509: certificate has expired or is not yet valid: current time 2022-12-13T17:43:04+01:00 is after 2021-09-30T14:01:15Z
NOTI[01a]17:43:04.805: Total entries found: 871
NOTI[01b]17:43:04.806: Total entries extracted 865
NOTI[01c]17:43:04.807: Total entries dropped 6
INFO[01d]17:43:07.103: Successfully restarted dnsmasq
NOTI[01e]17:43:07.105: Blacklist update completed......

It seems that something is wrong with the dbl.oisd.nl/basic list, even though navigating to that URL seems to be fine in my browser.
Is there a way to ignore the certificate issue, so I can test of it works on my network?

Problem with booting without present internet connection

EdgeRouter ERLite-3, with EdgeOS 1.10.11

In situation when booting router and internet connenction is not fully ready yet (in my case caused by not ready DNS resolver) blacklist update fails:

NOTI[001]2020-11-21 12:21:34.815: Starting blacklist update...
CRIT[002]2020-11-21 12:21:37.816: No internet access, aborting blacklist update!

This is quite normal. The problem is, that existing blacklist configuration is not saved and router is wide open.

When running update (or waiting for cron/task scheduler to do that):

root@edge:/config/scripts# ./update-dnsmasq 
Removing stale dnsmasq blacklist files, because no blacklist configuration has been detected

Still, no blackllist active. Addig -safe switch changes nothig. Manually pointing to config file:

root@edge:/config/scripts# ./update-dnsmasq -f /config/config.boot

makes all updates:

root@edge:/config/scripts# tail /var/log/update-dnsmasq.log
INFO[022]2020-11-21 12:42:59.807: openphish: extracted: 3130
INFO[023]2020-11-21 12:42:59.808: openphish: dropped: 1430
INFO[024]2020-11-21 12:43:18.893: githubSteveBlack: downloaded: 57865
INFO[025]2020-11-21 12:43:18.894: githubSteveBlack: extracted: 43834
INFO[026]2020-11-21 12:43:18.895: githubSteveBlack: dropped: 14031
NOTI[027]2020-11-21 12:43:20.010: Total entries found: 126632
NOTI[028]2020-11-21 12:43:20.011: Total entries extracted 109280
NOTI[029]2020-11-21 12:43:20.011: Total entries dropped 17352
INFO[02a]2020-11-21 12:43:45.860: Successfully restarted dnsmasq
NOTI[02b]2020-11-21 12:43:45.860: Blacklist update completed......

However, the configuration is not active anyway. There is no blacklist branch in show services dns resolver.

The only method I foud to bring back backlist is full reconfiguration with sudo dpkg-reconfigure edgeos-dnsmasq-blacklist. This means all custiomizations and exception lists go to waste.

Am I doing something wrong?

false positive with hostsfile.org

http://www.hostsfile.org/Downloads/hosts.txt

contains

127.0.0.1	www.crunchbase.com

which is an obvious false-positive.

I've added it to my local whitelist using:

configure
set service dns forwarding blacklist exclude www.crunchbase.com
commit;save;exit

Stopped working after update to EdgeOS v1.10.8

I recently updated my ERPoe-5 to EdgeOS v1.10.8 and found that blacklist was no longer doing its magic.

First, I tried configuring of a new domain:

$ configure
# set service dns forwarding blacklist domains exclude bing.com           
The specified configuration node is not valid
Set failed

Second, I tried updating but was already on the current version.

$ sudo apt-get update && sudo apt-get upgrade edgeos-dnsmasq-blacklist
…
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

Finally, tried reinstalling, which did the trick and also included my previously included domains.

Question: Should updating EdgeOS require reinstall of blacklist?

FAQ Cleanup suggestion

Since the malwaredomains list has been stale and removed from the package, might be a good idea to remove/update the hyperlinks in the FAQ and/or replace it with a current list or replace the name with an obvious fake.

The No Coin Filter list is current, follows the same structure, and is still in the default package.
https://raw.githubusercontent.com/hoshsadiq/adblock-nocoin-list/master/hosts.txt

No config files in /etc/dnsmasq.d after apt install

I installed to an EdgeRouter X using the instructions here.

me@ubnt:~$ show configuration commands | grep blacklist
set service dns forwarding blacklist disabled false
set service dns forwarding blacklist dns-redirect-ip 0.0.0.0
set system package repository blacklist components main
set system package repository blacklist description 'Britannic blacklist debian stretch repository'
set system package repository blacklist distribution stretch
set system package repository blacklist password ''
set system package repository blacklist url 'https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public/'
set system package repository blacklist username ''
set system task-scheduler task update_blacklists executable arguments 10800
set system task-scheduler task update_blacklists executable path /config/scripts/update-dnsmasq-cronjob.sh
set system task-scheduler task update_blacklists interval 1d

~$ tail /var/log/update-dnsmasq.log
NOTI[001]2023-01-26 05:31:13.591: Starting blacklist update...
INFO[002]2023-01-26 05:31:13.624: Checking for stale blacklists...
INFO[003]2023-01-26 05:31:15.849: Successfully restarted dnsmasq
NOTI[004]2023-01-26 05:31:15.850: Blacklist update completed......

~$ ls /etc/dnsmasq.d
README                    dnsmasq-dhcp-config.conf

This seems to indicate that the package in installed and blacklist update is running but it's not actually writing any files for dnsmasq to process that blackhole DNS requests to bad hosts. Is there a way I can confirm that is the case? Are there other troubleshooting steps I should take?

Blacklist Configuration File Clarification in README.md

It's possible I am misunderstanding the instructions, and I'm hoping to get some clarification.

When I globally include a host/domain or add a new source, is it supposed to appear in the /config/user-data/blacklist.cfg file? It never does for me although if I try and add it again, it shows that it already exists. If I were to reboot the router without it showing in the file, will it remain?

Also, at what point does the blacklist.cfg file get backed up? Is it only when the package is removed? I ask since the instructions mention deleting the file, and I couldn't tell if it was because the blacklisting was disabled or the entire package deleted, or if it backed it up at some other time. After I've added a new host/domain or a new source, the blacklist.cfg file hasn't been backed up.

I appreciate the efforts you've made with this program! Thanks.

troubleshooting

hello there!
sometimes i get some legit content blocked when im navigating from my mobile phone while connected to the wireless.
im wondering if I could output some logs in order to troubleshoot blacklisting to be able to make exceptions. because in the computer browser certainly i can enable developer tools and see the domains being blocked but in the mobile phone im not sure how to proceed since some content might be inside an app. any ideas?
thanks!

exclude fails configuration syntax check

Installation went smoothly, however I am unable to add exceptions to the blacklist, according to the README.

EdgeRouter Lite v1.10.1
Blacklist Release v1.0.9

configure
set service dns forwarding blacklist exclude photos.app.goo.gl
commit
update-dnsmasq: NOTI[001]17:53:43.097: Starting blacklist update...
update-dnsmasq: INFO[002]17:53:43.100: Removing stale blacklists...
update-dnsmasq: INFO[003]17:53:43.106: excluded-global: downloaded: 1
update-dnsmasq: INFO[004]17:53:43.108: excluded-global: extracted: 1
update-dnsmasq: INFO[005]17:53:43.109: excluded-global: dropped: 0
update-dnsmasq: NOTI[006]17:53:43.111: Total excluded-global: 1, dropped: 0
update-dnsmasq: INFO[007]17:53:43.113: excluded-domains: downloaded: 0
update-dnsmasq: INFO[008]17:53:43.115: excluded-domains: extracted: 0
update-dnsmasq: INFO[009]17:53:43.116: excluded-domains: dropped: 0
update-dnsmasq: INFO[00a]17:53:43.118: excluded-hosts: downloaded: 0
update-dnsmasq: INFO[00b]17:53:43.119: excluded-hosts: extracted: 0
update-dnsmasq: INFO[00c]17:53:43.120: excluded-hosts: dropped: 0
update-dnsmasq: ERRO[00d]17:53:43.278: ReloadDNS():
 error: Restarting DNS forwarder and DHCP server: configuration syntax check failed!

%!(EXTRA string=exit status 1)

Commit failed

can´t commit

hi
just update the black list on my edgerouter but now won´t let me commit
I add and exclution but gime and update error say no internet access but pretty sure the router have internet
try to reboot the router same problem try disable the blacklist but can´t

Return NXDOMAIN instead of NOERROR to dns-redirect-ip

Is there a way to set responses for blacklisted domains to be an NXDOMAIN (Domain does not exist) response instead of a NOERROR response with an A record pointing to dns-redirect-ip?

If not, can a new feature be added to switch to this behavior instead?

Problem on router restarts

Hi..

First at all thanks; works perfect!!!

But I have a little problem; everytime the router reboots the script does not start and I have to uninstall and then reinstall....

¿Any sugestion?

Whitelist source

Currently, there is no way (that I can tell) to add a source file as a whitelist. This would save time manually entering commonly whitelisted domains, such as annudeepND's whitelist.

Blocks Slack attachments

For some reason a recent update has prompted Slack to not be able to upload or download attachments, because it blocks files.slack.com.

After using little snitch to see what slack was accessing, I whitelisted files.slack.com, and the problem was fixed.

This is an easy fix, but it’s a very odd thing to block, and could really confuse some other people who are probably pestering Slack support about it.

Time for an update build?

Lots of folks are posting problems with the ERX units and it's a definite result of those two primary lists, OISD and Steve Black, growing so large.

Here are the default results from package installs:
July 2022: Total entries extracted 136104
Today: Total entries extracted 234118

I think it makes sense to remove one of them and then update the FAQ with a special ERX/ERX-SFP section with the commands that users can copy/paste if they want to remove/add the other.

Removing Steve Black / Keeping OISD
Total entries extracted 59691

Removing OISD / Keeping Steve Black
Total entries extracted 194322

Also, as an aside, I was curious about the simple_tracking list with the 34 domains. It appears that while the disconnect.me site and browser extension hasn't been updated in quite a while, they do have a json file that gets updated more frequently.
What's even stranger is that when I remove it from the default package, I get a "Total entries extracted 234360," and putting it back the number goes back down to 234118. Just thought, I pass that one.

Thanks again for all your work on this! Cheers!

Hangs EdgeOS Upgrade When in firstboot Directory

Hi there,

Thanks for making this package, it's great to be able to have dns blacklisting right on my ER without any separate network hardware.

EdgeOS has a feature where any .deb files placed in /config/data/firstboot/install-packages will be installed when the OS is first booted (ie. when upgrading from one release to another). This is a useful feature to install some packages that aren't included by default, and ensure they'll be available to the system when it is being configured.

Ideally, I could put the edgeos-dnsmasq-blacklist file here and the existing config that's being loaded would take effect. Unfortunately, some part of the process does not work, and the device hangs somewhere and never completes the boot process, and I lose access to it. Unfortunately, I can't provide many more details than that. If I had a spare device, I would console in and see what exactly is happening, but I'm unwilling to intentionally break my device at the moment.

If you're willing to investigate further, the script that kicks all this off is /etc/ubnt/ubnt-rcS/ubnt-rcS.sh, and it attempts to install all packages with the following command:

DEBIAN_FRONTEND=noninteractive dpkg --force-confold --force-confdef -i "${pkgs[@]}" 2>&1 >> /var/log/ubnt-firstboot-pkg-install.log

I suspect that the auto-config setup portion of the Debian installation script isn't happy when being run this early in the boot process, but again, I have no other data, that's just a hunch. Oh, further info, this has happened to me at least twice, under 2.0.x releases, and most likely all with 1.7.7.x releases of blacklist.

At the very least, you could possibly add a warning in the readme about this, and how it might just be the best idea to re-install the package after the first boot has been completed.

Consider removing youtubeadsblacklist

youtubeadsblacklist hasn't seen an update in nearly 2 years. When I leave this source in the config, Youtube ads are blocked – but, unfortunately, so are Youtube videos. Removing the source re-enables playback.

cc: @anudeepND

Package update needed...

Hey there - Two items which require your expertise....

Looks like OISD changed their url again and as a result the base set of adblocks is significantly reduced.

Also - looks like apt-get is only pulling down 1.2.4.8 and manual intervention is required to get 1.2.4.9 installed. My guess is that it needs to be updated here?

Thread with details at the UI forums....

Cheers!