Comments (7)
Thanks. I will have a look into it when I have the time. This seems like a valid solution. With the lists from https://codeberg.org/HomemadeAdvanced/PiHole/src/branch/main/PiHoleAdlistsGermany.txt I have 27734794 with 17329998 unique domains.
from pihole_adlist_tool.
I'm not sure I understand what you want to archieve: You can select
"Enable the minimal number of adlists that cover all domains that would have been blocked"
which will enable only the set with the least adlists necessary to block everything that would have been blocked. If you enable "Enable only adlists with covered unique domains" you might miss some domains that are not unique (e.g. contained in onĺy two adlists)
from pihole_adlist_tool.
Currently I have many domains covered by many lists. The total number of blocked domains is twice the number of unique domains. These redundant domains should be reduced, but without the feature of including the visited domains of the last time.
from pihole_adlist_tool.
So what you want is:
"If a adlist contains only domains that are also part of other adlist, deactivate this adlist"? And check this for all adlist at the same time so the maximum number of adlist can be disabled?
(This would all be independent of your browsing behavior. )
This is not possible with the current tool. It is designed to be based on your browsing habits, not focusing on the adlists alone. But I do see some value in your idea. I'll think about it.
from pihole_adlist_tool.
That's exactly what I would like to use. It would be great if it could be implemented.
from pihole_adlist_tool.
I think this also can never be made possible because adlists may change independently, so list1 may add/remove domains that are not/still in list2. Pi-hole will already make all domains "unique":
[i] Number of gravity domains: 4514301 (4038026 unique domains)
[i] Number of exact blacklisted domains: 24
[i] Number of regex blacklist filters: 25
[i] Number of exact whitelisted domains: 13
[i] Number of regex whitelist filters: 4
But… wait… if you're interested in it this might be a brute-force solution by comparing all lists with each other lists. So be careful when you have "many" adlists because it will run "no-of-adlists minus 1 x no-adlists devided by 2" times:
for d1 in *.domains; do
for d2 in *.domains; do
[ "$d1" = "$d2" ] && break
echo $d1 vs. $d2
comm -3 <(sort $d1) <(sort $d2) | wc -l
done
done
Each result with "0" shows you two lists which contents are 100% the same.
For testing purpose I copied one adlist to proof that a "100% the same" duplicate will be found by this:
cp list.1.raw.githubusercontent.com.domains zzz.list.1.raw.githubusercontent.com.domains
❓ Does this help you and can this issue be closed?
from pihole_adlist_tool.
See man page:
-3 suppress column 3 (lines that appear in both files)
from pihole_adlist_tool.
Related Issues (20)
- Regex analysis not supported on docker HOT 2
- Receiving an error "database disk image is malformed" HOT 1
- Starting with version 2.4.0, you get multiple 'environment: line 5: docker: command not found' messages HOT 3
- "SUDO_SQLITE: command not found" HOT 3
- Does regex checking account for Filter Lists? HOT 2
- Regex Mode does not work in docker HOT 2
- Wrong detection of PIHOLE_DOCKER?! HOT 8
- Shellcheck is complaining a lot about pihole_adlist_tool HOT 1
- Installation HOT 3
- Question HOT 3
- Can't read /etc/pihole/pihole-FTL.db HOT 5
- Running tool outside the Docker-container throws some errors HOT 7
- Error: stepping, database is locked HOT 1
- Add assigned group(s) for adlist in result HOT 3
- Instructions HOT 2
- Parse error near line 14: no such column: additional_info HOT 12
- ABP style adlists HOT 4
- Please add a licence HOT 1
- List the domains which generated a hit HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pihole_adlist_tool.