cedws / apple-telemetry Goto Github PK

View Code? Open in Web Editor NEW

187.0 14.0 19.0 106 KB

Domain blocklists, IP blocklists, Little Snitch .lsrules, and cloaking files for blocking Apple telemetry

License: GNU General Public License v3.0

Python 51.95% Makefile 48.05%

blocklists privacy telemetry apple little-snitch

apple-telemetry's Introduction

apple-telemetry

⚠️ Looking for maintainers! #25

Domain blocklists, IP blocklists, and cloaking files for domains that Apple devices frequently contact. DNSCloak is an excellent app capable of using these files, but it's a bit of a pain to get the blocklists onto your device. Services such as updates, App Store, and Apple Pay will probably be blocked by this list. You can remove entries manually, but this is tedious, so I recommend just temporarily disabling the blocklist when necessary.

Here's a vague overview of what each domain is for.

Why?

Users should have more control over the telemetry sent by their smartphones. Not only that, but I've seen a large improvement to my device's battery life while using these blocklists.

Usage (iOS)

Download DNSCloak, or another app capable of using DNS blocklists
Navigate to releases
Copy the desired blocklist to your clipboard (cloaking-rules is recommended)
Download Textor, or another app capable of saving text to Files
In Textor, create a new file and paste the blocklist
Remove domains for services you wish to use (such as Apple Pay)
Navigate to the DNSCloak settings and select the saved blocklist

Usage (macOS)

Add desired hosts to block to /etc/hosts

Notes

This is not an exhaustive list. There are hundreds of domains contacted on a regular basis by Apple devices, with no consistent naming scheme or obvious purpose. If you want to block all domains, you should use a whitelist.
Some domains may not be related to Apple. I've had to filter out the domains from my own blacklist log manually. However, some domains such as weather.com are contacted by built-in applications (Weather). Entries like these belong in this repository.

Contribution

If you come across any other domains, please submit a PR! Add the entries to the blacklist file then run make (you'll also need Python 3 - use the Poetry package manager). It can take a few minutes because it has to make a lot of DNS queries.

Similar Projects

https://gitlab.com/intr0/AppleBlock

apple-telemetry's People

Contributors

Stargazers

Watchers

Forkers

deen666 jujonet maathimself r1133ardo daruti648 underling3311 iveskins alfian878787 arbal ty-cs sysadmindoc hbxc0re oellewa ink-splatters schrekia spencerisgiddy thomasmerz gruberdev frankherchet

apple-telemetry's Issues

hosts that are required for system updates

The following hosts are required to perform OS updates and should probably not be included in the blacklist:

gdmf.apple.com
gdmf.apple.com.akadns.net
swcdn.apple.com
swscan.apple.com

There may be OCSP hosts required as well (for trustd to validate OS updates), this is not an exhaustive list.

Remove duplicate input and output entries.

Fixed in 4b80672.

Little Snitch .lsrules file

I made a .lsrules file in order to provide this list to Little Snitch users. You can find it here (https://filebin.net/jr27k96b95j2og65/apple_blocking.lsrules?t=plbgi4cv), it expires in 1 week from now on 2020-07-30 19:52:11 UTC.

Please upload it on this repo and update it along the original blacklist file. For anyone who want to do it, here is the Python code I used to get it, feel free to modify it. Note that it works with any list of domains, you may want to update the \n separator in the split function.

rules_list = open('path/to/txt', 'r').read().split('\n')

rule_pattern = '{{"action": "deny","direction": "outgoing","priority": "regular","process": "any","remote-domains":"{0}"}}'

# TO DO : update ip number and date
beginning = '{"description": "Source: https://github.com/adversarialtools/apple-telemetry | Unique IP number: 1123 |  Last Update: 25/04/2020 00:00 UTC+8","name": "apple_blocking","rules": ['

end = ']}'

rules_str = ','.join([rule.format(d) for d in domains])

with open('path/to/little_snitch_file.lsrules', 'w') as f:
    f.write(beginning + rules + end)

Determine which apps are broken by which rules

It would be very useful to have a table in a md file listing what each rule / group of rules will break in iOS / maxOS, with details of what is not working so that each user can choose between privacy and usability.
There could also be various information on the role of each connexion.

Anyone can test the rules, so there would just need PR to update the rows.

For instance (not for real, that's an example) :

Domain	Role	Concerned applications	Broken
*.itunes-apple.com.example.net	Retrieves albums' cover	iTunes	Albums' covers won't show
*.randomstuff.net	Untested	Untested	Untested
*.app-store.something.com	?	App Store, iTunes, Podcast	Display a blank screen, installed apps won't update
		iTunes	Display a blank screen, play random music will play at 2:00 AM
		Podcast	Display a blank screen

Otherwise we could also just have 3 levels of inconvenience (2 = app does not work, 1 = some features are broken, 0 = everything works but privacy-killing connexions are blocked) which could be easier to read :

Domain	Application	Level
*.itunes-apple.com.example.net	iTunes	1
	App Store	0
*.randomstuff.net	?	?
*.app-store.something-1.com	App Store	1
*.app-store.something-2.com	App Store	2

I guess this file could be updated everytime a new line is added to the domains list.

remove support.apple.com

This is a website, not an API.

Organise domains into files.

I'd like to organise the repo a bit more by grouping domains into individual files, then have a script stitch them together into the blocklists.

Unblock domains for Certificate validation for Apple devices

https://support.apple.com/en-us/HT210060 says:

Certificate validation
Apple devices must be able to connect to the following hosts to validate digital certificates used by the hosts in this article.

I checked my Pi-hole and found that there are some/two domains/hosts being blocked in your blocklist which shouldn't be blocked:

 Match found in https://raw.githubusercontent.com/liamengland1/mischosts/master/apple-telemetry:
   ocsp.apple.com
 Match found in https://raw.githubusercontent.com/liamengland1/mischosts/master/apple-telemetry:
   valid.apple.com

Please check and remove them from your blocklist. Thank you!

https://raw.githubusercontent.com/liamengland1/mischosts/master/apple-telemetry

# Blocks Apple telemetry and tracking but won't break your device
# Sourced from https://github.com/adversarialtools/apple-telemetry/blob/master/blacklist
# Originally hosted at https://github.com/llacb47/apple-telemetry/blob/master/blacklist

Add new domains to blacklist

apps.mzstatic.com
api-apps-lb.itunes-apple.com.akadns.net
play.itunes.apple.com.edgesuite.net
p23-buy.itunes-apple.com.akadns.net
apple.news
30.courier-push-apple.com.akadns.net
mt-ingestion-service-pv.itunes.apple.com
gsp85-ssl.ls.apple.com

I found more Apple Domains that aren't in the list

I don't know if you are wanting more domains but I just found some more.. I saw you updated the list 9 days ago so might as well help out. :)
I was doing a search with the dns "e673.dsce9.akamaiedge.net" and found more alternative names that weren't in the list. I also saw "e4478.a.akamaiedge.net" but this one wasn't on the list and that's why I was digging even more because I wanted to know what it is and why is it need for what traffic.
The list has duplicates but just use a duplicate remover to remove them.
https://www.textfixer.com/tools/remove-duplicate-lines.php

Also, I might as well ask, have you ever done any reverse dns looksups or back grounds checks to see what kind of traffic is going where and why to these domains are need or used? If so it would much appreciated if you could share you knowledge to help out one another!!
https://threatintelligenceplatform.com/report/e673.dsce9.akamaiedge.net/RWlLCuGgLD
https://threatintelligenceplatform.com/report/e4478.a.akamaiedge.net/DJmj8B9bhw

Edit:
When I add this list there was 1070 and now there is 1123 total. Check it out vs the normal list.
Last, I was wondering if I could be apart of this project :)
https://github.com/MrWaste/apple-telemetry

Sincerely,
Mr. Waste

web-experience.itunes.apple.com
vpp-app.itunes.apple.com
vocabulary.itunes.apple.com
videos.apple.com
uts-preview.itunes.apple.com
uts-api-siri.itunes.apple.com
upp.itunes.apple.com
tv.apple.com
tf-feedback.itunes.apple.com
sync.itunes.apple.com
su.itunes.apple.com
store.mzstatic.com
sp.itunes.apple.com
sitemaps.itunes.apple.com
siri-search.itunes.apple.com
sf-api-token-service.itunes.apple.com
search.itunes.apple.com
se.itunes.apple.com
se-edge.itunes.apple.com
sb.tv.apple.com
sb.music.apple.com
s5.mzstatic.com
s4.mzstatic.com
s3.mzstatic.com
s2.mzstatic.com
s1.mzstatic.com
s.mzstatic.com
radio.itunes.apple.com
radio-services.itunes.apple.com
radio-quickplay.itunes.apple.com
radio-activity.itunes.apple.com
podcasts.apple.com
pd.itunes.apple.com
pcr.apple.com
partiality.itunes.apple.com
music.apple.com
metrics.mzstatic.com
itunesu.itunes.apple.com
itunes.apple.com
itc.mzstatic.com
is5-ssl.mzstatic.com
is4-ssl.mzstatic.com
is3-ssl.mzstatic.com
is2-ssl.mzstatic.com
is1-ssl.mzstatic.com
init.itunes.apple.com
finance-app.itunes.apple.com
files.itunes.apple.com
embed.itunes.apple.com
edge.itunes.apple.com
edge-search.itunes.apple.com
dzc-metrics.mzstatic.com
desktop-store.itunes.apple.com
desktop-music.itunes.apple.com
desktop-music-legacy.itunes.apple.com
configuration.apple.com
carrierbundle.itunes.apple.com
books.apple.com
bookkeeper.itunes.apple.com
bag.itunes.apple.com
b5.mzstatic.com
b4.mzstatic.com
b3.mzstatic.com
b2.mzstatic.com
b1.mzstatic.com
apps.mzstatic.com
apps.apple.com
api.videos.apple.com
api.podcasts.apple.com
api.music.apple.com
api.itunes.apple.com
api.edu.apple.com
api.books.apple.com
api.apps.apple.com
api-edge.apps.apple.com
amp-api.podcasts.apple.com
amp-api-search-edge.apps.apple.com
amp-api-edge.apps.apple.com
accertify.mzstatic.com
a5.mzstatic.com
a4.mzstatic.com
a3.mzstatic.com
a2.mzstatic.com
a1.mzstatic.com

Remove Apple location service from blacklist

I use your list in my pihole. But I had to whitelist the following regex to use Apple Maps.

ls(.\|-)apple.com

I would recommend to remove those matches to be able to use the service.

Domain to remove for update ?

Hello,

I create a dns side protection, and i want to lock apple domain, but i want to let the possibility to update since i dont touche often the dns (i will lock it on the client side the store domain and ios update), but can you tell me witch domain i have to remove to autorise only these update ?

edit : i know there is a wiki, but app store isn't in it ^^

Thanks

mask-api.icloud.com - new host

mask-api.icloud.com is a new host that 12.x connects to (presumably for IP masking, an iCloud service) that should be added to the blocklist.

Use releases.

Pushing the output files is not ideal, so I would like to release them periodically using the repository's release page.

Determine which domains should be removed.

There's a tonne of domains in the blacklist now which can't be proven to be related to Apple or iOS. It's difficult to tell which ones belong and which ones don't. Querying search engines for most of these domains reveals nothing.

Examples:

*.akamaiedge.net
*.akadns.net

If anyone has advice, please let me know.

Please add the domains to block list

daypass.api-glb-sto.smoot.apple.com
p46-fmf.icloud.com
p46-fmip.icloud.com
p46-contacts.icloud.com
p46-keyvalueservice.icloud.com
p46-fmfmobile.icloud.com
p46-avaliability.icloud.com
p46-quota.icloud.com
init.push.apple.com
mesu-cdn.origin-apple.com.akadns.net
fmip.fe.apple-dns.net
quota.fe.apple-dns.net
e10489.a.akamaiedge.net
s.mzstatic.com
is-ssl.mzstatic.com.itunes-apple.com.akadns.net
p23-buy.itunes.apple.com
static.icloud.com.edgekey.net
e673.dsce9.akamaiedge.net
configuration.apple.com.akadns.net
eu-irl-00001.s3.dualstack.eu-west-1.amazonaws.com
me-apple-dns.net
gcs-eu-00002.content-storage-download.googleapis.com
e6858.dsce9.akamaiedge.net
www.apple.com.edgekey.net
sf-api-token-service.itunes.apple.com
api.apps.apple.com
api-edge.apps.apple.com
configuration.ls.apple.com
health.apple.com
app.smartmailcloud.com
news.apple.com
is1-ssl.mzstatic.com
is2-ssl.mzstatic.com
is3-ssl.mzstatic.com
is4-ssl.mzstatic.com
is5-ssl.mzstatic.com

Use Amass for (sub)domain collection.

Most domains currently in the repository were collected by hand from logs. I'd like to use OWASP's Amass to search a list of root domains and collect a large list of subdomains.

I've already got something together for this on a branch, but I need to run it and give it a test. There's a huge number of domains that Apple owns that will probably take days or even weeks to find. I'll be running this on an AWS instance for now to see if it ever finishes the search.

Three things come to mind that could go wrong:

The instance crashes and the entire process will have to be redone
The instance uses massive amounts of bandwidth and bankrupts me
The configured DNS server (1.1.1.1) blacklists me

Looking for maintainers

I'm working full-time and unfortunately I don't have the capacity or desire to support this project anymore. If you're interested in maintaining the repository, including the list of domains and managing releases, please let me know below.

ocsp.godaddy.com

When blocked, linuxmint.com can't be reached. Generating error with code SEC_ERROR_OCSP_SERVER_ERROR.

Add new domains

any-appleweather-cache.internal.query.a03.yahoodns.net
api.apple-cloudkit.com
api.apple-cloudkit.fe.apple-dns.net
apple-finance.query.yahoo.com
appleid.cdn-apple.com
apple-mobile.query.yahooapis.com
appleweather-cache.internal.query.g03.yahoodns.net
apptrailers.itunes.apple.com.c.footprint.net
apsu.apple.com
ax.phobos.apple.com.edgesuite.net
basejumper.apple.com
c.apple.news
cdsassets.apple.com
commnat-cohort-gc.ess.apple.com
commnat-main-gc.ess.apple.com
configuration.apple.com.edgekey.net
gspe19.ls.apple.com
gspe21.ls.apple.com
iadsdk.apple.com.edgekey.net
init.ess.apple.com.edgesuite.net
init.gc.apple.com.edgesuite.net
init-p01md.apple.com.edgesuite.net
internalcheck.apple.com
isg-apple.com.akadns.net
itunes.apple.com.edgekey.net
mzuserxp.itunes-apple.com.akadns.net
p47-buy.itunes.apple.com
pancake.apple.com.edgekey.net
pd-st.itunes.apple.com
play.itunes-apple.com.akadns.net
ssl.apple.com
static.gc.apple.com.edgekey.net
static.ips.apple.com
swcdn.apple.com
3.courier-push-apple.com.akadns.net
4.courier-push-apple.com.akadns.net
5.courier-push-apple.com.akadns.net
6.courier-push-apple.com.akadns.net
7.courier-push-apple.com.akadns.net
8.courier-push-apple.com.akadns.net
9.courier-push-apple.com.akadns.net
11.courier-push-apple.com.akadns.net
13.courier-push-apple.com.akadns.net
14.courier-push-apple.com.akadns.net
15.courier-push-apple.com.akadns.net
16.courier-push-apple.com.akadns.net
17.courier-push-apple.com.akadns.net
18.courier-push-apple.com.akadns.net
19.courier-push-apple.com.akadns.net
21.courier-push-apple.com.akadns.net
22.courier-push-apple.com.akadns.net
23.courier-push-apple.com.akadns.net
24.courier-push-apple.com.akadns.net
25.courier-push-apple.com.akadns.net
26.courier-push-apple.com.akadns.net
27.courier-push-apple.com.akadns.net
29.courier-push-apple.com.akadns.net
32.courier-push-apple.com.akadns.net
33.courier-push-apple.com.akadns.net
34.courier-push-apple.com.akadns.net
35.courier-push-apple.com.akadns.net
36.courier-push-apple.com.akadns.net
38.courier-push-apple.com.akadns.net
41.courier-push-apple.com.akadns.net
42.courier-push-apple.com.akadns.net
49.courier-push-apple.com.akadns.net
50.courier-push-apple.com.akadns.net
a1047.phobos.apple.com
a1076.phobos.apple.com
a1109.phobos.apple.com
a1190.phobos.apple.com
a1191.phobos.apple.com
a1257.phobos.apple.com
a1270.phobos.apple.com
a1364.phobos.apple.com
a1426.phobos.apple.com
a1437.phobos.apple.com
a1473.phobos.apple.com
a1484.phobos.apple.com
a149.phobos.apple.com
a1500.phobos.apple.com
a1566.phobos.apple.com
a1650.phobos.apple.com
a1715.phobos.apple.com
a1724.phobos.apple.com
a1808.phobos.apple.com
a1816.phobos.apple.com
a1850.phobos.apple.com
a1861.phobos.apple.com
a1891.phobos.apple.com
a247.phobos.apple.com
a26.phobos.apple.com
a284.phobos.apple.com
a3.phobos.apple.com
a383.phobos.apple.com
a407.phobos.apple.com
a423.phobos.apple.com
a479.phobos.apple.com
a487.phobos.apple.com
a507.phobos.apple.com
a512.phobos.apple.com
a53.phobos.apple.com
a560.phobos.apple.com
a628.phobos.apple.com
a63.phobos.apple.com
a663.phobos.apple.com
a669.phobos.apple.com
a672.phobos.apple.com
a761.phobos.apple.com
a840.phobos.apple.com
a859.phobos.apple.com
a910.phobos.apple.com
a93.phobos.apple.com
a948.phobos.apple.com
a949.phobos.apple.com
a977.phobos.apple.com
a983.phobos.apple.com

how to use the little snitch .lsrules file?

The documentation is lacking on this matter and I don't see an "import" option in LS 5.3.

Definitely a good project.

Reminds me of this: https://gitlab.com/intr0/AppleBlock

Seems we both found many overlapping "phone home" Apple DNS requests. Initially started it as a hobby project to protect people from having a specific anti-ad app from being removed from people's devices by Apple. Moved it to GitLab some time later. Good to see something similar being actively maintained. 👍🏼

Links need to be updated

In your readme file you reference links ‘https://stan.sh/posts/dns-requests-of-ios-devices’ which no longer exists and is also in an unsecure domain.
You also reference ‘https://cedwards.xyz/blog/ios-dns-privacy-investigation/’ which no longer exists either. Perhaps these links should be updated as they are of interest due to privacy concerns.

Where can i find the list?

Hello! I want to try your blocking list, but i can not find the list for dnscloack you provided in the first post. Can you help me? There are some files in assets, but when i download it with ios safari downloader, i can‘t open the files.

Romy :o)