adguardteam / dnslibs Goto Github PK

DNS filtering library that's used in AdGuard products

License: Apache License 2.0

CMake 3.24% C++ 62.88% Python 0.76% Objective-C 2.41% Objective-C++ 4.71% Shell 0.35% Java 4.93% C 3.11% C# 12.68% PHP 0.47% Jinja 0.03% JavaScript 0.69% CSS 3.74%

open-source adguard dns-filter

dnslibs's Introduction

AdGuard C++ DNS libraries

A DNS proxy library that supports all existing DNS protocols including DNS-over-TLS, DNS-over-HTTPS, DNSCrypt and DNS-over-QUIC.

Build instructions

Native library

Prerequisites

Conan C++ package manager 2.0.5 or higher
CMake 3.24 or higher
GCC 9 or higher / Clang 8 or higher

Building

If it is a clean build, export custom conan packages to the local conan repository. See https://github.com/AdguardTeam/NativeLibsCommon/blob/master/README.md for details.

Execute the following commands in Terminal:

mkdir build && cd build
cmake ..

For testing execute the following:

make -j 4 tests
ctest -j 4

MacOS/iOS framework

Prerequisites

Conan C++ package manager 2.0.5 or higher
CMake 3.24 or higher
Clang 8 or higher
Xcode 11 or higher

Building

If it is a clean build, export custom conan packages to the local conan repository. See https://github.com/AdguardTeam/NativeLibsCommon/blob/master/README.md for details.

Execute the following commands in Terminal:

cd <path/to/dnsproxy>
bash platform/mac/framework/build_dnsproxy_framework.sh --bp <build_path> --fwp <path/to/dnsproxy>/platform/mac/framework/

For details of the building script usage execute the following:

bash platform/mac/framework/build_dnsproxy_framework.sh --help

As a result the DNS proxy framework will be located in <build_path>/<target_name>.<framework_type>, where <build_path> is the build path one passed to script, <target_name> is the framework name, <framework_type> is either framework (if one specified the target platform) or xcframework (if one didn't specify any platform, or passed all).

Useful notes

Proxy configuration: native, mac, android
Rules syntax
Developer documentation

Testing changes as a dependency

To test local changes in the library when it is used as a Conan package dependency, do the following:

If the default vcs_url in <root>/conanfile.py is not suitable, change it accordingly.
Commit the changes you wish to test.
Execute ./script/export_conan.py local. This script will export the package, assigning the last commit hash as its version.
In the project that depends on dns-libs, update the version to <commit_hash> (where <commit_hash> is the hash of the target commit): Replace dns-libs/1.0.0@adguard_team/native_libs_common with dns-libs/<commit_hash>@adguard_team/native_libs_common.
Re-run the cmake command. Note:
- If you have already exported the library in this way, the cached version must be purged: conan remove -f dns-libs/<commit_hash>.

License

Apache 2.0

dnslibs's People

Contributors

Stargazers

Watchers

Forkers

captainofm youmuyou msiva21 vasili8m sfionov archetype-software jdixoncs ekmixon dungnt8805 gwynn-1 thuannv acidburn0zzz marcus-arcadius clintonkildepstein spencerisgiddy agimbrac dandelionsprout

dnslibs's Issues

Add the ability to block dns requests by IP address

Incorrect response formatting for records with CNAME

Now we get answers like

CNAME, example2.org
example2.org 222 IN CNAME example3.org
example3.org 333 IN CNAME example4.org
example4.org 444 IN A 127.0.0.1

the first answer is short, but other answers are long

We need all the answers to be short like

CNAME, example2.org
CNAME, example3.org
CNAME, example4.org
A, 127.0.0.1

NETunnelManager Proof-Of-Concept

[Android] Wrong filter ID in DnsRequestProcessedEvent

Steps to reproduce:

Specify 2 filters. In my case it's AG SDN filter (ID == 15) and tiny custom filter with only one rule: 0.0.0.0 example.com (ID == -3)
Try to resolve example.com
Filter ID in DnsRequestProcessedEvent is wrong

... bytesSent=0, bytesReceived=0, rules=[0.0.0.0 example.com], filterListIds=[15], whitelist=false...

UDP/TCP listener for DNS proxy

Also there should be decision about how to dispatch synchronous queries.
I think that using libdispatch on all patforms may be useful.

Enhance rule validation

Cert verification callback for DoT/DoH

To merge DoT/DoH without it.

DOH client

This task consists of two parts:

Prepare a simple DNS-over-HTTPS client lib that uses standard set of tools (or even curl)
Test how memory efficient it is
Depending on the result we should decide which parts of the client should be implemented using native libs (Win or Mac/IOS).

Block AAAA requests in networks without IPv6 interface

Useful option.
AdguardTeam/dnsproxy#70

Add fallback upstream

Please, add fallback upstream by analogy with golang dnsproxy:
https://github.com/AdguardTeam/dnsproxy/blob/master/proxy/proxy.go (428)

This is a plain DNS upstream(s) that should be used if all other upstreams are failed to resolve query.

Plain DNS client

Implement the client
Handle truncated responses

Fix "Unexpected EOF" for DoT and TCP

Start using the same logger anywhere in project

Android adapter library

Fix crashes we found in AdGuard for iOS

See the details here:
AdguardTeam/AdguardForiOS#1432

You might need @IvanIin to get the actual crash reports.

Separate bootstrap from UpstreamSettings

Please, separate bootstrap from UpstreamSettings since the same bootstrap will be used for all upstreams.

Provide the method to check if an upstream was created successfully

For now an upstream creation procedure always returns without an error.

Besides that, move the address of upstream server inside the upstream options structure. It's strange that one is passed separately.

Crash in utils::split_by_any_of

Thread 3 name:
Thread 3 Crashed:
0   AGDnsProxy                    	0x0000000104887564 ag::utils::split_by_any_of(std::__1::basic_string_view<char, std::__1::char_traits<char> >, std::... + 92
1   AGDnsProxy                    	0x0000000104887574 ag::utils::split_by_any_of(std::__1::basic_string_view<char, std::__1::char_traits<char> >, std::... + 108
2   AGDnsProxy                    	0x00000001047cf494 rule_utils::parse(std::__1::basic_string_view<char, std::__1::char_traits<char> >, std::__1::shar... + 96
3   AGDnsProxy                    	0x00000001047cc960 count_rules(unsigned int, std::__1::basic_string_view<char, std::__1::char_traits<char> >, void*) + 40
4   AGDnsProxy                    	0x0000000104889150 ag::file::for_each_line(int, bool (*)(unsigned int, std::__1::basic_string_view<char, std::__1::c... + 704
5   AGDnsProxy                    	0x00000001047cc570 filter::load(ag::dnsfilter::filter_params const&) + 376
6   AGDnsProxy                    	0x00000001047c94e0 engine::engine(ag::dnsfilter::engine_params) + 196
7   AGDnsProxy                    	0x00000001047c894c ag::dnsfilter::create(ag::dnsfilter::engine_params) + 80
8   AGDnsProxy                    	0x00000001047bec54 ag::dns_forwarder::init(ag::dnsproxy_settings const&, ag::dnsproxy_events const&) + 2952
9   AGDnsProxy                    	0x00000001047bb830 ag::dnsproxy::init(ag::dnsproxy_settings, ag::dnsproxy_events) + 664
10  AGDnsProxy                    	0x00000001047adff4 -[AGDnsProxy initWithConfig:handler:] + 5028
11  Tunnel                        	0x00000001046377b0 specialized DnsProxyService.start(upstreams:bootstrapDns:fallback:serverName:filtersJson:ipv6Avai... + 2980 (<compiler-generated>:0)
12  Tunnel                        	0x0000000104634688 @objc DnsProxyService.start(upstreams:bootstrapDns:fallback:serverName:filtersJson:ipv6Available:) + 244 (<compiler-generated>:0)
13  Tunnel                        	0x00000001045caac4 -[PacketTunnelProvider startDnsProxyWithSystemDnsIps:] + 724 (PacketTunnelProvider.m:686)
14  Tunnel                        	0x00000001045c7df8 __65-[PacketTunnelProvider startTunnelWithOptions:completionHandler:]_block_invoke + 196 (PacketTunnelProvider.m:197)
15  Tunnel                        	0x00000001045c8fe4 __74-[PacketTunnelProvider updateTunnelSettingsInternalWithCompletionHandler:]_block_invoke_2 + 284 (PacketTunnelProvider.m:412)
16  NetworkExtension              	0x00000001d0111540 __83-[NEExtensionPacketTunnelProviderContext setTunnelConfiguration:completionHandler:]_block_invoke + 148 (NEExtensionPacketTunnelProviderContext.m:97)
17  CoreFoundation                	0x00000001bdea2c20 __invoking___ + 144
18  CoreFoundation                	0x00000001bdd72d30 -[NSInvocation invoke] + 300 (NSForwarding.m:3306)
19  Foundation                    	0x00000001be3855d0 __NSXPCCONNECTION_IS_CALLING_OUT_TO_REPLY_BLOCK__ + 20 (NSXPCConnection.m:125)
20  Foundation                    	0x00000001be160d84 -[NSXPCConnection _decodeAndInvokeReplyBlockWithEvent:sequence:replyInfo:] + 608 (NSXPCConnection.m:269)
21  Foundation                    	0x00000001be38685c __88-[NSXPCConnection _sendInvocation:orArguments:count:methodSignature:selector:withProxy:]_bloc... + 236 (NSXPCConnection.m:1427)
22  libxpc.dylib                  	0x00000001bda74c18 _xpc_connection_reply_callout + 60 (serializer.c:119)
23  libxpc.dylib                  	0x00000001bda68ea0 _xpc_connection_call_reply_async + 76 (connection.c:695)
24  libdispatch.dylib             	0x00000001bdb69204 _dispatch_client_callout3 + 16 (object.m:521)
25  libdispatch.dylib             	0x00000001bdb270b0 _dispatch_mach_msg_async_reply_invoke$VARIANT$mp + 348 (mach.c:2442)
26  libdispatch.dylib             	0x00000001bdb152d0 _dispatch_lane_serial_drain$VARIANT$mp + 300 (inline_internal.h:2525)
27  libdispatch.dylib             	0x00000001bdb15e28 _dispatch_lane_invoke$VARIANT$mp + 468 (queue.c:3863)
28  libdispatch.dylib             	0x00000001bdb1f314 _dispatch_workloop_worker_thread + 588 (queue.c:6445)
29  libsystem_pthread.dylib       	0x00000001bdbb8b88 _pthread_wqthread + 276 (pthread.c:2351)
30  libsystem_pthread.dylib       	0x00000001bdbbb760 start_wqthread + 8

DNS stamps library

https://github.com/ameshkov/dnsstamps

Add ability to pass custom filtering rules without file

Please, add ability to pass custom (user) filtering rules without file creation

White-list rule is ignored when matching CNAME

Check this out:

We must not have blocked it due to this rule:
https://github.com/AdguardTeam/AdGuardSDNSFilter/blob/master/Filters/exceptions.txt#L45

client-api.itunes.apple.com AAAA doesn't resolve

Nameservers used:
192.168.10.125
2001:db8:7c02:1::1

Log:

2020/01/29 11:03:56:630 [2386326(dns proxy resolve queue)]  (DnsLogRecordsWriter) handleEvent got answer for domain: e673.dsce9.akamaiedge.net. answer: nonnil
2020/01/29 11:03:58:619 [2386325(dns proxy resolve queue)]  (DnsLibs) [2020-01-29 11:03:58.618827] [2386325] [DNS forwarder] [debug] [56833] handle_message Upstream failed to perform dns query: Timed out
2020/01/29 11:03:58:621 [2386325(dns proxy resolve queue)]  (DnsLogRecordsWriter) handle event error occured - Upstream failed to perform dns query: Timed out
2020/01/29 11:04:00:636 [2386325(dns proxy resolve queue)]  (DnsLibs) [2020-01-29 11:04:00.635992] [2386325] [DNS forwarder] [debug] [56833] handle_message Upstream failed to perform dns query: Timed out
2020/01/29 11:04:00:638 [2386325(dns proxy resolve queue)]  (DnsLibs) [2020-01-29 11:04:00.637888] [2386325] [DNS forwarder] [debug] [56833] Server failure response:
;; ->>HEADER<<- opcode: QUERY, rcode: SERVFAIL, id: 56833
;; flags: qr rd ra ; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0 
;; QUESTION SECTION:
;; client-api.itunes.apple.com.	IN	AAAA
;; ANSWER SECTION:
;; AUTHORITY SECTION:
;; ADDITIONAL SECTION:
;; Query time: 0 msec
;; WHEN: Thu Jan  1 03:00:00 1970
;; MSG SIZE  rcvd: 0

Whitelisted requests are not in query log

[Android] Add setters to Android UpstreamSettings and DnsProxySettings

Please, add setUpstreams, setBootstrap etc methods

Add helper method to check DNS upstream availability and parse SDNS stamps

Please, add helper methods by analogy with dnsproxy

Add upstreams sorting by RTT

Please, add upstreams sorting (by analogy with dnsproxy)

It may be useful in the following case:

There are several upstreams specified by the user. For example:

1.2.3.4
5.6.7.8
9.10.11.12
1.1.1.1

The first three are not valid and the user will wait for the timeout of each of them for each request.

Upstream 1.1.1.1 will get to the beginning after sorting from fast to slow.

Simple DNS cache

Support building combined library for iOS and iPhone simulator

For a convenience of ios development. For now the different libraries need to be built.

Update README -- add documentation

request blocking does not work correctly

whitelist rules must have higher priority in any case

how to reproduce:

add blocking rule "||datamind.ru^"
add whitelist rule "@@||sync.datamind.ru^|"
request "sync.datamind.ru"

expected result:
receive an answer with IP address

actual result:
receive an answer with NXDOMAIN

[Android] Add original response to DnsRequestProcessedEvent if request was filtered by CNAME

Please, add original response to DnsRequestProcessedEvent if request was filtered by CNAME.
We need to show the user why this request was filtered.

Update all the existing loggers on changing of the logging method

For now setting a new logging callback doesn't change the callbacks of the already registered loggers

Always reply to request, even when upstream timed out

To prevent temporary DNS server ban by iOS in split-tunnel mode.
SERVFAIL is good for this.

DnsLibs NETunnelProvider build

Domain 'www.example.org' matched against rule '|example.org^'

how to reproduce:

add rule ' |example.org^' to domains filter
request 'www.example.org'

expected result:
dnslibs returns the correct answer

actual result:
dnslibs returns NXDOMAIN

Unit-tests and CI

[Android] Add constructor for DnsRequestProcessedEvent

Please, add constructor for DnsRequestProcessedEvent.
We need this part for statistics calculation tests.

Add the option to select blocking type

Please, add ability to select blocking type by analogue with go-dnsproxy and AG Home

There are several options:

Default
||example.com^ rule should block both A and AAAA requests to example.com with NXDomain
0.0.0.0 example.com rule should block both A and AAAA requests to example.com with unspecified ip (0.0.0.0 or [::]) response
1.2.3.4 example.com should be used only for A requests to example.com
NXDomain
||example.com^, 0.0.0.0 example.com should block both A and AAAA requests with NXDomain
Unspecified IP
||example.com^, 0.0.0.0 example.com should block both A and AAAA requests with unspecified ip (0.0.0.0 or [::]) response

Dnscrypt client

https://github.com/ameshkov/dnscrypt

https://github.com/NLnetLabs/unbound/tree/master/dnscrypt

Refactor bootstrapper

There should be convenient way to create bootstrapper from url (?) and options (including ipv6 availability)
Both DoT and DoH should use bootstrapper.get() for every request.
For DoH there should be a method to automatically convert it to curl "resolved" entry passed to CURLOPT_RESOLVE.
m_round_robin should not advance automatically.
New method to report that server got from bootstrapper doesn't work. This advances m_round_robin.
DoH is creating bootstrapper itself. DoT doesn't.
Please make bootstrap passable to DoH or make DoT create bootstrapper too.
ipv6_avail to dnsproxy_settings

DOT client

Connection pooling
TLS
etc

Make sure that EDNS is properly handled inside proxy

dnsfilter library

Memory-efficient
Must support hosts files syntax
Supports a limited set of Adblock-style modifiers: badfilter, important
"Just domains" syntax

It makes sense to take something from this library:
https://github.com/AdguardTeam/urlfilter

Test filter lists:

Match CNAME records against the blocklists

See AGH feature:
AdguardTeam/AdGuardHome#1185

Here's what we should do:

Match CNAME records against the blocklists

Indicate in the query log that the response was blocked by matching a CNAME. It'd be ideal if we print the original response there as well.

Please note, that if there's an exception rule matching the DNS query's question, you should not check the CNAME.

add filter Lenny_filter.txt
to dns proxy
in logs you can see [debug] unique domains table size: 2 (instead of 3)
try to resolve host 'yandex.ru' and get NXDOMAIN as a result - it is good
try to resolve host 'meduza.io'

expected result:
NXDOMAIN

actual result:
resolved ip address

adguardteam / dnslibs Goto Github PK

dnslibs's Introduction

AdGuard C++ DNS libraries

Build instructions

Native library

Prerequisites

Building

MacOS/iOS framework

Prerequisites

Building

Useful notes

Testing changes as a dependency

License

dnslibs's People

Contributors

Stargazers

Watchers

Forkers

dnslibs's Issues

Recommend Projects

Recommend Topics

Recommend Org