dnsimple / erldns Goto Github PK

Compiled src/erldns_packet_cache.erl
src/erldns_zone_parser.erl:139: referring to built-in type set as a remote type; please take out the module name
src/erldns_zone_parser.erl:139: referring to built-in type set as a remote type; please take out the module name
ERROR: compile failed while processing /home/basanta/dnsSoftware/erl-dns: rebar_abort

I can't tell where cowboy or hottub are being used. Are they?

Right now do_fetch_zone will always respect the digest. Provide a means to force it to ignore the digest.

It is possible, in some circumstances, for erl-dns to receive a UDP query from the address and port it's listening on. (See https://jira.mesosphere.com/browse/DCOS_OSS-2109 for debugging and discussion.) When this happens, erl-dns treats its own reply as an incoming message and responds to it in an infinite loop. Even if the conditions for an accidental loop are not met, this is still a potential denial of service vector.

I believe an appropriate solution would be to ignore any messages that appear to originate from the listening socket.

Am attempting to embed erl-dns in another project, which uses a later version of cowboy than the specified 0.8.6. I forked erldns and updated the cowboy dependency, but while it passes test.sh am unable to connect to the server (this is the only difference. I did update to run on R17, and that worked).

Any advice? Dunno if it's a trivial change or not.

erldns_server_sup for some reason is started not from supervision tree. This makes erldns not crash, but run defunctional when servers crash for some reason. Can I fix it or there are some reason I've missed?

Make sure that record field types are correct.

Check if a CNAME record with a uppercase name part is handled properly.

I believe there are bugs in that code and it should be unit-tested and corrected.

Here: https://github.com/aetrion/erl-dns/blob/master/src/erldns_zone_cache.erl#L91 -- if erl-dns is returning the authority for a TLD that it's hosting, it's problematic and it will return zone_not_found if it's hosting a single-level domain. Is there a reason for this, or can we get rid of it?

Hi,
I've been digging through the code and examples and have a question I still came up bank with. Let me start with what I try to achive:

I am writing a cloud orchestration system (https://github.com/project-fifo) and I currently am trying to allow DNS queries that dynamically resolve to machines created in FiFo. so basically the logic I'm looking for is in pseudo code:

if cahced(query)
then
  cached_reply()
else
  if lookup_in_fifo()
  then
    cache_loopup(),
    reply_result()
  else
     not_found()
  end
end

I've tried to put in a custom handler but I don't find the logic of when those handlers (like the example handler) is called, it seems that a zone entry needs to exist in some way for it even to be called.

So far I've found out that:

• a zone file needs to exist with a SOA record so the query isn't shut down right away
• the query for a non existing record goes to the erldns_resolver and ends up in resolve_best_match_referral never reaching the custom handlers.

the zone file I tried it with is:

[
    {
        "name": "fifo.cloud",
        "records": [
            {
                "name": "fifo.cloud",
                "type": "SOA",
                "ttl": 3600,
                "data": {
                    "mname": "ns1.fifo.cloud",
                    "rname": "admin.fifo.cloud",
                    "serial": 2013022001,
                    "refresh": 86400,
                    "retry": 7200,
                    "expire": 604800,
                    "minimum": 300
                }
            },
            {
                "name": "fifo.cloud",
                "type": "A",
                "ttl": 3600,
                "data": {
                    "ip": "1.2.3.4"
                }
            },
            {
                "name": "fifo.cloud",
                "type": "AAAA",
                "ttl": 3600,
                "data": {
                    "ip": "2001:6A8:0:1:210:4BFF:FE4B:4C61"
                }
            },
            {
                "name": "fifo.cloud",
                "type": "NS",
                "ttl": 3600,
                "data": {
                    "dname": "ns1.fifo.cloud"
                }
            }
        ]
    }
]

and the query I ran is something like:

dig -p8053 @127.0.0.1 my-host.fifo.cloud a

./build.sh: 4: ./build.sh: ./rebar: not found
./build.sh: 5: ./build.sh: ./rebar: not found
./build.sh: 6: ./build.sh: ./rebar: not found

Should I change this to rebar3 or is there something else I may be missing for the install process?

When starting erldns under OTP 21 using an erldns.config.example configuration and the run.sh script, erldns crashes with:

=CRASH REPORT==== 10-Sep-2018::09:47:06.215049 ===
  crasher:
    initial call: lager_handler_watcher:init/1
    pid: <0.113.0>
    registered_name: []
    exception exit: noproc
      in function  gen:do_for_proc/2 (gen.erl, line 228)
      in call from gen_event:rpc/2 (gen_event.erl, line 239)
      in call from lager_handler_watcher:install_handler2/3 (/Users/anthonyeden/d/dnsimple/erldns/_build/default/lib/lager/src/lager_handler_watcher.erl, line 117)
      in call from lager_handler_watcher:init/1 (/Users/anthonyeden/d/dnsimple/erldns/_build/default/lib/lager/src/lager_handler_watcher.erl, line 51)
      in call from gen_server:init_it/2 (gen_server.erl, line 374)
      in call from gen_server:init_it/6 (gen_server.erl, line 342)
    ancestors: [lager_handler_watcher_sup,lager_sup,<0.98.0>]
    message_queue_len: 0
    messages: []
    links: [<0.101.0>]
    dictionary: []
    trap_exit: false
    status: running
    heap_size: 987
    stack_size: 27
    reductions: 259
  neighbours:

The readme provides misleading info that zones can be loaded either from JSON file or zone server. The zone server functionality seems to be removed.

There is currently a packet cache, however this requires an exact match for the question and additional parts of the original packet. This issue is a proposal for a new cache after the packet cache that can look at parts of the question and additional and provide a cached response. This cache would likely be short-lived like the packet cache, although the TTL of the response cache could be adjustable.

Once GH-25 is merged create a Riak storage implementation.

This is a potential optimization that would allow erl-dns to handle a higher number of QPS by having multiple pre-forked Erlang processes listening on the same UDP port.

How to add ptr record for domain?

(Using the example config) if I don't send a DNS request in the first couple of seconds, the server doesn't accept any and they time out. If I do, it works fine.

I think the pools don't have enough workers in the example config or something.

Any ideas?

Thanks,

When staring up and loading the zones (from the example json), we get

2018-10-29 10:28:32.395 [warning] <0.155.0>@erldns_zone_parser:json_record_to_erlang:471 Unsupported record (data: [undefined,undefined,undefined,undefined,undefined])
2018-10-29 10:28:32.395 [warning] <0.155.0>@erldns_zone_parser:json_record_to_erlang:471 Unsupported record (data: [undefined,undefined,undefined,undefined,undefined])

warnings in the debug log.

If this is expected, it is potentially confusing for new people running this. Could be worth either removing these or determing which records they are and supporting them.

Implement storage abstraction for Aetrion branch.

There appears to be an issue with packet cache sweeping in the erldns_storage_json module. Specifically I have noticed items in the packet cache that seem to be more than 2 weeks old whereas they should have a much shorter lifetime (1 minute or less).

At this point I'm not sure why items are not being swept properly - more investigation is necessary. Specifically I plan on executing the MatchSpec against a running instance packet_cache ets table to see what keys are returned during the sweep.

Why is there a dependency to a UX tool observer in the app file? It makes it painful to use erldns in a headless environment that lacks observer as rebar3 as prod release will barf on the missing application. Am I missing something or am I doing something wrong? Thanks.

I observed the following errors occurring today in a production system (name and IPs anonymized).

2018-06-26 21:20:54.277 [error] <0.998.0>@erldns_worker:handle_decoded_udp_message:147 Worker process crashed (error: exit, reason: {noproc,{gen_server,call,[<0.1000.0>,{process,{dns_message,47717,false,0,false,false,false,false,false,false,0,1,0,0,1,[{dns_query,<<"example.com">>,1,28}],[],[],[{dns_optrr,512,0,0,true,[]}]},#Port<0.1551>,52805,{udp,{1,2,3,4}}},500]}})

This appears to be a case where a worker proc either failed to start properly or failed after starting. The worker process manager should clean up the worker list state in this case, removing the ID for the dead worker and starting a new worker.

Consider adding an example of how to run it by the files generated through rebar?

For DANE

Hello,

We have installed erldns for testing purposes with around 100 zone json files. Is there any way we can indicate to include all *.zone.json files from priv directory as zones in erldns.config?

Best Regards,

Malkhaz

I was looking at erldns_zone_cache:put_zone (https://github.com/aetrion/erl-dns/blob/master/src/erldns_zone_cache.erl#L189-L199), and it seems like the errors that come out of erldns_storage are suppressed. Is there a reason not to return the error code from the backend?

https://tools.ietf.org/html/rfc7553

Useful for resolving a hostname to an alternate URI. Hopefully someday browser vendors will support this for URL redirection.

Have tested running under R17.1. Added line to rebar.config to support this:

+{require_otp_vsn, "R16|17"}.

Some domains are returning warnings as we use a TTL for the RRSIG record that doesn't match the TTL of the record itself.

If a zone fails to parse correctly then fail for the entire zone. Keep track of which zones failed this way and return SERVFAIL for them.

It appears that TCP responses are truncating when the EDNS0 UDP size is specified. TCP responses should never be truncated.

Your nameservers are returning cached results without paying attention to EDNS options, EDNS version or EDNS flags in present in the query which should be eliciting different results. Additionally your servers are incorrectly echoing unknown EDNS options leading to DNS COOKIE mismatches (RFC 7873).

If the server behind the cache correctly implemented EDNS this would be less of a issue as it would be ignoring unknown EDNS options and EDNS flags and presenting similar answers independent of the EDNS options and flags in the query (RFC 6891). The cache needs to be EDNS version aware so it can correctly perform EDNS version negotiation.

The DNS can only work correctly if both DNS clients and DNS servers follow the DNS protocol. Please contact your DNS vendor for a fix.

Mark

% dig careerhub.com.au. @50.31.242.53 soa +edns=1 +noednsneg ; dig careerhub.com.au. @50.31.242.53 soa
;; Warning: Client COOKIE mismatch

; <<>> DiG 9.11.0a3 <<>> careerhub.com.au. @50.31.242.53 soa +edns=1 +noednsneg
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 13189
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 34e53df9ea31e66a (bad)
;; QUESTION SECTION:
;careerhub.com.au.              IN      SOA

;; ANSWER SECTION:
careerhub.com.au.       3600    IN      SOA     ns1.careerhub.com.au. admin.dnsimple.com. 1438314364 86400 7200 604800 300

;; Query time: 184 msec
;; SERVER: 50.31.242.53#53(50.31.242.53)
;; WHEN: Fri Jun 24 11:30:24 EST 2016
;; MSG SIZE  rcvd: 115

;; Warning: Client COOKIE mismatch

; <<>> DiG 9.11.0a3 <<>> careerhub.com.au. @50.31.242.53 soa
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 25504
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 1, flags:; udp: 4096
; COOKIE: 4448cbf37dd43b07 (bad)
;; QUESTION SECTION:
;careerhub.com.au.              IN      SOA

;; ANSWER SECTION:
careerhub.com.au.       3600    IN      SOA     ns1.careerhub.com.au. admin.dnsimple.com. 1438314364 86400 7200 604800 300

;; Query time: 182 msec
;; SERVER: 50.31.242.53#53(50.31.242.53)
;; WHEN: Fri Jun 24 11:30:24 EST 2016
;; MSG SIZE  rcvd: 115

Followed the instructions and this is all the output it prints out:

Erlang/OTP 19 [erts-8.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V8.1  (abort with ^G)
1>
=INFO REPORT==== 22-Oct-2016::03:43:42 ===
    application: mnesia
    exited: stopped
    type: temporary

dig times out:

dig -p8053 @127.0.0.1 example.com a

; <<>> DiG 9.9.5-9+deb8u7-Debian <<>> -p8053 @127.0.0.1 example.com a
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached

Any interest in adding a "Powers Mesosphere DC/OS" in the repo? You can if you want to.

FQDN with uppercase letters in the A record's leftmost label resulted in only the last record being used.

Currently most of rebar dependencies in rebar.config do not have specified revision nor tag. Therefore sudden change in any of those dependencies might brake the erl-dns package and any other packages using it. Please specify those versions in every dependency.

Currently both TCP and UDP ANY queries are throttled. Remove the TCP throttling since it cannot be spoofed in the same fashion and thus does not pose the same risk.

In other words, if we are not authoritative for a zone, then return REFUSED instead of SERVFAIL

Mnesia implementation is broken for two methods (that I've tested).
erldns_zone_cache:zone_names_and_versions()
erldns_zone_cache:get_zone_records()

Steps to reproduce:

git clone [email protected]:dnsimple/erldns.git
cp erldns.config.example erldns.config

Add to erldns.config:

{erldns, 
    {storage, [
          		{type, erldns_storage_mnesia},
	        	{dir, "."}
		    ]
    },
    ....Default config
},
{mnesia, [{dir, "."}]}

Run:

rebar3 shell

Query:

dig @127.0.0.1 -p 8053 example.com

 <<>> DiG 9.16.1-Ubuntu <<>> @127.0.0.1 -p 8053 example.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 26391
;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 3993fc8d0b301228 (echoed)
;; QUESTION SECTION:
;example.com.			IN	A

;; AUTHORITY SECTION:
example.com.		300	IN	SOA	ns1.example.com. admin.example.com. 2013022001 86400 7200 604800 300

;; Query time: 3 msec
;; SERVER: 127.0.0.1#8053(127.0.0.1)
;; WHEN: Tue Jun 09 11:19:42 CEST 2020
;; MSG SIZE  rcvd: 98

See issue in shell

zone_names_and_versions

1> erldns_zone_cache:zone_names_and_versions().
** exception exit: {aborted,function_clause}
     in function  mnesia:wrap_trans/6 (mnesia.erl, line 497)

How to fix

Edit erldns_zone_cache.erl

From:

zone_names_and_versions() ->
  erldns_storage:foldl(fun({_, Zone}, NamesAndShas) -> NamesAndShas ++ [{Zone#zone.name, Zone#zone.version}] end, [], zones).

To:

zone_names_and_versions() ->
  erldns_storage:foldl(fun(Zone, NamesAndShas) -> NamesAndShas ++ [{Zone#zone.name, Zone#zone.version}] end, [], zones).

rebar3 shell
1> erldns_zone_cache:zone_names_and_versions().
[{<<"example.com">>,[]}]

This breaks the JSON version though so not ideal.

get_zone_records:

4> erldns_zone_cache:get_zone_records(<<"example.com">>).     
** exception exit: {aborted,{bad_type,[{{{<<"example.com">>,'_'},'$1'},
                                        [],
                                        ['$$']}]}}
     in function  mnesia:wrap_trans/6 (mnesia.erl, line 497)
     in call from erldns_zone_cache:get_zone_records/1 (/home/nisbus/code/osource/erldns_clean/src/erldns_zone_cache.erl, line 153)

Not sure how to fix this one

Thanks

We recently saw an issue where the configured IP isn't available and instead of continuing past this and initializing the other IPs and Ports or not booting at all it skips all this and comes up with no opened ports.
This was also not written to the crash log. we captured this error in bugsnag in the info.log

2018-04-04 22:08:52.396 [info] <0.1019.0>@erldns_event_handler:handle_event:38 Starting the UDP and TCP supervisor
2018-04-04 22:08:52.396 [info] <0.1029.0>@erldns_udp_server:start:111 Starting UDP server for inet on address {1,1,1,1} and port 53 (sockopts: [{raw,1,15,<<1,0,0,0>>}])
2018-04-04 22:08:52.397 [info] <0.985.0>@bugsnag:process_trace:100 Processing trace [{erldns_udp_server,start,4,[{file,"src/erldns_udp_server.erl"},{line,112}]},{erldns_udp_server,init,1,[{file,"src/erldns_udp_server.erl"},{line,72}]},{gen_server,init_it,2,[{file,"gen_server.erl"},{line,365}]},{gen_server,init_it,6,[{file,"gen_server.erl"},{line,333}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,247}]}]
2018-04-04 22:08:52.397 [info] <0.985.0>@bugsnag:deliver_payload:123 Sending exception: <<"{\"apiKey\":\"ourkey\",\"notifier\":{\"name\":\"Bugsnag Erlang\",\"version\":\"1.0.0\",\"url\":\"https://github.com/aeden/bugsnag-erlang\"},\"events\":[{\"payloadVersion\":\"2\",\"device\":{\"hostname\":\"host\"},\"app\":{\"releaseStage\":\"production\"},\"exceptions\":[{\"errorClass\":\"unknown\",\"message\":\"Supervisor erldns_server_sup had child undefined started with erldns_udp_server:start_link(udp_inet_anycast_4, inet, {1,1,1,1}, 53, [{raw,\\n                                                                             1,\\n                                                                             15,\\n                                                                             <<1,0,0,0>>}]) at undefined exit with reason no case clause matching {error,eaddrnotavail} in erldns_udp_server:start/4 in context start_error\",\"stacktrace\":[{\"file\":\"src/erldns_udp_server.erl\",\"lineNumber\":112,\"method\":\"start\"},{\"file\":\"src/erldns_udp_server.erl\",\"lineNumber\":72,\"method\":\"init\"},{\"file\":\"gen_server.erl\",\"lineNumber\":365,\"method\":\"init_it\"},{\"file\":\"gen_server.erl\",\"lineNumber\":333,\"method\":\"init_it\"},{\"file\":\"proc_lib.erl\",\"lineNumber\":247,\"method\":\"init_p_do_apply\"}]}]}]}">>

This is on the feature/dnssec branch.

Given a zone with:

*.example.com CNAME proxy.example.com
proxy.example.com A 1.2.3.4

And a query for foo.example.com, the response is being returned as

proxy.example.com CNAME proxy.example.com
proxy.example.com A 1.2.3.4

Whereas it should be:

foo.example.com CNAME proxy.example.com
proxy.example.com A 1.2.3.4

Is erl-dns supposed to have AXFR support? I see the erldns_axfr module, but no logic around it.

erl-dns doesn't check the qr flag in the DNS queries it receives, and thus treats response messages as if they were new queries. While this isn't necessarily a problem on its own, it can potentially multiply the impact of the looping problem in #72 by performing nontrivial query processing work instead of just rejecting the response message as bad.

I've been looking at the udp code and I have some suggestions.

Use: {recbuf,1024*1024} on gen_udp:open
The UDP socket receive buffer can fill up surprisingly quickly and the default is dangerously low.

Use {active,100} or similar for receiving packets. It improves throughput.

Use inet_udp:send instead of gen_udp:send. If gen_udp:send is used, inet:getaddrs/3 will be called on every send. This causes a needless erlang:start_timer and erlang:cancel_timer call.

When adding dns_erlang to an Elixir project, I get the following error at compile-time:

❯ DEBUG=1 mix deps.compile dns_erlang
===> Expanded command sequence to be run: []
===> Provider: {default,do}
===> Expanded command sequence to be run: [{default,app_discovery},
                                           {bare,compile}]
===> Provider: {default,app_discovery}
===> Not adding provider asn compile from module provider_asn1_compile because it already exists from module provider_asn1_compile
===> Not adding provider asn clean from module provider_asn1_clean because it already exists from module provider_asn1_clean
===> Not adding provider asn compile from module provider_asn1_compile because it already exists from module provider_asn1_compile
===> Not adding provider asn clean from module provider_asn1_clean because it already exists from module provider_asn1_clean
===> Provider: {bare,compile}
===> Compiling dns_erlang
===> Provider: {asn,compile}
===> run_hooks("/home/hecate/dev/splash/deps/dns_erlang", pre_hooks, compile) -> no hooks defined

===> run_hooks("/home/hecate/dev/splash/deps/dns_erlang", pre_hooks, erlc_compile) -> no hooks defined

===> erlopts [debug_info,inline,inline_list_funcs,{d,namespaced_types}]
===> files to compile ["/home/hecate/dev/splash/deps/dns_erlang/src/DNS-ASN1.erl",
                       "/home/hecate/dev/splash/deps/dns_erlang/src/dnssec.erl",
                       "/home/hecate/dev/splash/deps/dns_erlang/src/dns_record_info.erl",
                       "/home/hecate/dev/splash/deps/dns_erlang/src/dns_record.erl",
                       "/home/hecate/dev/splash/deps/dns_erlang/src/dns.erl"]
===> Compiling src/DNS-ASN1.erl failed
src/DNS-ASN1.erl:44: illegal pattern
src/DNS-ASN1.erl:45: variable 'Exception' is unbound
src/DNS-ASN1.erl:47: variable 'Stk' is unbound
src/DNS-ASN1.erl:49: variable 'Stk' is unbound
src/DNS-ASN1.erl:58: illegal pattern
src/DNS-ASN1.erl:59: variable 'Exception' is unbound
src/DNS-ASN1.erl:61: variable 'Stk' is unbound
src/DNS-ASN1.erl:63: variable 'Stk' is unbound

===> Compilation failed: {error,[["src/DNS-ASN1.erl:44: illegal pattern\n",
                                  "src/DNS-ASN1.erl:45: variable 'Exception' is unbound\n",
                                  "src/DNS-ASN1.erl:47: variable 'Stk' is unbound\n",
                                  "src/DNS-ASN1.erl:49: variable 'Stk' is unbound\n",
                                  "src/DNS-ASN1.erl:58: illegal pattern\n",
                                  "src/DNS-ASN1.erl:59: variable 'Exception' is unbound\n",
                                  "src/DNS-ASN1.erl:61: variable 'Stk' is unbound\n",
                                  "src/DNS-ASN1.erl:63: variable 'Stk' is unbound\n"]],

Rebar3 report
 version 3.6.1
 generated at 2019-10-18T20:39:56+00:00
=================
Please submit this along with your issue at https://github.com/erlang/rebar3/issues (and feel free to edit out private information, if any)
-----------------
Task: bare
Entered as:
  bare
-----------------
Operating System: x86_64-unknown-linux-gnu
ERTS: Erlang/OTP 22 [erts-10.4.3] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe]
Root Directory: /nix/store/c7hpif120y23m2z9f3r3hg1ffccns4ng-erlang-22.0.4/lib/erlang
Library directory: /nix/store/c7hpif120y23m2z9f3r3hg1ffccns4ng-erlang-22.0.4/lib/erlang/lib
-----------------
Loaded Applications:
bbmustache: 1.5.0
certifi: 2.0.0
cf: 0.2.2
common_test: 1.17.3
compiler: 7.4.2
crypto: 4.5.1
cth_readable: 1.4.2
dialyzer: 4.0.1
edoc: 0.11
erlware_commons: 1.2.0
eunit: 2.3.7
eunit_formatters: 0.5.0
getopt: 1.0.1
hipe: 3.19
inets: 7.0.8
kernel: 6.4.1
providers: 1.7.0
public_key: 1.6.7
relx: 3.26.0
sasl: 3.4
snmp: 5.3
ssl_verify_fun: 1.1.3
stdlib: 3.9.2
syntax_tools: 2.2
tools: 3.2

-----------------
Escript path: /home/hecate/.mix/rebar3
Providers:
  app_discovery as clean compile compile cover ct deps dialyzer do edoc escriptize eunit get-deps help install install_deps list lock new path pkgs release relup report shell state tar tree unlock update upgrade upgrade upgrade version xref