middleware: kubernetes about coredns HOT 18 CLOSED

See https://github.com/kubernetes/kubernetes/blob/master/cluster/addons/dns/kube2sky/kube2sky.go

from coredns.

@miekg what can I do to help with this?

from coredns.

I filed this issue after this twitter discussion: https://twitter.com/miekg/status/710937362885439488
Not really sure what should be done here...Register via CoreDNS? Maybe we should ask Brandon Philips again?

The linked code above mostly deals with adding entries to etcd that skydns/coredns can than return.

from coredns.

The big idea would be to skip etcd all together and have a layer that talks directly to the k8s APIs with some amount of caching. This would simplify the kubernetes DNS from etcd, kube2sky, and skydns to just coredns + some library code to talk to k8s.

from coredns.

@miekg is possible to interact with coredns using miekg/dns the RR interface? (using coredns as a library inside another program)
I started aledbf/kube-to-coredns as a POC for this integration. I know I can generate/update a zone file and use the reload feature (but I prefer to skip the file generation)

from coredns.

CoreDNS should allow for library inclusion, except I ripped stuff out while forking it from Caddy, YMMV. Although we can (and should!) put this code back in, if I ripped too much.

IOW: please try and see where it fails.

from coredns.

@aledbf but I thought you want this as a middleware in CoreDNS?

Also, I havent' play with Kubernetes at all, all I know from it is via SkyDNS bugs :P

from coredns.

@madhusudancs this is relevant to the kube DNS revamp discussion earlier today in the sig-federation meeting.

\cc @philips

from coredns.

@miekg this is a rewrite of skydns that removes etcd kubernetes/kubernetes#23930

from coredns.

That's just like skydns version 1 that used raft directly. So what does this middleware (for CoreDNS) need to do? I.e. must it also listen (and forward?) kubernetes api calls?

If skydns is being gutted to not use etcd is a backend then it seems much easier to port that over to coredns imo

from coredns.

cc @ArtfulCoder

from coredns.

@miekg I agree, you need to chat with @ArtfulCoder to see if there is an area of collaboration here.

from coredns.

There are a couple of approaches that could be taken:

1. modify coreDNS to remove the etcd datastore and instead query k8s for the relevant service data to respond to the DNS request,
2. modify coreDNS to leave the etcd datastore in place, but fail-over to querying k8s for the relevant service data if the etcd datastore does not hold records to respond to the DNS request,
3. fork/extend kube2sky to create a kube2core helper tool.

There are certainly other design approaches.

Option 1 potentially puts a lot of API load on k8s. Not sure if this will be a problem but it does make DNS response time dependent on k8s API performance.

Option 2 allows for coreDNS-level caching of the service data. This could mitigate the API load on k8s at the cost of needing some kind of expiration processing for the data cached at coreDNS-level.

Option 3 is potentially a drop-in replacement for kube2sky. The benefits being easy transition from current use of skydns to coredns, and having a separation between the component responsible for maintaining data consistency (between k8s <-> coredns) and the component serving the DNS protocol. The drawback is that the kube2sky setup is more complicated than option 1 or 2.

Any thoughts on these different approaches?

from coredns.

The ideal solution is that a backend is made for k8s instead of etcd. And a caching layer is introduced on top of that k8s backend so we don't have to make the HTTP API request on every DNS query.

from coredns.

I'm working on the assumption that the existing coreDNS caching middleware will work, or can be extended to work to decouple k8s API requests from the DNS requests.

I'm starting to hack on a middleware component to interface with the k8s API. Here is a draft of directives for the kubernetes API middleware (based on the etcd directives):

.:53 {
    kubernetes coredns.local {
               endpoint http://localhost:8080
               upstream 8.8.8.8:53 8.8.4.4:53
    }
    cache 160 coredns.local
    proxy . 8.8.8.8:53 8.8.4.4:53
}

The intention behind this draft is:

• use the k8s backend to resolve queries in the 'coredns.local' zone.
• use http://localhost:8080 as the kubernetes API endpoint.
• use '8.8.8.8:53' and '8.8.4.4:53' as recursive servers.
• cache results from the 'coredns.local' zone for 160 seconds.

There may be a need to provide a flag to mimic the skydns behavior of returning all records that are children of the query string.

My current focus is on coding up a middleware that issues a http request per DNS query. Caching is a second-order concern.

from coredns.

@mrichmon yes this makes sense. Abstracting away differences between etcd and the kubernetes middleware is of secondary concern. Although from the looks of it, it will be a lot of copy and pasting; in that regard maybe we should look at creating an abstraction? SkyDNS has this abstraction already, but no implementations.

The interface could be as simple as

type Backend interface {
   Records(name string, exact bool) ([]msg.Services, error)
}

No need to add ReverseRecords in there as well.

from coredns.

Basic PoC for kubernetes integration merged in PR: #153
This PoC provides A record lookup for services using the naming scheme: servicename.namespace.zone

Details in the middleware/kubernetes/README.md file.

from coredns.

Closing this as we have this.

from coredns.