Giter VIP home page Giter VIP logo

Comments (8)

gosharplite avatar gosharplite commented on May 25, 2024

I think the issue might be the communication between namerd and linkerd. (see linkerd issues #825 and PR #1005)

I have also tried interpreter io.l5d.namerd.http. It does not work.

      interpreter:
        kind: io.l5d.namerd.http
        experimental: true
        dst: /$/inet/namerd.default.svc.cluster.local/4180

Hope there is a way to check linkerd is talking to namerd successfully.
Any suggestion?

from linkerd-examples.

gosharplite avatar gosharplite commented on May 25, 2024

Logs of linkerd reveal namerd.default.svc.cluster.local can't be resolved.

$ ./kubectl logs -f l5d-07mxz -c l5d
I 0214 09:13:58.637 THREAD100: Failed to resolve namerd.default.svc.cluster.local. Error java.net.UnknownHostException: namerd.default.svc.cluster.local: Name or service not known
I 0214 09:13:58.638 THREAD100: Resolution failed for all hosts in ArraySeq((namerd.default.svc.cluster.local,4100,Map()))
I 0214 09:13:59.004 THREAD100: Failed to resolve namerd.default.svc.cluster.local. Error java.net.UnknownHostException: namerd.default.svc.cluster.local
I 0214 09:13:59.004 THREAD100: Resolution failed for all hosts in ArraySeq((namerd.default.svc.cluster.local,4100,Map()))
I 0214 09:13:59.023 THREAD100: Failed to resolve namerd.default.svc.cluster.local. Error java.net.UnknownHostException: namerd.default.svc.cluster.local
I 0214 09:13:59.023 THREAD100: Resolution failed for all hosts in ArraySeq((namerd.default.svc.cluster.local,4100,Map()))
I 0214 09:14:03.643 THREAD100: Failed to resolve namerd.default.svc.cluster.local. Error java.net.UnknownHostException: namerd.default.svc.cluster.local
I 0214 09:14:03.643 THREAD100: Resolution failed for all hosts in ArraySeq((namerd.default.svc.cluster.local,4100,Map()))
I 0214 09:14:04.014 THREAD100: Failed to resolve namerd.default.svc.cluster.local. Error java.net.UnknownHostException: namerd.default.svc.cluster.local
I 0214 09:14:04.014 THREAD100: Resolution failed for all hosts in ArraySeq((namerd.default.svc.cluster.local,4100,Map()))
I 0214 09:14:04.033 THREAD100: Failed to resolve namerd.default.svc.cluster.local. Error java.net.UnknownHostException: namerd.default.svc.cluster.local
I 0214 09:14:04.033 THREAD100: Resolution failed for all hosts in ArraySeq((namerd.default.svc.cluster.local,4100,Map()))

Using an ubuntu pod inside the k8s cluster, namerd.default.svc.cluster.local can be resolved.

$ ./kubectl exec -it ubuntu bash
root@ubuntu:/# nslookup namerd.default.svc.cluster.local
Server:		10.1.0.10
Address:	10.1.0.10#53

Name:	namerd.default.svc.cluster.local
Address: 10.1.234.160

This is interesting.

from linkerd-examples.

gosharplite avatar gosharplite commented on May 25, 2024

I find out containers in my daemonset pod can't resolve k8s service name due to hostNetwork: true. Need to figure out how to let them reach k8s dns server :)

from linkerd-examples.

gosharplite avatar gosharplite commented on May 25, 2024

A quick way to resolve this issue is to use nodePort of namerd in linkerd config file:)

from linkerd-examples.

wmorgan avatar wmorgan commented on May 25, 2024

@gosharplite glad you resolved it!

from linkerd-examples.

pvcnt avatar pvcnt commented on May 25, 2024

Hi @gosharplite, I think I stumbled into the same issue. After digging, I eventually found out about the whole hostNetwork thing, but could you be a little bit more specific about "using nodePort of namerd in linkerd config"?

Thanks!

from linkerd-examples.

pvcnt avatar pvcnt commented on May 25, 2024

Okay, I figured out by myself, I obviously had to define a NodePort service for namerd. However, I do not find this solution very elegant...

from linkerd-examples.

ethanrubio avatar ethanrubio commented on May 25, 2024

@adleong mentioned on slack that I could use #/io.l5d.k8s/<ns>/<port>/<service> dst for namerd in the linkerd configmap but when I try to do a request to the http-outgoing namespace I get an error of

No hosts are available for /#/io.l5d.k8s/service-mesh/thrift/namerd, Dtab.base=[], Dtab.local=[]. Remote Info: Not Available

I checked namerd to see if it would resolve this path correctly, and it does.

screen shot 2017-08-10 at 12 23 56 am

I even tried using the nodePort that's exposed when namerd is brought up i.e. /$/inet/namerd.service-mesh.svc.cluster.local/30109 but I still get a fail

I 0810 07:03:09.065 UTC THREAD49: Resolution failed for all hosts in ArraySeq((namerd.service-mesh.svc.cluster.local,30109,Map()))
I 0810 07:03:09.097 UTC THREAD49: Failed to resolve namerd.service-mesh.svc.cluster.local. Error java.net.UnknownHostException: namerd.service-mesh.svc.cluster.local

Here's my config for linkerd and namerd. I've taken out TLS, circuit breaking, and retries to simplify things as well.

# linkerd
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: l5d-config
  namespace: service-mesh
data:
  config.yaml: |-
    admin:
      port: 9990

    telemetry:
    - kind: io.l5d.prometheus
    - kind: io.l5d.recentRequests
      sampleRate: 0.25
    - kind: io.l5d.zipkin
      host: zipkin-collector.default.svc.cluster.local
      port: 9410
      sampleRate: 1 # change to 0.01 for production

    usage:
      orgId: headspace

    routers:
    - protocol: http
      label: http-outgoing
      interpreter:
        kind: io.l5d.namerd
        dst: /#/io.l5d.k8s/service-mesh/thrift/namerd
        namespace: http-outgoing
      servers:
      - port: 4140
        ip: 0.0.0.0

    - protocol: http
      label: http-incoming
      interpreter:
        kind: io.l5d.namerd
        dst: /#/io.l5d.k8s/service-mesh/thrift/namerd
        namespace: http-incoming
        transformers:
        - kind: io.l5d.k8s.localnode
          hostNetwork: true
      servers:
      - port: 4141
        ip: 0.0.0.0

    - protocol: h2
      label: grpc-outgoing
      experimental: true
      identifier:
        kind: io.l5d.header.path
        segments: 1
      interpreter:
        kind: io.l5d.namerd
        dst: /#/io.l5d.k8s/service-mesh/thrift/namerd
        namespace: grpc-outgoing
      servers:
      - port: 4340
        ip: 0.0.0.0

    - protocol: h2
      label: grpc-incoming
      experimental: true
      identifier:
        kind: io.l5d.header.path
        segments: 1
      interpreter:
        kind: io.l5d.namerd
        dst: /#/io.l5d.k8s/service-mesh/thrift/namerd
        namespace: grpc-incoming
        transformers:
        - kind: io.l5d.k8s.localnode
          hostNetwork: true
      servers:
      - port: 4341
        ip: 0.0.0.0

    - protocol: http
      label: http-ingress
      identifier:
        kind: io.l5d.path
        segments: 1
        consume: false
      interpreter:
        kind: io.l5d.namerd
        dst: /#/io.l5d.k8s/service-mesh/thrift/namerd
        namespace: http-ingress
      servers:
      - port: 80
        ip: 0.0.0.0
---
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  labels:
    app: l5d
  name: l5d
  namespace: service-mesh
spec:
  template:
    metadata:
      labels:
        app: l5d
    spec:
      hostNetwork: true
      serviceAccountName: linkerd
      volumes:
      - name: l5d-config
        configMap:
          name: "l5d-config"
      - name: certificates
        secret:
          secretName: certificates
      containers:
      - name: l5d
        image: buoyantio/linkerd:1.1.3
        env:
        - name: POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: NODE_NAME
          valueFrom:
            fieldRef:
              fieldPath: spec.nodeName
        args:
        - /io.buoyant/linkerd/config/config.yaml
        ports:
        - name: http-outgoing
          containerPort: 4140
          hostPort: 4140
        - name: http-incoming
          containerPort: 4141
        - name: grpc-outgoing
          containerPort: 4340
          hostPort: 4340
        - name: grpc-incoming
          containerPort: 4341
        - name: http-ingress
          containerPort: 80
        - name: admin
          containerPort: 9990
        volumeMounts:
        - name: "l5d-config"
          mountPath: "/io.buoyant/linkerd/config"
          readOnly: true
        - name: "certificates"
          mountPath: "/io.buoyant/linkerd/certs"
          readOnly: true
      - name: kubectl
        image: buoyantio/kubectl:v1.6.2
        args:
        - "proxy"
        - "-p"
        - "8001"
---
apiVersion: v1
kind: Service
metadata:
  name: l5d
  namespace: service-mesh
  annotations:
    prometheus.io/scrape: 'true'
    prometheus.io/path: '/admin/metrics/prometheus'
    prometheus.io/port: '9990'
spec:
  selector:
    app: l5d
  type: LoadBalancer
  ports:
  - name: http-outgoing
    port: 4140
  - name: http-incoming
    port: 4141
  - name: grpc-outgoing
    port: 4340
  - name: grpc-incoming
    port: 4341
  - name: http-ingress
    port: 80
  - name: admin
    port: 9990

# namerd
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: namerd-config
  namespace: service-mesh
data:
  config.yml: |-
    admin:
      port: 9990

    namers:
    - kind: io.l5d.k8s
    - kind: io.l5d.k8s
      prefix: /io.l5d.k8s.http
      transformers:
      - kind: io.l5d.k8s.daemonset
        namespace: service-mesh
        port: http-incoming
        service: l5d
        hostNetwork: true
    - kind: io.l5d.k8s
      prefix: /io.l5d.k8s.grpc
      transformers:
      - kind: io.l5d.k8s.daemonset
        namespace: service-mesh
        port: grpc-incoming
        service: l5d
        hostNetwork: true
    - kind: io.l5d.rewrite
      prefix: /portNsSvcToK8s
      pattern: "/{port}/{ns}/{svc}"
      name: "/k8s/{ns}/{port}/{svc}"

    storage:
      kind: io.l5d.k8s
      host: localhost
      port: 8001
      namespace: default

    interfaces:
    - kind: io.l5d.thriftNameInterpreter
      ip: 0.0.0.0
      port: 4100
    - kind: io.l5d.httpController
      ip: 0.0.0.0
      port: 4180
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: namerd
  namespace: service-mesh
spec:
  replicas: 3
  template:
    metadata:
      labels:
        app: namerd
    spec:
      serviceAccountName: namerd
      dnsPolicy: ClusterFirst
      volumes:
      - name: namerd-config
        configMap:
          name: namerd-config
      containers:
      - name: namerd
        image: buoyantio/namerd:1.1.3
        args:
        - /io.buoyant/namerd/config/config.yml
        ports:
        - name: thrift
          containerPort: 4100
        - name: http
          containerPort: 4180
        - name: admin
          containerPort: 9990
        volumeMounts:
        - name: "namerd-config"
          mountPath: "/io.buoyant/namerd/config"
          readOnly: true
      - name: kubectl
        image: buoyantio/kubectl:v1.6.2
        args:
        - "proxy"
        - "-p"
        - "8001"
---
apiVersion: v1
kind: Service
metadata:
  name: namerd
  namespace: service-mesh
spec:
  selector:
    app: namerd
  type: LoadBalancer
  ports:
  - name: thrift
    port: 4100
  - name: http
    port: 4180
  - name: admin
    port: 9990

I got on this path because I had issues with getting the NODE_NAME env to work. I previously did not have hostName=true and I was able to send requests to the http-outgoing namespace to by using

http_proxy=http://l5d.service-mesh.svc.cluster.local:4140

But obviously this wasn't a good solution, so I want to do this properly. Thanks for all of your help!

from linkerd-examples.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.