linkerd / linkerd-zipkin Goto Github PK

linkerd telemeter plugins for writing trace data to zipkin

Scala 86.28% Shell 12.73% Dockerfile 0.99%

linkerd-zipkin's Introduction

linkerd-zipkin

This repo contains linkerd telemeters for writing tracing data to Zipkin, using linkerd's plugin interface and Zipkin's finagle-zipkin library.

Building

Plugin jar

To build a plugin jar, run:

$ ./sbt assembly

That will build a linkerd-zipkin plugin jar in plugins/. Put that jar in linkerd's class path to make the telemeters available. Further information on installing plugins is available in the linkerd plugin documentation.

Docker

If you're using Docker, this repo provides a Dockerfile that you can use to layer in the plugin jar on top of the linkerd base image. To build, run:

$ ./sbt assembly
$ docker build -t linkerd-zipkin:latest .

That will build a docker image called linkerd-zipkin:latest, which runs linkerd and autoloads the telemeters from this repo. You can tag that image as needed and push to your own docker repo for deployment.

Usage

Two telemeters are provided, for writing to Zipkin with different transports.

io.zipkin.http

This telemeter writes tracing data to zipkin over HTTP. Sample configuration:

telemetry:
- kind: io.zipkin.http
  host: localhost:9411
  initialSampleRate: 0.02

Available configuration options for this telemeter:

Option	Default	Description
`host`	localhost:9411	The network location of the Zipkin http service.
`hostHeader`	zipkin	The Host header used when sending spans to Zipkin
`compressionEnabled`	true	True implies that spans will be gzipped before transport
`initialSampleRate`	0.001 (0.1%)	Percentage of traces to sample in the range [0.0 - 1.0]

io.zipkin.kafka

This telemeter writes tracing data to zipkin using Kafka. Sample configuration:

telemetry:
- kind: io.zipkin.kafka
  bootstrapServers: localhost:9092
  initialSampleRate: 0.02

Available configuration options for this telemeter:

Option	Default	Description
`bootstrapServers`	localhost:9092	Initial set of kafka servers to connect to (comma separated)
`topic`	zipkin	Kafka topic zipkin traces will be sent to
`initialSampleRate`	0.001 (0.1%)	Percentage of traces to sample in the range [0.0 - 1.0]

linkerd-zipkin's People

Contributors

Stargazers

Watchers

Forkers

aschepis 1hanymhajna dev25 uswitch sozuuuuu negz joeyb cchatfield peatio abhinaysai foxbroadcasting aminla dtacalau laashub-soa isabella232

linkerd-zipkin's Issues

Add Tags to Tracing

Trying to add additional tags in traces sent using this module. Would like to send host ip as well as potentially other tags. Is this supported?

Release for linkerd 1.5.0

I'd like to try out linkerd 1.5.0, but I use this plugin. I attempted to upgrade the plugin to Finagle 18.9.1, but doing so seems to require upgrading to io.zipkin.finagle2, which I don't currently have the bandwidth to figure out.

Publish to Maven central

zipkin-http error: name resolution is negative

Curious if anyone here had insight into this error. I am seeing it roughly every 5 seconds on instances where it is showing up.

THREAD130: zipkin-http: name resolution is negative (local dtab: Dtab())
source: stderr
tag: linkerd/91a2f8a72676

We recently added linkerd-zipkin module to our env where we are running linkerd, namerd, consul.

Support TLS-encrypted communication

When using port 443, traces are never received from Zipkin. I suspect this is because the traces are still sent via HTTP even though the port is 443 and the server initiates a TLS handshake. It would be good to also be able to support custom CA certificates too, although that might be doable by simply installing them on the machine/container where linkerd is running (?).

How set replication-factor for zipkin topic

By default set replication-factor = 1, how to use this topic in cluster of kafka with 3 instance?

Thanks

Upgrade to finagle 7 / zipkin-finagle 1.1

Tracking ticket to upgrade this repo to finagle 7. Depends on openzipkin/zipkin-finagle#35 (already released) and linkerd/linkerd#1600 (not yet released).

Publish linkerd-zipkin Docker images

As a user of this plugin I end up having to build my own Docker image every time a new linkerd is released. It would be handy if you did that for me. :)

Wire up stats reporting for tracing telemeters

The finagle-zipkin library will export stats for the tracers that it builds, if you pass it a stats receiver when constructing the tracer. In the initial implementation, we're using NullStatsReceiver, but we should eventually hook those stats into the rest of the telemetry system.

Update linkerd-zipkin for linkerd 1.6.3

Publish a version of linkerd-zipkin for the linkerd 1.6.3 release

JAR uploaded to maven central does not include all dependencies

The JAR in maven central only seems to include the code from this repo, and does not have its dependencies bundled. This means that we have to build from source ourselves if we want to use this plugin. Is this intentional?

java.net.ConnectException: Connection refused

After starting use of this module we are seeing alot of java / kafka errors from linkerd.

java.net.ConnectException: Connection refused
WARN 0620 18:19:05.201 UTC kafka-producer-network-thread | producer-1: Error in I/O with localhost/127.0.0.1
at java.lang.Thread.run(Thread.java:748)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191)
E 0620 18:18:52.913 UTC THREAD36 TraceId:b34e5cf7b68e40ef: service failure: com.twitter.finagle.CancelledConnectionException

Config:
telemetry:

kind: io.zipkin.http
host: :9411
initialSampleRate: 0.02

Where Collector-Domain.com points at an AWS NLB and targets are jaeger-collector containers running in ECS.

This issue happens intermittently on hosts and seems to only be solved by a restart of linkerd. Any insight would be appreciated.

Plugin not compatible with Linkerd 1.3.0

linkerd-zipkin plugin has to be updated to make it compatible with Linkerd 1.3.0

	at java.lang.Class.newInstance(Class.java:442)
	at com.twitter.app.LoadService$.$anonfun$apply$7(LoadService.scala:73)
	at scala.collection.TraversableLike.$anonfun$flatMap$1(TraversableLike.scala:241)
	at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:59)
	at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:52)
	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
	at scala.collection.TraversableLike.flatMap(TraversableLike.scala:241)
	at scala.collection.TraversableLike.flatMap$(TraversableLike.scala:238)
	at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
	at com.twitter.app.LoadService$.apply(LoadService.scala:62)
	at com.twitter.finagle.util.LoadService$.apply(LoadService.scala:14)
	at io.buoyant.linkerd.Linker$.LoadedInitializers$lzycompute(Linker.scala:65)
	at io.buoyant.linkerd.Linker$.LoadedInitializers(Linker.scala:64)
	at io.buoyant.linkerd.Linker$.parse$default$2(Linker.scala:79)
	at io.buoyant.linkerd.Main$.loadLinker(Main.scala:61)
	at io.buoyant.linkerd.Main$.main(Main.scala:31)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.twitter.app.App.$anonfun$nonExitingMain$3(App.scala:233)
	at scala.Option.foreach(Option.scala:257)
	at com.twitter.app.App.nonExitingMain(App.scala:232)
	at com.twitter.app.App.nonExitingMain$(App.scala:210)
	at io.buoyant.linkerd.Main$.nonExitingMain(Main.scala:20)
	at com.twitter.app.App.main(App.scala:198)
	at com.twitter.app.App.main$(App.scala:196)
	at io.buoyant.linkerd.Main$.main(Main.scala:20)
	at io.buoyant.linkerd.Main.main(Main.scala)
Caused by: java.lang.IllegalArgumentException: Multiple differing Toggle config resources found for com.twitter.finagle.netty4, jar:file:/io.buoyant/linkerd/1.3.0/bundle-exec!/com/twitter/toggles/configs/com.twitter.finagle.netty4.json, jar:file:/io.buoyant/linkerd/1.3.0/plugins/linkerd-zipkin-0.0.1.jar!/com/twitter/toggles/configs/com.twitter.finagle.netty4.json
	at com.twitter.finagle.toggle.StandardToggleMap$.selectResource(StandardToggleMap.scala:163)
	at com.twitter.finagle.toggle.StandardToggleMap$.loadJsonConfigWithEnv(StandardToggleMap.scala:180)
	at com.twitter.finagle.toggle.StandardToggleMap$.loadJsonConfig(StandardToggleMap.scala:122)
	at com.twitter.finagle.toggle.StandardToggleMap$.apply(StandardToggleMap.scala:100)
	at com.twitter.finagle.toggle.StandardToggleMap$.apply(StandardToggleMap.scala:85)
	at com.twitter.finagle.netty4.package$.&lt;init&gt;(package.scala:17)
	at com.twitter.finagle.netty4.package$.&lt;clinit&gt;(package.scala)
	... 56 more
Exception thrown in main on startup

Add support for scribe transport

The intial telemeters provided are http and kafka. For consistency with zipkin-finagle, we should consider adding scribe as well.

Adding a dependency on zipkin-finagle's scribe module, however, causes a version conflict with libthrift. To reproduce, first add the dependency:

diff --git a/build.sbt b/build.sbt
index acead10..264d03b 100644
--- a/build.sbt
+++ b/build.sbt
@@ -33,6 +33,7 @@ val `linkerd-zipkin` =
         twitterUtil("stats"),
         zipkin("http"),
         zipkin("kafka"),
+        zipkin("scribe"),
         scalatest() % "test"
       ),
       aggregate in assembly := false,

And then run ./sbt assembly. That produces a bunch of errors of the format:

[error] (*:assembly) deduplicate: different file contents found in the following:
[error] /Users/kl/.ivy2/cache/com.twitter/libthrift/jars/libthrift-0.5.0-7.jar:org/apache/thrift/EncodingUtils.class
[error] /Users/kl/.ivy2/cache/org.apache.thrift/libthrift/jars/libthrift-0.9.3.jar:org/apache/thrift/EncodingUtils.class