Comments (31)
We are building for Java 1.7 by default, is it possible for you to upgrade?
from kamon.
No, it's not possible. Do you recommend any other setup or tools for monitoring with Datadog?
from kamon.
Well, there is no particular reason why we are building for 1.7 other than being the default on our side.. I just published kamon 0.3.2-8bf93c1f26453ef298d965aaa8b9cb932f8b2a64
in our snapshots repository (http://snapshots.kamon.io) in case you want to give it a try, it was built with JDK 1.6 and should work fine for you.. we will make sure that our next releases are 1.6 compatible too, thanks for reporting!
from kamon.
Thanks a lot! I used this info: http://kamon.io/backends/datadog/ and I have a datadog agent running on my machine. Do you have a sample of the datadog.conf settings? I can't find anything with "kamon" in the metric explorer.
from kamon.
did you include the Datadog extension in your akka configuration as show bellow?
akka {
extensions = ["kamon.datadog.Datadog"]
}
from kamon.
Yes, I have. Here's my kamon config:
kamon {
datadog {
# Hostname and port in which your Datadog is running. Remember that Datadog packets are sent using UDP and
# setting unreachable hosts and/or not open ports wont be warned by the Kamon, your data wont go anywhere.
hostname = "127.0.0.1"
port = 8125
# Interval between metrics data flushes to Datadog. It's value must be equal or greater than the
# kamon.metrics.tick-interval setting.
flush-interval = 1 second
# Max packet size for UDP metrics data sent to Datadog.
max-packet-size = 1024 bytes
# Subscription patterns used to select which metrics will be pushed to Datadog. Note that first, metrics
# collection for your desired entities must be activated under the kamon.metrics.filters settings.
includes {
actor = [ "*" ]
trace = [ "*" ]
dispatcher = [ "*" ]
}
# Application prefix for all metrics pushed to Datadog. The default namespacing scheme for metrics follows
# this pattern:
# application.entity-name.metric-name
application-name = "kamon"
}
Default dispatcher for all Kamon components, unless a more specific one is configured.
default-dispatcher = "akka.actor.default-dispatcher"
metrics {
tick-interval = 1 second
# Time interval for recording values on all registered gauges.
gauge-recording-interval = 100 milliseconds
dispatchers {
# Dispatcher for periodical gauge value recordings.
gauge-recordings = ${kamon.default-dispatcher}
# Dispatcher for subscriptions and metrics collection actors.
metric-subscriptions = ${kamon.default-dispatcher}
}
filters = [
{
actor {
includes = [ "user/batch-statement-generation/*"]
excludes = []
}
},
{
trace {
includes = [ "*" ]
excludes = []
}
},
{
dispatcher {
includes = [ "default-dispatcher" ]
excludes = []
}
}
]
precision {
actor {
processing-time {
highest-trackable-value = 3600000000000
significant-value-digits = 2
}
time-in-mailbox {
highest-trackable-value = 3600000000000
significant-value-digits = 2
}
mailbox-size {
highest-trackable-value = 999999999
significant-value-digits = 2
}
}
trace {
elapsed-time {
highest-trackable-value = 3600000000000
significant-value-digits = 2
}
segment {
highest-trackable-value = 3600000000000
significant-value-digits = 2
}
}
dispatcher {
maximum-pool-size {
highest-trackable-value = 999999999
significant-value-digits = 2
}
running-thread-count {
highest-trackable-value = 999999999
significant-value-digits = 2
}
queued-task-count {
highest-trackable-value = 999999999
significant-value-digits = 2
}
pool-size {
highest-trackable-value = 999999999
significant-value-digits = 2
}
}
}
}
trace {
# If ask-pattern-tracing is enabled, a WARN level log message will be generated if a future generated by the ask
# pattern fails with a AskTimeoutException
and the log message will contain a stack trace captured at the moment
# the future was created.
ask-pattern-tracing = off
}
weaver {
# AspectJ options supported by LTW
# showWeaveInfo: show informational messages whenever the weaver touches a class file.
# verbose: show informational messages about the weaving process.
# debug: show a messages for each class passed to the weaver indicating whether it was woven, excluded or ignored.
# showWarn: show warning messages about the weaving process.
showWeaveInfo = off
verbose = off
debug = off
showWarn = off
}
}
What is the minimum configuration that I need to have for datadog.conf?
I have one host dashboard in Datadog - shows only my system metrics (cpu usage, memory, etc)
from kamon.
The minimum would be not providing any custom configuration for kamon-datadog. The default values assume that your datadog agent is running locally on port 8125, so it should work just by adding this to your configuration file:
akka {
extensions = ["kamon.datadog.Datadog"]
}
I can't find that section on your message, are you really sure it is there? Then, you should modify the filters accordingly to your needs.
from kamon.
Yes, this is my akka config:
Akka settings
akka {
loglevel = "DEBUG"
actor {
debug {
autoreceive = on
lifecycle = off
receive = off
}
}
extensions = ["kamon.datadog.Datadog"]
log-dead-letters-during-shutdown = on
}
I have the following set for datadog.conf
dd_url: https://app.datadoghq.com
api-key:
Do I need this set?
listen_port: 8125
from kamon.
It's an Akka standalone application - I'm assuming it would work.
Here's the datadog.conf for the agent
https://github.com/DataDog/dd-agent/blob/master/datadog.conf.example
from kamon.
the port that you care about in the datadog agent configuration is the dogstatsd_port
which by default is 8125 and should match the one configured by default in kamon-datadog. During the application startup you should see a "Starting the Kamon(Datadog) extension" message. Are you seeing that?
Also, are you starting your application with the -javaagent:/path-to-aspectj-weaver.jar option?
from kamon.
Yes, I did so and here's the Starting Kamon(Datadog) msg:
[INFO] [07/03/2014 12:29:21.148] [main] [DatadogExtension(akka://batch-statements)] Starting the Kamon(Datadog) extension
Jul 3, 2014 12:29:21 PM kamon.weaver.logging.KamonWeaverMessageHandler showMessage
INFO: weaving 'kamon.datadog.DatadogExtension$$anonfun$buildMetricsListener$1'
All I have set in datadog.conf are:
dd_url: https://app.datadoghq.com
api-key:
Here's what I have in the metric explorer in datadoghq
from kamon.
If you do a netstat -an | grep 8125
you see the port 8125 listening for UDP traffic?, what datadog agent version do you have? What's the output of running your-datadog-agent-folder/bin/info
?
from kamon.
Yes, it's listed for UDP. The agent version is 4.4.0. The datadog agent doesn't have a bin folder. Here's the installation folder:
from kamon.
and, what do the statsd.bad_lines_seen
and statsd.packets_received
metrics show in your datadog dashboard?
from kamon.
They aren't there anymore with this configuration. I'm assuming they showed up while I was tweaking all kinds of configuration on datadog.conf
from kamon.
dogstatsd.packet.count graph shows zero.
from kamon.
I see that you are running windows, maybe there is some weird thing going on with the windows firewall that might be blocking those packets?
from kamon.
My firewall is off. Should there be a "kamon" app listed here under "Apps" or in metric explorer (even though the packets are not sent)?
It's not receiving any metrics?
from kamon.
Surely the agent is not receiving any metrics from Kamon... What about sending a metric packet manually to DatadogStatsD and make sure it is working?, if you use this:
echo "sample-latency:190|ms" | nc -u -w0 127.0.0.1 8125
you should be able to see that metric on your dashboard shortly after.
from kamon.
Hello there! I was wondering if you were able to manually post the the metrics to your datadog agent?, also I did a more detailed review on the reasons of the bug that initially started this issue and found that we needed to pass a couple more options to the Java compiler for that specific class, that has been solved by c6b8918.
from kamon.
Hi, I was able to see that the packet was received in datadog.dogstatsd.packet.count graph on the dashboard. However, I don't see a metric listed as "sample-latency".
While running the application with the SNAPSHOTS from earlier, I don't see any packets on the graph (count=0).
I tried that build c6b8918 but no packets on the graph. Also, no new metrics.
from kamon.
@labrg I'm running out of ideas now, if you are manually posting a metric to the agent and it doesn't show it on your datadog dashboard we have reached a point where there is very little I can do to help, it might be that something is not configured properly in your agent but probably the datadog support guys can do a lot more for you than I.. could you please contact them and make sure that when you report a metric it gets to Datadog and then come back to use to troubleshoot what might be wrong with Kamon?, I'm really sorry for not being able to do more for you at this point.
from kamon.
@ivantopo Thanks for all your feedback. I got in touch with datadog - according to them, a bug affects their windows agent. Here's the change they recommended (adding a pipe at the end of the packet)
sample-latency:190|ms|
This does show the metric on the dashboard.
Is there anything that can be done with Kamon to work with this bug or should I talk to datadog about it?
from kamon.
Did they mention if reporting metrics with that format will work properly on Linux/OSX?
from kamon.
Also, please try sending a packet like the one bellow and tell me if it is accepting it correctly:
kamon.actor.processing-time:10|ms|@0.5|#actor:user/kamon
from kamon.
I've asked their support if it would work on Linux/OSX. I'll let you know as soon as I hear from them.
Also, that packet was accepted over tag actor:user/kamon:
from kamon.
and what about this one:
kamon.actor.time-in-mailbox:20|ms|#actor:user/kamon
from kamon.
Yes, that works as well.
from kamon.
well, those are the two ways in which Kamon send metrics to datadog, maybe there is a problem with multiple metrics in the same packet? Tell me if something like this works:
kamon.actor.some-metric:20|ms|#actor:user/kamon\nkamon.actor.some-metric:30|ms|#actor:user/kamon
from kamon.
@ivantopo Yeah, it works with bash. (echo -e ...)
from kamon.
@labrg I just opened #52 to track the compatibility issues between kamon-datadog and the Windows Datadog agent, let's track that problem there and close this one which original motivation has been already solved, thanks.
from kamon.
Related Issues (20)
- Kafka instrumentation. Add supporting for different context type propagation HOT 1
- Kamon-Cassandra Datastax OSS Metrics Factory
- kamon.io, kanela agent unable to post metrics to https enabled graphite backend
- Java 8 support, new relic http client HOT 2
- Context propagation is broken from Akka 2.8.0 onwards HOT 3
- Bumping 2.6.3 to 2.6.4 (kamon-bundle and kamon-prometheus) breaks logging in our apps (Akka HTTP, Play) HOT 9
- Old kanela-agent which does not support JDK 21 HOT 5
- Maven metadata does not contain info about Kamon Bundle v2.6.5 HOT 1
- Kamon doesn't propage Kamon context HOT 7
- Sbt 1.9.7 upgrade breaks Kamon Play HTTP Prometheus Metrics HOT 2
- Upgrade newrelic.telemetry to 0.16.0 HOT 2
- Add support for Micrometer
- Instrumentation is broken With play 3.0, scala 3.3.1, kamon-bundle and Prometheus HOT 1
- Support for postgresql 42.7.2 - java.lang.NoSuchFieldError: checkConnectionQuery HOT 6
- Issue while using jaeger reporter with scala
- Akka-http metrics do not work when using http2
- ClassCastException when using Slick and Kamon Jdbc
- Trying to load Akka instrumentation when using Pekko in Scala 3 HOT 1
- Updating Postgres Driver causes NoSuchFieldError from Kamon JDBC HOT 2
- How to use local-tail-sampler
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kamon.