simonnagl / netdata-java-orchestrator Goto Github PK
View Code? Open in Web Editor NEWNetdata Java Plugin Daemon
License: GNU General Public License v3.0
Netdata Java Plugin Daemon
License: GNU General Public License v3.0
The JMX-Plugin should be able to auto-detect and connect to local running java processes.
This needs some research:
This is probably the solution: http://www.javaworld.com/article/2073919/core-java/groovy--jmx--and-the-attach-api.html
Hi . I want to remove the jmx NetdataJavaDaemon chart. How to do that?
@simonnagl
Currently the installer always copies the newest configuration files to the configuration directory. The installer should not overwrite existing configuration files.
We may support charts with combined information of all monitored JMX processes.
For example we could provide a chart CPU which displays CPU usage of java processes with one dimension per process.
Author: @ktsaou
Integrating this into netdata would also be nice. I guess this would require some more work to build it using the netdata automake system and install it with the rest.
What do you mean with integrating?
Putting it completely into the netdata project? When I started implementing this I decided to do it in a separate project mainly because java has a much different build system. My thoughts where these: Netdata could add this to the installer. The installer will decide if java is there. If yes it downloads this plugin and runs the plugins installer. If no we do not need this plugin. We won't be able to compile it anyways.
Add a process CPU percent chart.
Value ProcessCpuTime from java.lang:type=OperatingSystem is in nanoseconds.
Recommendations:
I'm trying to use this to pull MBean information from my minecraft server(s) to monitor internal tick times.
My first issue is that if I define the server it won't find it, however if I leave the "jmxServers" list empty it will find the connection regardless if "autoDetectLocalVirtualMachines" is set to true or false. If I define a jmxServers list entry, it doesn't show any information at all.
For right now I'm testing it with an empty list and autoDetect set to true.
Here's the jmxServer entry that I have tried. This server is started with -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=42001 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=192.168.1.137
{
"name": "Minecraft1",
"serviceUrl": "service:jmx:rmi:///jndi/rmi://127.0.0.1:42001/jmxrmi"
}
I have tested the connection locally with jmxterm to ensure it works and the MBean is available:
$>open 127.0.0.1:42001
#Connection to 127.0.0.1:42001 is opened
$>get -b net.minecraft.server:type=Server averageTickTime
#mbean = net.minecraft.server:type=Server:
averageTickTime = 1.5295846;
I've also tested this with VisualVM over the network and am able to get this MBean data.
I've gone and thrown in some extra logging in a few places just to verify what information it's passing around but it just confirms that it's not able to find it.
2021-08-18 10:12:35: java.d: WARNING: org.firehol.netdata.module.jmx: dimension objectName = net.minecraft.server:type=Server
2021-08-18 10:12:35: java.d: WARNING: org.firehol.netdata.module.jmx: getMBeanQueryForName has been called, object = net.minecraft.server:type=Server, attr = averageTickTime
2021-08-18 10:12:35: java.d: WARNING: org.firehol.netdata.module.jmx: Could not query one dimension. Skipping... Reason: [org.firehol.netdata.module.jmx.exception.JmxMBeanServerQueryException] Could not query attribute 'averageTickTime
' of MBean 'net.minecraft.server:type=Server' ... javax.management.InstanceNotFoundException: net.minecraft.server:type=Server ... net.minecraft.server:type=Server Detail: net.minecraft.server:type=Server
Is there a reason you know of why it wouldn't be able to find the MBean/attribute that I'm able to pull using other methods? I'm still trying to look through java and see, but I'm not super well versed in java.
Add Uptime Chart of the JVM
Value Uptime of java.lang:type=Runtime returns milliseconds.
Recommendations:
#7 Add divider/multiplier/factor option to JmxDimensionConfiguration
Add Memory usage Charts
Dependes on:
#6 Composite Data Support
It would be appropriate to have a test which runs the compiled char with some example configuration and tests the output of the plugin.
Edit: I am not sure this brings any benefit to this project...
@ktsaou started a discussion about configuration languages in #39. Let's continue it here.
Currently we use json. Me may switch this.
Arguments of @ktsaou:
xml
.json
does not support comments so extensive documentation is needed in separate files.yaml
if there was a javascript-only implementation yaml
library.My Arguments:
yaml
extension. I never used it but we could try.Installation should be easy. Only one command should be needed to set everything up.
The JMX API supports to query multiple attributes of one MBean with one request. We should support that too.
Needed research:
getAttributes
slower if only fetching one attribute?Needed changes:
MBeanQueryInfo
: MBeanQueryInfo
should contain information about the MBean and a list of Attributes to collect and where to store them.MBeanServerCollector
to fit.MBeanServerColletor
to fit.from: java.lang:type=Threading
We need to find a way to configure this type of collection.
@h0lzi wrote at netdata/netdata#364:
One thing I notice. If I restart the tomcat the plugin will not reconnect.
I'm not sure what the best way to handel that is. Maybe if there was a connection try every x seconds for y minutes.
(Make that a config option?). Off cause it needs to bail out eventually.
But besides that basic stuff seems to work for me.
He is right. We need an option to periodically recheck if the configured JMX servers are there and rerun auto detection if it is on.
Migrated from netdata/netdata#3057
Clones netdata/netdata#3046
Currently the JMX module of the java plugin can only connect to JMX Servers without authentication. We should provide a way to configure authentication for custom servers.
This request did come from @h0lzi.
Add support for MBeans returning CompositeData.
Use exec java ... to run java to prevent a bash process in the netdata process tree
author: @ktsaou
Otherwise the initialization process may hang for a very long time.
https://community.oracle.com/blogs/emcmanus/2007/05/23/making-jmx-connection-timeout
I have add some mem metrics, I got some error:
{
// Unique ID of this chart in this list.
"id": "server",
"title": "session",
// Unit of the collected value
"units": "",
// orders charts. Lower numbers make the charts appear before the ones with higher numbers
"priority": "8000",
// Possible chart types: AREA, LINE, STACKED
"chartType": "AREA",
// Possible dimension algorithms:
// ABSOLUTE, INCREMENTAL, PERCENTAGE_OF_ABSOLUTE_ROW, PERCENTAGE_OF_INCREMENTAL_ROW
"dimensionAlgorithm": "ABSOLUTE",
// List of dimensions to add to the chart.
"dimensions": [
{
"from": "Catalina:type=Manager,host=localhost,context=/",
"value": "activeSessions",
"name": "activeSession",
"multiplier": "1",
"divisor": "1",
"hidden": "false"
},
{
"from": "Catalina:type=Manager,host=localhost,context=/",
"value": "maxActive",
"name": "maxActive",
"multiplier": "1",
"divisor": "1",
"hidden": "false"
},
{
"from": "Catalina:type=Manager,host=localhost,context=/",
"value": "maxActiveSessions",
"name": "maxActiveSessions",
"multiplier": "1",
"divisor": "1",
"hidden": "false"
},
{
"from": "Catalina:type=Manager,host=localhost,context=/",
"value": "essionCounter",
"name": "sessionCounter",
"multiplier": "1",
"divisor": "1",
"hidden": "false"
},
{
"from": "Catalina:type=Manager,host=localhost,context=/",
"value": "rejectedSessions",
"name": "rejectedSessions",
"multiplier": "1",
"divisor": "1",
"hidden": "false"
},
{
"from": "java.lang:type=Memory",
"value": "HeapMemoryUsage.committed",
"name": "HeapMemoryUsagecommitted"
},
{
"from": "java.lang:type=Memory",
"value": "HeapMemoryUsage.max",
"name": "HeapMemoryUsagemax"
},
{
"from": "java.lang:type=Memory",
"value": "HeapMemoryUsage.used",
"name": "HeapMemoryUsageused"
},
{
"from": "java.lang:type=Memory",
"value": "NonHeapMemoryUsage.committed",
"name": "NonHeapMemoryUsagecommitted"
},
{
"from": "java.lang:type=Memory",
"value": "NonHeapMemoryUsage.max",
"name": "NonHeapMemoryUsagemax"
},
{
"from": "java.lang:type=Memory",
"value": "NonHeapMemoryUsage.used",
"name": "NonHeapMemoryUsageused"
},
{
"from": "java.lang:type=Memory",
"value": "ObjectPendingFinalizationCoun",
"name": "ObjectPendingFinalizationCoun"
}
]
},
erro log:
rehol.netdata.module.jmx: Could not query one dimension. Skipping... Reason: Could not query attribute 'HeapMemoryUsage.committed' of MBean 'java.lang:type=Memory' Detail: No such attribute: HeapMemoryUsage.committed
2018-01-31 01:31:02: java.d: WARNING: org.firehol.netdata.module.jmx: Could not query one dimension. Skipping... Reason: Could not query attribute 'HeapMemoryUsage.max' of MBean 'java.lang:type=Memory' Detail: No such attribute: HeapMemoryUsage.max
2018-01-31 01:31:02: java.d: WARNING: org.firehol.netdata.module.jmx: Could not query one dimension. Skipping... Reason: Could not query attribute 'HeapMemoryUsage.used' of MBean 'java.lang:type=Memory' Detail: No such attribute: HeapMemoryUsage.used
2018-01-31 01:31:02: java.d: WARNING: org.firehol.netdata.module.jmx: Could not query one dimension. Skipping... Reason: Could not query attribute 'NonHeapMemoryUsage.committed' of MBean 'java.lang:type=Memory' Detail: No such attribute: NonHeapMemoryUsage.committed
2018-01-31 01:31:02: java.d: WARNING: org.firehol.netdata.module.jmx: Could not query one dimension. Skipping... Reason: Could not query attribute 'NonHeapMemoryUsage.max' of MBean 'java.lang:type=Memory' Detail: No such attribute: NonHeapMemoryUsage.max
2018-01-31 01:31:02: java.d: WARNING: org.firehol.netdata.module.jmx: Could not query one dimension. Skipping... Reason: Could not query attribute 'NonHeapMemoryUsage.used' of MBean 'java.lang:type=Memory' Detail: No such attribute: NonHeapMemoryUsage.used
2018-01-31 01:31:02: java.d: WARNING: org.firehol.netdata.module.jmx: Could not query one dimension. Skipping... Reason: Could not query attribute 'ObjectPendingFinalizationCoun' of MBean 'java.lang:type=Memory' Detail: No such attribute: ObjectPendingFinalizationCoun
2018-01-31 01:31:02: java.d: WARNING: org.firehol.netdata.module.jmx: Could not query one dimension. Skipping... Reason: Could not query attribute 'activeSessions' of MBean 'Catalina:type=Manager,host=localhost,context=/' Detail: Catalina:type=Manager,host=localhost,context=/
2018-01-31 01:31:02: java.d: WARNING: org.firehol.netdata.module.jmx: Could not query one dimension. Skipping... Reason: Could not query attribute 'maxActive' of MBean 'Catalina:type=Manager,host=localhost,context=/' Detail: Catalina:type=Manager,host=localhost,context=/
2018-01-31 01:31:02: java.d: WARNING: org.firehol.netdata.module.jmx: Could not query one dimension. Skipping... Reason: Could not query attribute 'maxActiveSessions' of MBean 'Catalina:type=Manager,host=localhost,context=/' Detail: Catalina:type=Manager,host=localhost,context=/
2018-01-31 01:31:02: java.d: WARNING: org.firehol.netdata.module.jmx: Could not query one dimension. Skipping... Reason: Could not query attribute 'essionCounter' of MBean 'Catalina:type=Manager,host=localhost,context=/' Detail: Catalina:type=Manager,host=localhost,context=/
2018-01-31 01:31:02: java.d: WARNING: org.firehol.netdata.module.jmx: Could not query one dimension. Skipping... Reason: Could not query attribute 'rejectedSessions' of MBean 'Catalina:type=Manager,host=localhost,context=/' Detail: Catalina:type=Manager,host=localhost,context=/
2018-01-31 01:31:02: java.d: WARNING: org.firehol.netdata.module.jmx: Could not query one dimension. Skipping... Reason: Could not query attribute 'HeapMemoryUsage.committed' of MBean 'java.lang:type=Memory' Detail: No such attribute: HeapMemoryUsage.committed
2018-01-31 01:31:02: java.d: WARNING: org.firehol.netdata.module.jmx: Could not query one dimension. Skipping... Reason: Could not query attribute 'HeapMemoryUsage.max' of MBean 'java.lang:type=Memory' Detail: No such attribute: HeapMemoryUsage.max
2018-01-31 01:31:02: java.d: WARNING: org.firehol.netdata.module.jmx: Could not query one dimension. Skipping... Reason: Could not query attribute 'HeapMemoryUsage.used' of MBean 'java.lang:type=Memory' Detail: No such attribute: HeapMemoryUsage.used
2018-01-31 01:31:02: java.d: WARNING: org.firehol.netdata.module.jmx: Could not query one dimension. Skipping... Reason: Could not query attribute 'NonHeapMemoryUsage.committed' of MBean 'java.lang:type=Memory' Detail: No such attribute: NonHeapMemoryUsage.committed
2018-01-31 01:31:02: java.d: WARNING: org.firehol.netdata.module.jmx: Could not query one dimension. Skipping... Reason: Could not query attribute 'NonHeapMemoryUsage.max' of MBean 'java.lang:type=Memory' Detail: No such attribute: NonHeapMemoryUsage.max
2018-01-31 01:31:02: java.d: WARNING: org.firehol.netdata.module.jmx: Could not query one dimension. Skipping... Reason: Could not query attribute 'NonHeapMemoryUsage.used' of MBean 'java.lang:type=Memory' Detail: No such attribute: NonHeapMemoryUsage.used
2018-01-31 01:31:02: java.d: WARNING: org.firehol.netdata.module.jmx: Could not query one dimension. Skipping... Reason: Could not query attribute 'ObjectPendingFinalizationCoun' of MBean 'java.lang:type=Memory' Detail: No such attribute: ObjectPendingFinalizationCoun
Hi,
I am using Netdata 1.17.0 and Java Version 1.8
and i installed this "netdata-java-orchestrator" using the script "netdata-java-orchestrator-installer.sh" as mentioned in "README.md" and everything is successfully executed and i got "JMX-CHARTS" in the Netdata but i didn't the "Alarms". If there is any procedure to configure Alarms please help me out
This requires to add family option to configuration.
Hi there,
in line 225 of netdata-java-orchestrator-installer.sh, Maven is configured to run in offline mode. As a result, the dependencies cannot be resolved properly and the installation fails. Removing the -o switch solves the problem.
Cheers,
marcus
After #24 the configuration property java.util.logging.SimpleFormatter.format will be in the installer. I think it should go into the source code or the resources of the jar file.
Add documentation for:
I decided to do this as simple as possible. We just add priority to configuration and let metadata do the ordering.
Is there a way to monitor jvm process heap with this plugin? If not this feature could be very useful. Great plugin btw!
Add a chart with the values of java.nio:type=BufferPool beans.
Recherche is needed what the values represent exactly.
Clone of netdata/netdata#3632
For description and discussion please use the clone.
For the charts of the default configuration, add health configuration.
If a Java process gets killed we currently do not collect values any more, but the chart get's empty. For auto-configured processes, if the process is started with a new PID there is no chance to attach this process to the old charts.
We do not want to spam the metadata dashboard creating new charts and not removing the old ones. We want to mark charts which we won't refresh any more as obsolete.
Every source file should contain this header.
/*
* Copyright (C) 2017 Simon Nagl
*
* metadata-plugin-java-daemon is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*
*/
Add a process CPU Load chart.
Collect value ProcessCpuLoad from java.lang:type=OperatingSystem
Recommendation:
We need to decouple the dependencies of JmxPlugin and other singletons to make them testable.
We might use Guice to inject the dependencies at runtime or maybe it is enogh for now to initialize them before the program runs.
The project name is misleading. The program is intended to run as a netdata plugin.
netdata will run the process. This has nothing to do with a daemon process.
Also I am not sure if I should keep this project open for other java collectors (currently there is only an JMX collector).
If yes the new name will be netdata-jmx-plugin
If not the new name would netdata-java-plugin
@ktsaou: Do you want me to help making this decision? I am asking because I wanted to ask you would accept moving this project to the netdata organization. I am asking you know because the naming, and with it the decision closing this project to other java collectors/plugins might affect your decision.
If you do not want to have anything to do with this please just ignore my question.
eg: jvm version , I see the comment, only number attribute is supported
java.lang:type=Threading DeadlockedThreads. Display number of DeadlockedThreads.
DeadlockedThreads is a long[]. We want to count the size of long[]. We need to find a way to configure this kind of collection.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.