Sophisticated monitoring agent for Java
Home Page: http://zorka.io
License: GNU General Public License v3.0
Zorka is a programmable general purpose monitoring agent for Java applications. It features bytecode instrumentation and is capable of tracing (profiling) production environments. It is designed to integrate seamlessly your Java applications with popular monitoring systems.
For more information see Zorka project page.
On Zabbix 2.4, importing Template_Zorka_JVM_active.xml failed for me.
Error was,
Cannot find item "jvm.memutil["Perm Gen"]" on "Template_Zorka_JVM" used in trigger "{HOSTNAME} PermGen utilization is > 90%".
A plain text output for the tracer would very useful, as this would allow easy integration with centralized log aggregators (Splunk/SumoLogic/LogStash/etc. etc.) for APM or audit purposes. The existing formats (fressian and zabbix) are much more difficult to work with in these systems.
I've made a rough start, but am a long way from a pull request -- and would need some help understanding the TraceRecord structure to get a proper version, but the format I've used is below.
This is just an idea of what would be most easy for these types of systems, which generally work best with any basic key/value structure.
[Sun Mar 15 18:14:51 EST 2015] class=com.mysql.jdbc.PreparedStatement, method=executeQuery, SQL="select this_.USER_ID as USER_ID1_0_0_, this_.USERNAME as USERNAME2_0_0_, this_.PASSWORD as PASSWORD3_0_0_, this_.EMAIL as EMAIL4_0_0_ from USERS this_, DB=testdb", time=2004633
[Sun Mar 15 18:14:51 EST 2015] class=com.mysql.jdbc.StatementImpl, method=executeQuery, SQL="SHOW WARNINGS", DB=testdb, time=1149703
[Sun Mar 15 18:14:51 EST 2015] class=org.apache.catalina.core.StandardEngineValve, method=invoke, URI=/TestApp/, STATUS=200, CookieIn__JSESSIONID=B688A700826A68B4A75681B1BD4230AC, CookieIn__session_id_8000=0422fca3e7b35d4462851080012858df39ff7666, CookieIn__splunkweb_csrf_token_8000=15573489094520950050, HdrIn__host=localhost:8080, HdrIn__cookie=JSESSIONID=B688A700826A68B4A75681B1BD4230AC; session_id_8000=0422fca3e7b35d4462851080012858df39ff7666; splunkweb_csrf_token_8000=15573489094520950050, time=15823899
Hi,
I've got strange behavior with active checks, item status is flapping between supported/not supported state. Relevant log records:
zabbix server log:
12361:20160426:121956.557 trapper got '{"request":"active checks","host_metadata":"","port":10055,"host":"ipa-test-mule-02.example.com"}'
12361:20160426:121956.557 In send_list_of_active_checks_json()
12361:20160426:121956.557 In get_hostid_by_host() host:'ipa-test-mule-02.example.com'
12361:20160426:121956.557 query [txnlev:0] [select hostid,status,tls_accept,tls_issuer,tls_subject,tls_psk_identity from hosts where host='ipa-test-mule-02.example' and status in (0,1) and flags<>2 and proxy_hostid is null]
12361:20160426:121956.558 End of get_hostid_by_host():SUCCEED
12361:20160426:121956.558 query [txnlev:0] [select itemid from items where type=7 and flags<>2 and hostid=10110]
12361:20160426:121956.558 In substitute_key_macros() data:'jvm.fdutil[]'
12361:20160426:121956.558 End of substitute_key_macros():SUCCEED data:'jvm.fdutil[]'
12361:20160426:121956.558 send_list_of_active_checks_json() sending [{"response":"success","data":[{"key":"jvm.fdutil[]","delay":30,"lastlogsize":0,"mtime":0}]}]
12361:20160426:121956.558 End of send_list_of_active_checks_json():SUCCEED
everything looks ok. then again in zabbix server log:
12359:20160426:122008.863 trapper got '{"request":"active checks","host":"ipa-test-mule-02.example.com"}'
12359:20160426:122008.863 In send_list_of_active_checks_json()
12359:20160426:122008.863 In get_hostid_by_host() host:'ipa-test-mule-02.example.com'
12359:20160426:122008.863 query [txnlev:0] [select hostid,status,tls_accept,tls_issuer,tls_subject,tls_psk_identity from hosts where host='ipa-test-mule-02.example.com' and status in (0,1) and flags<>2 and proxy_hostid is null]
12359:20160426:122008.863 End of get_hostid_by_host():SUCCEED
12359:20160426:122008.864 query [txnlev:0] [select itemid from items where type=7 and flags<>2 and hostid=10110]
12359:20160426:122008.864 In substitute_key_macros() data:'jvm.fdutil[]'
12359:20160426:122008.864 End of substitute_key_macros():SUCCEED data:'jvm.fdutil[]'
12359:20160426:122008.864 send_list_of_active_checks_json() sending [{"response":"success","data":[{"key":"jvm.fdutil[]","delay":30,"lastlogsize":0,"mtime":0}]}]
12359:20160426:122008.864 End of send_list_of_active_checks_json():SUCCEED
12359:20160426:122008.864 __zbx_zbx_setproctitle() title:'trapper #1 [processed data in 0.001284 sec, waiting for connection]'
12376:20160426:122008.879 __zbx_zbx_setproctitle() title:'self-monitoring [processing data]'
12376:20160426:122008.879 In collect_selfmon_stats()
12376:20160426:122008.879 End of collect_selfmon_stats()
12376:20160426:122008.879 __zbx_zbx_setproctitle() title:'self-monitoring [processed data in 0.000064 sec, idle 1 sec]'
12360:20160426:122009.867 __zbx_zbx_setproctitle() title:'trapper #2 [processing data]'
12360:20160426:122009.867 trapper got '{"request":"agent data","data":[{"host":"ipa-test-mule-02.example.com","key":"jvm.fdutil[]","value":"Unsupported item key.","state":1,"clock":1461662409,"ns":743142094}],"clock":1461662410,"ns":744940745}'
12360:20160426:122009.867 In recv_agenthistory()
12360:20160426:122009.867 In process_hist_data()
12360:20160426:122009.867 In process_mass_data()
12360:20160426:122009.867 End of process_mass_data()
12360:20160426:122009.867 End of process_hist_data():SUCCEED
12360:20160426:122009.867 In zbx_send_response()
12360:20160426:122009.867 zbx_send_response() '{"response":"success","info":"processed: 1; failed: 0; total: 1; seconds spent: 0.000107"}'
12360:20160426:122009.867 End of zbx_send_response():SUCCEED
12360:20160426:122009.867 End of recv_agenthistory()
12360:20160426:122009.867 __zbx_zbx_setproctitle() title:'trapper #2 [processed data in 0.000386 sec, waiting for connection]'
12370:20160426:122009.878 query [txnlev:1] [begin;]
12370:20160426:122009.878 In DCmass_update_items()
12370:20160426:122009.878 item "ipa-test-mule-02.example.com:jvm.fdutil[]" became not supported: Unsupported item key.
12370:20160426:122009.878 In DCadd_nextcheck()
12370:20160426:122009.878 End of DCadd_nextcheck()
12370:20160426:122009.878 query [txnlev:1] [update items set state=1,error='Unsupported item key.' where itemid=24048;]
12370:20160426:122009.879 End of DCmass_update_items()
looks like zorka requested list of active check again, zabbix server sended it, but then second later zorka answered that item is not supported? However, there are no info about that in zorka log:
2016-04-26 12:19:57 DEBUG ZabbixUtils Message: '{"request":"active checks","host_metadata":"","port":10055,"host":"ipa-test-mule-02.example.com"}'
2016-04-26 12:19:57 DEBUG ZabbixUtils Message length: 103
2016-04-26 12:19:57 DEBUG ZabbixActiveRequest Zorka send: ZBXDg{"request":"active checks","host_metadata":"","port":10055,"host":"ipa-test-mule-02.example.com"}
2016-04-26 12:19:57 DEBUG ZabbixActiveRequest Zorka get:{"response":"success","data":[{"key":"jvm.fdutil[]","delay":30,"lastlogsize":0,"mtime":0}]}
2016-04-26 12:19:57 DEBUG ZabbixActiveAgent ZabbixActive - schedule Tasks: {response=success, data=[{key=jvm.fdutil[], delay=30, lastlogsize=0, mtime=0}]}
2016-04-26 12:19:57 DEBUG ZabbixActiveAgent ZabbixActive - task: {key=jvm.fdutil[], delay=30, lastlogsize=0, mtime=0}
2016-04-26 12:19:57 DEBUG ZabbixActiveAgent ZabbixActive - new scheduled tasks: 1
2016-04-26 12:19:57 DEBUG ZabbixActiveAgent ZabbixActive - deleted old tasks: 1
2016-04-26 12:20:02 DEBUG ZabbixActiveTask Running task: jvm.fdutil[]
2016-04-26 12:20:02 DEBUG ZabbixActiveTask Translated task: jvm.fdutil()
2016-04-26 12:20:02 DEBUG ZorkaBshAgent Processing request BSH expression: jvm.fdutil()
2016-04-26 12:20:02 DEBUG ZabbixActiveTask Task response: jvm.fdutil[] -> 9.912109375
2016-04-26 12:20:02 DEBUG ZabbixActiveTask Cache size: 1
2016-04-26 12:20:32 DEBUG ZabbixActiveTask Running task: jvm.fdutil[]
2016-04-26 12:20:32 DEBUG ZabbixActiveTask Translated task: jvm.fdutil()
2016-04-26 12:20:32 DEBUG ZorkaBshAgent Processing request BSH expression: jvm.fdutil()
2016-04-26 12:20:32 DEBUG ZabbixActiveTask Task response: jvm.fdutil[] -> 9.912109375
2016-04-26 12:20:32 DEBUG ZabbixActiveTask Cache size: 2
2016-04-26 12:20:57 DEBUG ZabbixActiveSenderTask ZabbixActiveSender run...
2016-04-26 12:20:57 DEBUG ZabbixActiveSenderTask ZabbixActiveSender 2 items cached
2016-04-26 12:20:57 DEBUG ZabbixUtils Message: '{"clock":1461662457,"data":[{"clock":1461662402,"host":"ipa-test-mule-02.example.com","key":"jvm.fdutil[]","lastlogsize":0,"value":"9.912109375"},{"clock":1461662432,"host":"ipa-test-mule-02.example.com","key":"jvm.fdutil[]","lastlogsize":0,"value":"9.912109375"}],"request":"agent data"}'
2016-04-26 12:20:57 DEBUG ZabbixUtils Message length: 300
2016-04-26 12:20:57 DEBUG ZabbixActiveRequest Zorka send: ZBXD,{"clock":1461662457,"data":[{"clock":1461662402,"host":"ipa-test-mule-02.example.com","key":"jvm.fdutil[]","lastlogsize":0,"value":"9.912109375"},{"clock":1461662432,"host":"ipa-test-mule-02.example.com","key":"jvm.fdutil[]","lastlogsize":0,"value":"9.912109375"}],"request":"agent data"}
2016-04-26 12:20:57 DEBUG ZabbixActiveSenderTask ZabbixActiveSender message sent: {"clock":1461662457,"data":[{"clock":1461662402,"host":"ipa-test-mule-02.example.com","key":"jvm.fdutil[]","lastlogsize":0,"value":"9.912109375"},{"clock":1461662432,"host":"ipa-test-mule-02.example.com","key":"jvm.fdutil[]","lastlogsize":0,"value":"9.912109375"}],"request":"agent data"}
2016-04-26 12:20:57 DEBUG ZabbixActiveRequest Zorka get:{"response":"success","info":"processed: 2; failed: 0; total: 2; seconds spent: 0.000128"}
2016-04-26 12:20:57 DEBUG ZabbixActiveSenderTask ZabbixActiveSender 2 items removed from cache
2016-04-26 12:20:57 DEBUG ZabbixActiveSenderTask ZabbixActiveSender finished
so, at 12:20:56 zorka sends data to zabbix and item again became supported:
12360:20160426:122056.506 __zbx_zbx_setproctitle() title:'trapper #2 [processing data]'
12360:20160426:122056.506 trapper got '{"clock":1461662457,"data":[{"clock":1461662402,"host":"ipa-test-mule-02.example.com","key":"jvm.fdutil[]","lastlogsize":0,"value":"9.912109375"},{"clock":1461662432,"host":"ipa-test-mule-02.example.com","key":"jvm.fdutil[]","lastlogsize":0,"value":"9.912109375"}],"request":"agent data"}'
...
12360:20160426:122056.506 End of process_hist_data():SUCCEED
12360:20160426:122056.506 In zbx_send_response()
12360:20160426:122056.506 zbx_send_response() '{"response":"success","info":"processed: 2; failed: 0; total: 2; seconds spent: 0.000128"}'
12360:20160426:122056.506 End of zbx_send_response():SUCCEED
...
12371:20160426:122056.899 query [txnlev:1] [begin;]
12371:20160426:122056.900 In DCmass_update_items()
12371:20160426:122056.900 item "ipa-test-mule-02.example.com:jvm.fdutil[]" became supported
Are there some bug in zorka, or is it me doing something wrong?
Software versions:
Zabbix server v. 3.0.1, CentOS 7.2.1511
agent side:
zorka v. 1.0.15
jdk 1.8.0_77
CentOS 7.2.1511
zorka.properties:
[root@ipa-test-mule-02 zorka]# egrep -v '^#|^$' zorka.properties
scripts = jvm.bsh, zabbix.bsh, apps/muleesb.bsh
mule.stats = yes
zabbix.active = yes
zabbix.active.server.addr = 192.168.142.16:10051
zabbix.server.addr = 127.0.0.1,192.168.142.16
zabbix.listen.port = 10055
zorka.hostname = ipa-test-mule-02.example.com
http.trace.exclude = ~.*.png, ~.*.gif, ~.*.js, ~.*.css, ~.*.jpg, ~.*.jpeg, ~.*favicon.ico
Thanks!
Hi.
How can I get values from Springboot Actuator mbeans using Zorka? For example health data is available through JConsole by jmx operation 'health'. See attached screenshot.
Add proper BSH scripts and configuration for nagios plus documentation.
jvm.bsh - function memutil:
the mem_max value used can be -1 (G1GC), resulting in negative numbers being returned (large heap vm's overflow the zabbix float type)
In zabbix the item keeps flipping between supported and not supported. Depends on the amount of memory actually used how big the returned value becomes.
zabbix latest values for item 'G1 Eden Space utilization' (would expect a percentage between 0.0 and 100.0):
-20342374400
-17091788800
zabbix log:
jvm.memutil["G1 Eden Space"] became not supported: Type of received value [-1002438656000.000000] is not suitable for value type [Numeric (float)]
zabbix_get:
$> zabbix_get -s <host> -p <port> -k 'jvm.memutil["G1 Eden Space"]'
-1.60432128E10
$> zabbix_get -s <host> -p <port> -k 'zorka.jmx["java", "java.lang:name=G1 Eden Space,type=MemoryPool", "PeakUsage", "max"]'
-1
tbh I am not sure what the best way to go would be.
SpyFlagsProcessor.java, function process,
record.put(".STAGES", error
? ((f | SpyLib.ON_ERROR) & ~SpyLib.ON_RETURN)
: ((f | SpyLib.ON_RETURN) & ~SpyLib.ON_ERROR));
Here, ON_ERROR and ON_RETURN are used as bits.
ZorkaStatsCollector.java, function submit,
if (0 != ((Integer) record.get(".STAGES") & (1 << ON_RETURN))) {...
But here ON_ERROR and ON_RETURN are used as offsets.
when deploy zico-1.0.1 to ubuntu ,zorka occur errors: ERROR ZicoTraceOutput Error connecting /127.0.0.1:8640: ๆ็ป่ฟๆฅ
zico log as below:
2014-04-11 11:48:04,325 ERROR [pool-4-thread-1] c.j.z.c.ZicoServerConnector [ZicoServerConnector.java:101] Client /127.0.0.1:36693 not authorized.
2014-04-11 11:48:04,350 ERROR [pool-4-thread-1] c.j.z.c.ZicoServerConnector [ZicoServerConnector.java:125] Got ZICO exception (addr=/127.0.0.1:36693, status=10)
com.jitlogic.zorka.common.zico.ZicoException: Peer disconnected. Try again.
at com.jitlogic.zorka.common.zico.ZicoConnector.recv(ZicoConnector.java:77) ~[zorka-common-1.0.1.jar:na]
at com.jitlogic.zico.core.ZicoServerConnector.runCycle(ZicoServerConnector.java:73) ~[zico-core-1.0.1.jar:na]
at com.jitlogic.zico.core.ZicoServerConnector.run(ZicoServerConnector.java:118) ~[zico-core-1.0.1.jar:na]
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [na:1.6.0_30]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [na:1.6.0_30]
at java.lang.Thread.run(Thread.java:662) [na:1.6.0_30]
java.util.ConcurrentModificationException
at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:449)
at java.util.AbstractList$Itr.next(AbstractList.java:420)
at com.jitlogic.zorka.core.spy.TraceRecord.fixup(TraceRecord.java:374)
at com.jitlogic.zorka.core.spy.TraceBuilder.submit(TraceBuilder.java:293)
at com.jitlogic.zorka.core.spy.TraceBuilder.pop(TraceBuilder.java:214)
at com.jitlogic.zorka.core.spy.TraceBuilder.traceReturn(TraceBuilder.java:144)
at com.jitlogic.zorka.core.spy.MainSubmitter.traceReturn(MainSubmitter.java:102)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:199)
at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:282)
at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:767)
at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:697)
at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:889)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
at java.lang.Thread.run(Thread.java:595)
Hi,
So, every time option is in nanoseconds? Some itens have nothing on description, whats is the default?
Thks
Hallo,
we use Zorka in JBoss EAP server and have a OutOfMemory Problem. After sending the heap dump to JBoss support they analyzed it and told us, that Zorka is making the trouble:
Class Name | Objects | Shallow Heap | Retained Heap
------------------------------------------------------------------------------------------
com.jitlogic.zorka.common.util.TapOutputStream| 93 | 3,720 | >= 1,578,233,296
org.apache.cxf.io.CacheAndWriteOutputStream | 93 | 8,184 | >= 1,578,134,888
------------------------------------------------------------------------------------------
The heap dump shows the largest point of heap growth is 93 om.jitlogic.zorka.common.util.TapOutputStream and their underlying org.apache.cxf.io.CacheAndWriteOutputStreams. These are stored as thread locals on request threads through the _tap_xml_out com.jitlogic.zorka.deps.bsh.Variable's thread local value. Removing those thread locals would avoid substantial heap growth. Can you tell us why these streams are cached like this?
JBoss support told us, we should ask you how to drop TapOutputStream's thread local to avoid that heap overhead, because Zorka is creating thread locals that reference the cxf stream and prevent it from being GC'd. Typically, the cxf stream will not be cached on a thread local like this so it does not cause any persistent heap growth. Is Zorka specifically caching those for some kind of monitoring purpose? Can its caching of these cxf streams be disabled? If so, that change alone should drop in our case heap usage by about 1.5 gb in the provided dump.
We use Zorka 1.0.16
I see the zorka.log.dir property in AgentConfig.java but when it is set it does not use that value. It is value set in ZorkaConfig.java based on home directory.
AgentConfig.jav:
if (!properties.containsKey("zorka.log.dir")) { properties.put("zorka.log.dir", ZorkaUtil.path(homeDir, "log")); }
ZorkaConfig.java
'public String getLogDir() {
return ZorkaUtil.path(homeDir, "log");`
JAVA_OPTS="$JAVA_OPTS -javaagent:/zorka/agent.jar=/zorka"
Zorka works fine until issuing:
zabbix_get -s 127.0.0.1 -p 10055 -k 'zorka.reload[]'
This is the used zorka.properties file:
# Zorka configuration properties file.
# JBoss 7
scripts = jvm.bsh, zabbix.bsh, jboss/jboss7.bsh
# Zabbix agent is enabled by default. Uncomment this to disable zabbix integration.
# zabbix = no
# Zabbix Active Agent.
zabbix.active = yes
# Zabbix Server's Address (IP:Port)
zabbix.active.server.addr = zabbix1.example.com:10051
# Add IP addresses of your zabbix servers here.
# Only servers from this list will be allowed to access agent using zabbix protocol.
zabbix.server.addr = zabbix1.example.com,zabbix2.example.com,localhost
# Default port Zabbix protocol will listen on.
zabbix.listen.port = 10055
# Enter name of your application and host here.
# Should be unique for every monitored application.
zorka.hostname = foo.example.com
# Enables or disables automatic computation of stack maps in agent.
zorka.spy.compute.frames = yes
zorka.mbs.autoregister = no
Environment:
Hello,
When an app xxx is running and I want to know the active sessions, I see at jmx level that the next Mbean: jboss.web:type=Manager,host=localhost,path=/xxx have the attribute Name "JBossCacheManager", and then the activeSessions attribute not exist, but ActiveSessions yes.
Do you have any tip to configure zorka / zabbix to manage this circunstance with your zabbix.discovery["jboss", "jboss.web:type=Manager,*", "type", "host", "path"] function ?
I'm thinking on duplicate the discovery rule and to apply to this second discovery rule a filter, and then change the attribute names to its equiv but the first letter with capital letter.
Thank you for zorka!
Hi, I need the password from the configured websphere console in zorka.properties to be encripted, there is a way to make the password not visible or encripted?
Thanks for your help
scripts = jvm.bsh, zabbix.bsh ibm/was.bsh
Hi, guys.
We have tested Zorka 1.0.13 with zabbix active checks.
It works nice, but I've tried to use Zorka for host auto registration
https://www.zabbix.com/documentation/2.4/manual/discovery/auto_registration
and it works but it has some problems:
According to specification
https://www.zabbix.org/wiki/Docs/protocols/zabbix_agent/2.2
you don't send port zabbix.listen.port
in active check request
And you don't send
HostMetadata. An optional parameter that defines host metadata. If not defined, the value will be acquired from HostMetadataItem.
So it will be great to have a new configuration parameter
zabbix.hostmetadata = "string"
{
"host": "Host name",
"ip": "127.0.0.1",
"port": zabbix.listen.port,
"host_metadata": zabbix.hostmetadata,
"request": "active checks"
}
I very hope you will implement this, we need very much to auto-register our websphere application servers.
Thank you.
Hi everyone,
We are actually using Zabbix with Zorka agents for monitoring our Webphere servers, and we made a technical upgrade by setup the version of the SKD from 1.6 to 1.7. This operation caused a compatibility problem with the Zorka agent and it blocks the startup of the JVM nodes on WebSphere, so we had to suspend the agent (by removing this argument from the JVM "javaagent:/opt/IBM/WebSphere/AppServer/profiles/AppSrvUI01/servers/ServerUI1/zorka/zorka.jar=/opt/IBM/WebSphere/AppServer/profiles/AppSrvUI01/servers/ServerUI1/zorka" in order to start the servers.
When we try to start them, we catch this error:
WSVR0100W: Initialising error, ServerUI1 [class com.ibm.ws.runtime.component.ServerImpl]
java.lang.VerifyError: JVMVRFY012 forme de la pile incohรยฉrente; classe=com/ibm/ws/classloader/ProtectionClassLoader, mรยฉthode=(Ljava/lang/ClassLoader;)V, pc=0
at com.ibm.ws.runtime.component.ServerImpl.initialize(ServerImpl.java:343)
at com.ibm.ws.runtime.WsServerImpl.bootServerContainer(WsServerImpl.java:293)
at com.ibm.ws.runtime.WsServerImpl.start(WsServerImpl.java:224)
at com.ibm.ws.runtime.WsServerImpl.main(WsServerImpl.java:697)
at com.ibm.ws.runtime.WsServer.main(WsServer.java:59)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
at java.lang.reflect.Method.invoke(Method.java:620)
at com.ibm.wsspi.bootstrap.WSLauncher.launchMain(WSLauncher.java:234)
at com.ibm.wsspi.bootstrap.WSLauncher.main(WSLauncher.java:96)
at com.ibm.wsspi.bootstrap.WSLauncher.run(WSLauncher.java:77)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
at java.lang.reflect.Method.invoke(Method.java:620)
at org.eclipse.equinox.internal.app.EclipseAppContainer.callMethodWithException(EclipseAppContainer.java:587)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:198)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:369)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
at java.lang.reflect.Method.invoke(Method.java:620)
at org.eclipse.core.launcher.Main.invokeFramework(Main.java:340)
at org.eclipse.core.launcher.Main.basicRun(Main.java:282)
at org.eclipse.core.launcher.Main.run(Main.java:981)
at com.ibm.wsspi.bootstrap.WSPreLauncher.launchEclipse(WSPreLauncher.java:398)
at com.ibm.wsspi.bootstrap.WSPreLauncher.main(WSPreLauncher.java:161)
So we tried differents solutions:
We identified that these scripts in the source code "WAS.BSH" were the origin of the problem :
_spy.add(spy.instance("WAS_MBS_REGISTER")
.onReturn(spy.fetchArg("THIS", 0), jvm.autoregister_plugin(), pmi_register())
.include(spy.byMethod("com.ibm.ws.management.AdminServiceImpl", "")));
spy.add(spy.instance("WAS_CATCH_CLASS_LOADER")
.onReturn(spy.fetchArg("THIS", 0), class_loader_catcher())
.include(spy.byMethod("com.ibm.ws.classloader.ProtectionClassLoader", "")));_
We put them in comment. This is the only solution that worked but we don't know the usefulness of these scripts. Do you thinkk that we can continue working without them?
As we are constrained to Upgrade our SDK version as part of our project, we would like to find a solution with this parameter.
Please, if you have any suggestion about this problem or a confirmation about the shared scripts, let us know, we would be thankful.
Best regards,
Nnaks
Following the install steps for Zico 1.0.15, i have it running as such:
/opt/jdk/latest/bin/java -Dzorka.app=zico -Xms512m -Xmx2048m -XX:MaxPermSize=256m -Dlogback.configurationFile=/opt/zico-1.0.15/./logback.xml -Dzico.home.dir=/opt/zico-1.0.15/. -javaagent:/opt/zico-1.0.15/./zorka.jar=/opt/zico-1.0.15/. -jar zico.war
scripts = jvm.bsh, eclipse/jetty.bsh, zico.bsh
zabbix = no
zorka.hostname = zico.localhost
zorka.log.spy = DEBUG
tracer = yes
tracer.net = yes
tracer.net.addr = 127.0.0.1
zorka.spy.compute.frames = yes
zico.http.port = 5703
rds.file.size = 16M
rds.max.size = 2048M
rds.seg.size = 1024K
zico.security = no
auth = anonymous
But, I can't seem to get the ziko.localhost, for itself, to show up in the UI. Everything seems to be working fine; the zorka and zico logs both seem to indicate that there are no issues, I just don't see anything in the UI.
Any ideas? I'm pretty java saavy, from an ops side, and this one has me flummoxed.
Steps to reproduce:
Log from Wildfly:
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:386)
at sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:401)
Caused by: java.lang.NullPointerException
at java.net.Socket.<init>(Socket.java:428)
at java.net.Socket.<init>(Socket.java:244)
at com.jitlogic.zorka.core.integ.zabbix.ZabbixActiveAgent.start(ZabbixActiveAgent.java:165)
at com.jitlogic.zorka.core.AgentInstance.initBshLibs(AgentInstance.java:205)
at com.jitlogic.zorka.core.AgentInstance.start(AgentInstance.java:179)
at com.jitlogic.zorka.agent.AgentMain.premain(AgentMain.java:101)
... 6 more
FATAL ERROR in native method: processing of -javaagent failed
*** JBossAS process (5016) received ABRT signal ***
I have run zorka with a single application, put tracing to file and received not quite correct data:
Object org.springframework.web.context.request.AbstractRequestAttributesScope.get(String, ObjectFactory) took over 1 second and over 65% of entire time.
However it consists of only method that took less than 18%. The method itself is very simple
@Override
public Object get(String name, ObjectFactory<?> objectFactory) {
RequestAttributes attributes = RequestContextHolder.currentRequestAttributes();
Object scopedObject = attributes.getAttribute(name, getScope());
if (scopedObject == null) {
scopedObject = objectFactory.getObject();
attributes.setAttribute(name, scopedObject, getScope());
}
return scopedObject;
}
This code doesn't seem to have anything that can slow down purely to 0,8 seconds.
Also there are AOP methods, but how can I trace how much do they take?
On are systems we run many instances of java. Please allow passing of path to zorka.properties files in line below. That we could have one directory with a few zorka.properties files.
i.e. /opt/zorka/zorka.tomcat0.properties, /opt/zorka/zorka.tomcat0.properties, /opt/zorka/zorka.weglocic0.properties and /opt/zorka/zorka.weglocic0.properties
-javaagent:/opt/zorka/zorka.jar= /opt/zorka/zorka.tomcat0.properties -Djavax.management.builder.initial=weblogic.management.jmx.mbeanserv er.WLSMBeanServerBuilder
Hi team,
Could you please help us in understanding the Method Trace Header names like Time,CTime, Calls, Errors, Pct.
Time - Self time spent in each method
CTime- Not sure as it shows zero always
Calls - I could see huge number of calls been measured under this metric header like 11889 so it should been the number of calls to the method because we executed only one request.
Errors - Number of errors
Pct - Percentage of time spent from the overall response time.
Could you please review and let us know whether our assumptions are correct and clarify on CTime
Thanks
Sattish.
Due to a bug in the ComparatorProcessor, there is no difference between using the "spy.NE" and "spy.EQ" operator in a filter.
It's always the "spy.EQ" logic.
In the "process" method:
} else if (op == EQ || op == NE) {
return ((va == null && vb == null) || (va != null && va.equals(vb))) ? record : null;
}
Should be something like:
} else if (op == EQ) {
return ((va == null && vb == null) || (va != null && va.equals(vb))) ? record : null;
} else if (op == NE) {
return ((va == null && vb == null) || (va != null && !va.equals(vb))) ? record : null;
}
I recompiled it with changed sources and it works perfectly now.
(Hint: If you're a Windows user, make sure there are no blanks in the path of your source directory, otherwise unit tests will fail)
I see a lot of errors when trying to monitor springboot app running on Java 17:
2022-12-02 08:42:11 INFO MBeanServerRegistry Registering ZorkaControl MBean...
2022-12-02 08:42:11 ERROR SpyClassTransformer Error transforming class: com/sun/jmx/mbeanserver/StandardMBeanSupport [java.lang.IllegalArgumentException: Unsupported class file major version 61]
java.lang.IllegalArgumentException: Unsupported class file major version 61
at com.jitlogic.zorka.deps.asm.ClassReader.<init>(ClassReader.java:195)
at com.jitlogic.zorka.deps.asm.ClassReader.<init>(ClassReader.java:176)
at com.jitlogic.zorka.deps.asm.ClassReader.<init>(ClassReader.java:162)
at com.jitlogic.zorka.core.spy.SpyClassTransformer.doTransform(SpyClassTransformer.java:363)
at com.jitlogic.zorka.core.spy.SpyClassTransformer.transform(SpyClassTransformer.java:292)
at java.instrument/java.lang.instrument.ClassFileTransformer.transform(ClassFileTransformer.java:244)
at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:541)
at java.management/com.sun.jmx.mbeanserver.Introspector.makeDynamicMBean(Introspector.java:194)
at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:893)
at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:320)
at java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:523)
at com.jitlogic.zorka.core.mbeans.MBeanServerRegistry.registerZorkaControlDeferred(MBeanServerRegistry.java:288)
at com.jitlogic.zorka.core.mbeans.MBeanServerRegistry.registerZorkaControl(MBeanServerRegistry.java:278)
at com.jitlogic.zorka.agent.AgentMain.premain(AgentMain.java:89)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:491)
at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:503)
2022-12-02 08:42:11 ERROR SpyClassTransformer Error transforming class: com/sun/jmx/mbeanserver/StandardMBeanIntrospector [java.lang.IllegalArgumentException: Unsupported class file major version 61]
java.lang.IllegalArgumentException: Unsupported class file major version 61
at com.jitlogic.zorka.deps.asm.ClassReader.<init>(ClassReader.java:195)
at com.jitlogic.zorka.deps.asm.ClassReader.<init>(ClassReader.java:176)
at com.jitlogic.zorka.deps.asm.ClassReader.<init>(ClassReader.java:162)
at com.jitlogic.zorka.core.spy.SpyClassTransformer.doTransform(SpyClassTransformer.java:363)
at com.jitlogic.zorka.core.spy.SpyClassTransformer.transform(SpyClassTransformer.java:292)
at java.instrument/java.lang.instrument.ClassFileTransformer.transform(ClassFileTransformer.java:244)
at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:541)
at java.management/com.sun.jmx.mbeanserver.StandardMBeanSupport.getMBeanIntrospector(StandardMBeanSupport.java:65)
at java.management/com.sun.jmx.mbeanserver.MBeanSupport.<init>(MBeanSupport.java:137)
at java.management/com.sun.jmx.mbeanserver.StandardMBeanSupport.<init>(StandardMBeanSupport.java:60)
at java.management/com.sun.jmx.mbeanserver.Introspector.makeDynamicMBean(Introspector.java:194)
at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:893)
at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:320)
at java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:523)
at com.jitlogic.zorka.core.mbeans.MBeanServerRegistry.registerZorkaControlDeferred(MBeanServerRegistry.java:288)
at com.jitlogic.zorka.core.mbeans.MBeanServerRegistry.registerZorkaControl(MBeanServerRegistry.java:278)
at com.jitlogic.zorka.agent.AgentMain.premain(AgentMain.java:89)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:491)
at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:503)
2022-12-02 08:42:11 ERROR SpyClassTransformer Error transforming class: sun/launcher/LauncherHelper [java.lang.IllegalArgumentException: Unsupported class file major version 61]
java.lang.IllegalArgumentException: Unsupported class file major version 61
at com.jitlogic.zorka.deps.asm.ClassReader.<init>(ClassReader.java:195)
at com.jitlogic.zorka.deps.asm.ClassReader.<init>(ClassReader.java:176)
at com.jitlogic.zorka.deps.asm.ClassReader.<init>(ClassReader.java:162)
at com.jitlogic.zorka.core.spy.SpyClassTransformer.doTransform(SpyClassTransformer.java:363)
at com.jitlogic.zorka.core.spy.SpyClassTransformer.transform(SpyClassTransformer.java:292)
at java.instrument/java.lang.instrument.ClassFileTransformer.transform(ClassFileTransformer.java:244)
at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:541)
2022-12-02 08:42:11 ERROR SpyClassTransformer Error transforming class: java/lang/Package$VersionInfo [java.lang.IllegalArgumentException: Unsupported class file major version 61]
java.lang.IllegalArgumentException: Unsupported class file major version 61
at com.jitlogic.zorka.deps.asm.ClassReader.<init>(ClassReader.java:195)
at com.jitlogic.zorka.deps.asm.ClassReader.<init>(ClassReader.java:176)
at com.jitlogic.zorka.deps.asm.ClassReader.<init>(ClassReader.java:162)
at com.jitlogic.zorka.core.spy.SpyClassTransformer.doTransform(SpyClassTransformer.java:363)
at com.jitlogic.zorka.core.spy.SpyClassTransformer.transform(SpyClassTransformer.java:292)
at java.instrument/java.lang.instrument.ClassFileTransformer.transform(ClassFileTransformer.java:244)
at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:541)
at java.base/java.lang.Package.<init>(Package.java:554)
at java.base/java.lang.ClassLoader.definePackage(ClassLoader.java:2205)
at java.base/jdk.internal.loader.BuiltinClassLoader.definePackage(BuiltinClassLoader.java:974)
at java.base/jdk.internal.loader.BuiltinClassLoader.defineOrCheckPackage(BuiltinClassLoader.java:881)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.defineOrCheckPackage(ClassLoaders.java:211)
at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:849)
at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:760)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:681)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:639)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:467)
at java.base/sun.launcher.LauncherHelper.loadMainClass(LauncherHelper.java:791)
at java.base/sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:686)
2022-12-02 08:42:11 ERROR SpyClassTransformer Error transforming class: sun/nio/ByteBuffered [java.lang.IllegalArgumentException: Unsupported class file major version 61]
java.lang.IllegalArgumentException: Unsupported class file major version 61
at com.jitlogic.zorka.deps.asm.ClassReader.<init>(ClassReader.java:195)
at com.jitlogic.zorka.deps.asm.ClassReader.<init>(ClassReader.java:176)
at com.jitlogic.zorka.deps.asm.ClassReader.<init>(ClassReader.java:162)
at com.jitlogic.zorka.core.spy.SpyClassTransformer.doTransform(SpyClassTransformer.java:363)
at com.jitlogic.zorka.core.spy.SpyClassTransformer.transform(SpyClassTransformer.java:292)
at java.instrument/java.lang.instrument.ClassFileTransformer.transform(ClassFileTransformer.java:244)
at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:541)
at java.base/jdk.internal.loader.Resource.getByteBuffer(Resource.java:164)
at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:853)
at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:760)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:681)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:639)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:467)
at java.base/sun.launcher.LauncherHelper.loadMainClass(LauncherHelper.java:791)
at java.base/sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:686)
2022-12-02 08:42:11 ERROR SpyClassTransformer Error transforming class: java/net/URLClassLoader [java.lang.IllegalArgumentException: Unsupported class file major version 61]
java.lang.IllegalArgumentException: Unsupported class file major version 61
at com.jitlogic.zorka.deps.asm.ClassReader.<init>(ClassReader.java:195)
at com.jitlogic.zorka.deps.asm.ClassReader.<init>(ClassReader.java:176)
at com.jitlogic.zorka.deps.asm.ClassReader.<init>(ClassReader.java:162)
at com.jitlogic.zorka.core.spy.SpyClassTransformer.doTransform(SpyClassTransformer.java:363)
at com.jitlogic.zorka.core.spy.SpyClassTransformer.transform(SpyClassTransformer.java:292)
at java.instrument/java.lang.instrument.ClassFileTransformer.transform(ClassFileTransformer.java:244)
at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:541)
at java.base/java.lang.ClassLoader.findBootstrapClass(Native Method)
at java.base/java.lang.ClassLoader.findBootstrapClassOrNull(ClassLoader.java:1262)
at java.base/java.lang.System$2.findBootstrapClassOrNull(System.java:2310)
at java.base/jdk.internal.loader.ClassLoaders$BootClassLoader.loadClassOrNull(ClassLoaders.java:140)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:700)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:669)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:639)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
at java.base/java.lang.ClassLoader.defineClass1(Native Method)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1012)
at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150)
at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:862)
at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:760)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:681)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:639)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
at java.base/java.lang.Class.getDeclaredMethods0(Native Method)
at java.base/java.lang.Class.privateGetDeclaredMethods(Class.java:3402)
at java.base/java.lang.Class.getMethodsRecursive(Class.java:3543)
at java.base/java.lang.Class.getMethod0(Class.java:3529)
at java.base/java.lang.Class.getMethod(Class.java:2225)
at java.base/sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:847)
at java.base/sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:706)
2022-12-02 08:42:11 ERROR SpyClassTransformer Error transforming class: java/nio/file/attribute/PosixFilePermission [java.lang.IllegalArgumentException: Unsupported class file major version 61]
java.lang.IllegalArgumentException: Unsupported class file major version 61
at com.jitlogic.zorka.deps.asm.ClassReader.<init>(ClassReader.java:195)
at com.jitlogic.zorka.deps.asm.ClassReader.<init>(ClassReader.java:176)
at com.jitlogic.zorka.deps.asm.ClassReader.<init>(ClassReader.java:162)
at com.jitlogic.zorka.core.spy.SpyClassTransformer.doTransform(SpyClassTransformer.java:363)
at com.jitlogic.zorka.core.spy.SpyClassTransformer.transform(SpyClassTransformer.java:292)
at java.instrument/java.lang.instrument.ClassFileTransformer.transform(ClassFileTransformer.java:244)
at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:541)
at java.base/java.lang.ClassLoader.findBootstrapClass(Native Method)
at java.base/java.lang.ClassLoader.findBootstrapClassOrNull(ClassLoader.java:1262)
at java.base/java.lang.System$2.findBootstrapClassOrNull(System.java:2310)
at java.base/jdk.internal.loader.ClassLoaders$BootClassLoader.loadClassOrNull(ClassLoaders.java:140)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:700)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:669)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:639)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
at org.springframework.boot.loader.Launcher.createArchive(Launcher.java:163)
at org.springframework.boot.loader.ExecutableArchiveLauncher.<init>(ExecutableArchiveLauncher.java:53)
at org.springframework.boot.loader.JarLauncher.<init>(JarLauncher.java:42)
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65)
2022-12-02 08:42:11 ERROR SpyClassTransformer Error transforming class: java/lang/IndexOutOfBoundsException [java.lang.IllegalArgumentException: Unsupported class file major version 61]
java.lang.IllegalArgumentException: Unsupported class file major version 61
at com.jitlogic.zorka.deps.asm.ClassReader.<init>(ClassReader.java:195)
at com.jitlogic.zorka.deps.asm.ClassReader.<init>(ClassReader.java:176)
at com.jitlogic.zorka.deps.asm.ClassReader.<init>(ClassReader.java:162)
at com.jitlogic.zorka.core.spy.SpyClassTransformer.doTransform(SpyClassTransformer.java:363)
at com.jitlogic.zorka.core.spy.SpyClassTransformer.transform(SpyClassTransformer.java:292)
at java.instrument/java.lang.instrument.ClassFileTransformer.transform(ClassFileTransformer.java:244)
at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:541)
at java.base/java.lang.ClassLoader.findBootstrapClass(Native Method)
at java.base/java.lang.ClassLoader.findBootstrapClassOrNull(ClassLoader.java:1262)
at java.base/java.lang.System$2.findBootstrapClassOrNull(System.java:2310)
at java.base/jdk.internal.loader.ClassLoaders$BootClassLoader.loadClassOrNull(ClassLoaders.java:140)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:700)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:669)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:639)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
at org.springframework.boot.loader.jar.JarFile.<clinit>(JarFile.java:64)
at org.springframework.boot.loader.archive.JarFileArchive.<init>(JarFileArchive.java:73)
at org.springframework.boot.loader.archive.JarFileArchive.<init>(JarFileArchive.java:69)
at org.springframework.boot.loader.Launcher.createArchive(Launcher.java:163)
at org.springframework.boot.loader.ExecutableArchiveLauncher.<init>(ExecutableArchiveLauncher.java:53)
at org.springframework.boot.loader.JarLauncher.<init>(JarLauncher.java:42)
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65)
Hi, I first tryed with jboss7.1.1 standalone and zorka started ok
Now I'm using:
zorka-1.0.7
RHEL-6.4
EAP-6.0.1.zip
zabbix-2.2.6
I followed zorka configuration instrucctions for jboss7 domain mode (http://zorka.io/install/jboss7.html), and started domain.sh, then host-controller and process-controller processes started, but server instance didn't start.
On ../domain/log/process-controller.log showed java.IO.Exception BROKEN PIPE.
And ../domain/servers/as0/log/boot.log and server.log were not touched on the try.
This is not a desired configuration, but I could start ok, with only one server instance, by adding -javaagent:/path_to/zorka.jar=/path_to on JAVA_OPTS in file domain.conf.
Zabbix is even reading data from zorka agent ;-)
Maybe is something that I'm missing after reading the instrucctions.
Please can you provide more detailed instruccions on how to configure zorca to work with jboss domain for multiple instances.
I can make any test you need.
Thanks.
Hello!
We've got a problem with Zorka after upgrading to Zabbix 4.0:
# zabbix_get -s SERVER -p 10055 -k 'zorka.jmx["java","java.lang:type=Runtime","Uptime"]'
zabbix_get [16172]: Timeout while executing operation
But, with telnet, it works:
#telnet SERVER 10055
Trying 10.70.2.24...
Connected to SERVER.
Escape character is '^]'.
zorka.jmx["java","java.lang:type=Runtime","Uptime"]
ZBXD33784774Connection closed by foreign host.
The same happens with Zorka Dev. version (1.90.4).
I think this might be a bug related to this:
Support for the plain text protocol has been dropped and a header is now mandatory. A header has been added to Zabbix get requests, Zabbix server/proxy passive check requests and frontend requests to Zabbix server. As a consequence, Zabbix agents that are older than version 1.4 are no longer supported. Also, messages from self-written senders will be rejected if the header is absent. Whereas previously Zabbix trappers would accept messages without headers as well as messages with headers, now they will only accept messages with protocol header.
https://www.zabbix.com/documentation/current/manual/installation/upgrade_notes_400
If it's not a bug, I would appreciate any help on debugging/solving this.
Thank you!
Add trace marker flag to ignore creation of new trace if there is already one of the same tag being collected. This would be useful in some cases when traced code executes itself through traced entry points (eg. mutually dependent EJB code).
Hi!
We're giving Zorka agent a try regarding monitoring of application servers via Zabbix.
Now some applications stopped working after deploying Zorka agent. It turned out that the reason for that is somehow connected to having 'Single class loader for application' selected in 'WAR class loader policy'.
Since the AS in question can't be configured to use a 'Class loader for each WAR file in application' (dunno why, it's a statement from the developers), I wonder if this is something that can be "fixed' or worked-around in Zorka agent.
This is a log message I got from an application administrator encountering this issue:
WebContainer E com.ibm.ws.webcontainer.internal.WebContainer handleRequest SRVE0255E: A WebGroup/Virtual Host to handle ***.jsp has not been defined.
Another possibly important information is that the application itself completely stopped logging when Zorka was deployed.
I recall a similar issues when using Zorka agent on some of our JBoss applications servers. Can't give any details yet since the respective admin is currently on vacation.
Environment:
Not an issue but feature request:
It would be very helpful if Zorka could support TLS - both certificate based TLS and TLS PSK, in both passive and active modes. This would make Zorka complete equivalent to Zabbix Java gateway in terms of supported features.
Does it works with zabbix 5.4?
Am I supposed to add something in the data folder ?
SpyContext structures should be allocated for fully instrumented methods (spy instrumentation) but are unnecessary for methods that are only traces. There are potentially tens of thousands of traced methods, so these structures can consume considerable amount of memory.
Hi,
When I start tomcat with zorka.jar, it report follow errors. please help me, thanks
utoProxyCreator,mappingJacksonHttpMessageConverter,org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter#0,viewResolver,multipartResolver]; parent: org.springframework.beans.factory.support.DefaultListableBeanFactory@3dcc937c
WARN 2015-09-09 10:53:11,584 org.springframework.web.context.support.XmlWebApplicationContext Exception encountered during context initialization - cancelling refresh attempt
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'viewResolver' defined in class path resource [spring-mvc.xml]: Instantiation of bean failed; nested exception is java.lang.VerifyError: (class: org/springframework/web/servlet/view/AbstractCachingViewResolver$1, method: render signature: (Ljava/util/Map;Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V) Stack size too large
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:1039)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:985)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:487)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:458)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:293)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:223)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:290)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:191)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:636)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:934)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:479)
at org.springframework.web.servlet.FrameworkServlet.configureAndRefreshWebApplicationContext(FrameworkServlet.java:651)
at org.springframework.web.servlet.FrameworkServlet.createWebApplicationContext(FrameworkServlet.java:602)
at org.springframework.web.servlet.FrameworkServlet.createWebApplicationContext(FrameworkServlet.java:665)
at org.springframework.web.servlet.FrameworkServlet.initWebApplicationContext(FrameworkServlet.java:521)
at org.springframework.web.servlet.FrameworkServlet.initServletBean(FrameworkServlet.java:462)
at org.springframework.web.servlet.HttpServletBean.init(HttpServletBean.java:136)
at javax.servlet.GenericServlet.init(GenericServlet.java:158)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1284)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1197)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1087)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5231)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5518)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1575)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1565)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Not an issue (I just failed to find place where I can ask about it):
Is there a way (configuration) to prevent any instrumentation of Java classes (byte code changes) and to use Zorka just as a bridge b/w Zabbix server and JMX?
What Zorka configuration options should I use to avoid instrumentation - for example, to avoid changes in JBOSS_MODULES_SYSTEM_PKGS when integrating Zorka into JBoss / WildFly (http://zorka.io/p/docs/install/appsrv/wildfly/)?
Trying to configure the Agent for Jeus server. What changes do I need to get the BCI working?
There are cases where sing symbols as trace record attributes would save a lot of space in trace file.
When I used Zorka in active mode with the Zabbix integrtion, the first round of checks to do works fine. However, when Zabbix discovers new checks to do, Zorka doesn't use them.
The root cause of the problem is in ActiveCheckQueryItem.equals which doesn't check the equality of the key string as long as the key string isn't null. Brackets are missing in the code to align the if/else conditions correctly.
I pushed a fix for it in this fork :
https://github.com/dd00f/zorka
I also included a quick change to use hash sets to compare the list of existing active checks vs the new checks to perform to speed up execution. I had over 1000 checks to perform in my deployment.
Hi,
Obtaining error while trying to use Zorka with application in WAS:
2023-09-17 18:06:14 ERROR ZorkaBshAgent Error executing script servers/was.bsh [com.jitlogic.zorka.deps.bsh.EvalError: Error: : at Line: 18 : in file: servers/was.bsh : zorka .require ( "jvm.bsh" , "ejb.bsh" , "http.bsh" , "jms.bsh" , "apache/cxf.bsh" )
]
com.jitlogic.zorka.deps.bsh.EvalError: Error: : at Line: 18 : in file: servers/was.bsh : zorka .require ( "jvm.bsh" , "ejb.bsh" , "http.bsh" , "jms.bsh" , "apache/cxf.bsh" )
at com.jitlogic.zorka.deps.bsh.Interpreter.eval(Interpreter.java:691)
at com.jitlogic.zorka.core.ZorkaBshAgent.loadScript(ZorkaBshAgent.java:182)
at com.jitlogic.zorka.core.ZorkaBshAgent.require(ZorkaBshAgent.java:210)
at com.jitlogic.zorka.core.ZorkaBshAgent.probe(ZorkaBshAgent.java:241)
at com.jitlogic.zorka.core.spy.SpyClassTransformer.doTransform(SpyClassTransformer.java:313)
at com.jitlogic.zorka.core.spy.SpyClassTransformer.transform(SpyClassTransformer.java:292)
at sun.instrument.TransformerManager.transform(TransformerManager.java:200)
at sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:450)
at java.lang.ClassLoader.defineClassImpl(Native Method)
at java.lang.ClassLoader.defineClassInternal(ClassLoader.java:391)
at java.lang.ClassLoader.defineClass(ClassLoader.java:360)
at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.defineClass(DefaultClassLoader.java:188)
at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.defineClass(ClasspathManager.java:580)
at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findClassImpl(ClasspathManager.java:550)
at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findLocalClassImpl(ClasspathManager.java:481)
at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findLocalClass_LockClassName(ClasspathManager.java:460)
at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findLocalClass(ClasspathManager.java:447)
at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.findLocalClass(DefaultClassLoader.java:216)
at org.eclipse.osgi.internal.loader.BundleLoader.findLocalClass(BundleLoader.java:393)
at org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:469)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:422)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:410)
at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.loadClass(DefaultClassLoader.java:107)
at java.lang.ClassLoader.loadClass(ClassLoader.java:865)
at java.lang.ClassLoader.defineClassImpl(Native Method)
at java.lang.ClassLoader.defineClassInternal(ClassLoader.java:391)
at java.lang.ClassLoader.defineClass(ClassLoader.java:360)
at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.defineClass(DefaultClassLoader.java:188)
at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.defineClass(ClasspathManager.java:580)
at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findClassImpl(ClasspathManager.java:550)
at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findLocalClassImpl(ClasspathManager.java:481)
at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findLocalClass_LockClassName(ClasspathManager.java:460)
at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findLocalClass(ClasspathManager.java:447)
at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.findLocalClass(DefaultClassLoader.java:216)
at org.eclipse.osgi.internal.loader.BundleLoader.findLocalClass(BundleLoader.java:393)
at org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:469)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:422)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:410)
at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.loadClass(DefaultClassLoader.java:107)
at java.lang.ClassLoader.loadClass(ClassLoader.java:865)
at com.ibm.ws.sib.api.jms.impl.JmsSessionImpl.instantiateConsumer(JmsSessionImpl.java:1468)
at com.ibm.ws.sib.api.jms.impl.JmsSessionImpl.createConsumer(JmsSessionImpl.java:951)
at com.ibm.ws.sib.api.jms.impl.JmsSessionImpl.createConsumer(JmsSessionImpl.java:888)
at psdi.iface.jms.JMSConsumer.createClient(JMSConsumer.java:248)
at psdi.iface.jms.JMSConsumer.<init>(JMSConsumer.java:158)
at psdi.iface.jms.JMSConsumer.<init>(JMSConsumer.java:129)
at psdi.iface.jms.JMSConsumer.<init>(JMSConsumer.java:92)
at psdi.iface.jms.JMSQueueConsumer.<init>(JMSQueueConsumer.java:46)
at psdi.iface.jms.JMSQueueCronTask.createConsumer(JMSQueueCronTask.java:242)
at psdi.iface.jms.JMSQueueCronTask.cacheResources(JMSQueueCronTask.java:179)
at psdi.iface.jms.JMSQueueCronTask.refreshSettings(JMSQueueCronTask.java:221)
at psdi.iface.jms.JMSQueueCronTask.cronAction(JMSQueueCronTask.java:258)
at psdi.server.CronTaskManager.callCronMethod(CronTaskManager.java:2471)
at psdi.server.CronTaskManager.access$1100(CronTaskManager.java:104)
at psdi.server.CronTaskManager$CronThread.run(CronTaskManager.java:3362)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:522)
at java.util.concurrent.FutureTask.run(FutureTask.java:277)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:191)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.lang.Thread.run(Thread.java:818)
Caused by: com.jitlogic.zorka.deps.bsh.ParseException: Parse error at line 210, column 12. Encountered: (
at com.jitlogic.zorka.deps.bsh.Parser.generateParseException(Parser.java:5998)
at com.jitlogic.zorka.deps.bsh.Parser.jj_consume_token(Parser.java:5946)
at com.jitlogic.zorka.deps.bsh.Parser.Statement(Parser.java:2708)
at com.jitlogic.zorka.deps.bsh.Parser.BlockStatement(Parser.java:2814)
at com.jitlogic.zorka.deps.bsh.Parser.Block(Parser.java:2778)
at com.jitlogic.zorka.deps.bsh.Parser.MethodDeclaration(Parser.java:411)
at com.jitlogic.zorka.deps.bsh.Parser.BlockStatement(Parser.java:2809)
at com.jitlogic.zorka.deps.bsh.Parser.Line(Parser.java:202)
at com.jitlogic.zorka.deps.bsh.Interpreter.Line(Interpreter.java:1012)
at com.jitlogic.zorka.deps.bsh.Interpreter.eval(Interpreter.java:649)
... 61 more
Can you help with this, please?
I read documentation. By it doesn't have clear way how to run zorka with spring boot app by steps
My steps
Agent fails to bind on its listening port because it is used. Some delay and/or retry might be necessary.
I have installed zorka 1.0.14 to trace my application (Java-based on tomcat, spring is used)
It traced fine with tomcat, but I need extra information about how spring is being loaded.
Thus I followed this doc: http://zorka.io/install/spring.html
and added spring.bsh to my zorka.properties file.
scripts = jvm.bsh, zabbix.bsh, apache/tomcat.bsh, spring.bsh
To my surprise my application started failing for very weird reason
zorka_log.txt
I have tried to compile and run it with JDK 1.8 and JDK 1.7
The same happens when I add any of bsh script (I have tried also javax.bsh)
Please help
Hi,
Today I run for test zorka/zico 1.0.4, but when I logging to zorka panel with admin/zico I have not available menu (manage user, backup... etc). This is a configuration problem or something else?
I've been trying to setup the environment for a Weblogic12c application
After adapting the JAVA_OPTIONS
And I get the following error
weblogic.Server
Error opening zip file or JAR manifest missing : /home/dev/Middleware/WLS12.1.1/wlserver_12.1/zorca/zorka.jar
Does someone know this issue ?
thanks in advance
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google โค๏ธ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.