opentsdb / opentsdb-elasticsearch Goto Github PK
View Code? Open in Web Editor NEWSearch plugin for OpenTSDB
License: GNU Lesser General Public License v2.1
Search plugin for OpenTSDB
License: GNU Lesser General Public License v2.1
___ _____ ____ ____ ____ / _ \ _ __ ___ _ _|_ _/ ___|| _ \| __ ) | | | | '_ \ / _ \ '_ \| | \___ \| | | | _ \ | |_| | |_) | __/ | | | | ___) | |_| | |_) | \___/| .__/ \___|_| |_|_| |____/|____/|____/ |_| The modern time series database. OpenTSDB is a distributed, scalable Time Series Database (TSDB) written on top of HBase. OpenTSDB was written to address a common need: store, index and serve metrics collected from computer systems (network gear, operating systems, applications) at a large scale, and make this data easily accessible and graphable. Thanks to HBase's scalability, OpenTSDB allows you to collect thousands of metrics from tens of thousands of hosts and applications, at a high rate (every few seconds). OpenTSDB will never delete or downsample data and can easily store hundreds of billions of data points. OpenTSDB is free software and is available under both LGPLv2.1+ and GPLv3+. Find more about OpenTSDB at http://opentsdb.net
As follows:
uidmeta:
doc1. {"uid":"1", type:"METRIC", name:"sys.cpu.usage"}
doc2. {"uid":"1", type:"TAGK", name:"host.name"}
the document "doc1" will be overrided by "doc2"
source codes:
final StringBuilder uri = new StringBuilder(es.host())
.append("/")
.append(es.index())
.append("/")
.append(doc_type)
.append("/")
.append(meta.getUID());
so, i just add uid type(metric, tagk, tagv) before uid value。
as follows:
final StringBuilder uri = new StringBuilder(es.host())
.append("/")
.append(es.index())
.append("/")
.append(doc_type)
.append("/")
.append(meta.getType().name().toLowerCase()+meta.getUID());
Not very familiar with Java, so this could be my fault. I'm trying to build this, and edited the following section of pom file:
<dependency>
<groupId>net.opentsdb</groupId>
<artifactId>opentsdb</artifactId>
<version>2.0.0</version>
<scope>system</scope>
<systemPath>/home/vagrant/opentsdb-2.0.0/build/tsdb-2.0.0.jar</systemPath>
</dependency>
However, when I try to build I get ElasticSearch is not abstract and does not override abstract method collectStats(StatsCollector) in SearchPlugin
:
Full maven output:
[vagrant@localhost opentsdb-elasticsearch]$ mvn clean install
[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for net.opentsdb:otsdb-elasticsearch:jar:1.0
[WARNING] 'dependencies.dependency.systemPath' for net.opentsdb:opentsdb:jar should use a variable instead of a hard-coded path /home/vagrant/opentsdb-2.0.0/build/tsdb-2.0.0.jar @ line 56, column 14
[WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-source-plugin is missing. @ line 97, column 21
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building otsdb-elasticsearch 1.0
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ otsdb-elasticsearch ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ otsdb-elasticsearch ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /home/vagrant/opentsdb-elasticsearch/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ otsdb-elasticsearch ---
[WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent!
[INFO] Compiling 2 source files to /home/vagrant/opentsdb-elasticsearch/target/classes
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /home/vagrant/opentsdb-elasticsearch/src/net/opentsdb/search/ElasticSearch.java:[59,13] error: ElasticSearch is not abstract and does not override abstract method collectStats(StatsCollector) in SearchPlugin
[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.793s
[INFO] Finished at: Fri Sep 13 14:43:06 UTC 2013
[INFO] Final Memory: 11M/27M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile (default-compile) on project otsdb-elasticsearch: Compilation failure
[ERROR] /home/vagrant/opentsdb-elasticsearch/src/net/opentsdb/search/ElasticSearch.java:[59,13] error: ElasticSearch is not abstract and does not override abstract method collectStats(StatsCollector) in SearchPlugin
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
Hi,
Can you provide a jar file that I can use directly with opentsdb? It will be easier than building from source.
Thank you
I am trying to install elasticsearch and in the step of "Add a mapping for each JSON file in the ./scripts folder via: curl -X PUT -d @scripts/tsmeta_mapping.json http:///opentsdb/tsmeta/_mapping curl -X PUT -d @scripts/uidmeta_mapping.json http:///opentsdb/uidmeta/_mapping curl -X PUT -d @scripts/annotation_mapping.json http:///opentsdb/annotation/_mapping", I came to know that the json files are tsmeta_map.json, uidmeta_map.json, annotation_map.json rather than tsmeta_mapping.json, uidmeta_mapping.json, annotation_mapping.json. And after changing this also it is throwing error as "curl: (6) Couldn't resolve host 'opentsdb". Are there any changes in url or am I doing wrong anywhere else.
Thanks,
Sri.
hi,when i install this plugin,i run metasync but get these error.
MetaSync: Unknown exception processing row: [KeyValue(key=[0, 5, 7, 90, -90, -121, -32, 0, 0, 1, 0, 0, 112, 0, 0, 19, 0, 6, -117, 0, 0, 20, 0, 6, -99, 0, 0, 21, 0, 0, -77, 0, 0, 22, 0, 6, -116], family="t", qualifier=[-108, -69, -100, 59, -93, -69, -85, 59, -78, -69, -70, 59, -63, -69, -55, 59, -48, -69, -40, 59, -33, -69], value=[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], timestamp=1520866779000)]
java.lang.NullPointerException: null
at net.opentsdb.search.TSMetaAugment.serializeToBytes(TSMetaAugment.java:53) ~[opentsdb-elasticsearch-1.1.0-SNAPSHOT.jar:]
at net.opentsdb.search.ElasticSearch.indexTSMeta(ElasticSearch.java:175) ~[opentsdb-elasticsearch-1.1.0-SNAPSHOT.jar:]
at net.opentsdb.core.TSDB.indexTSMeta(TSDB.java:1958) ~[tsdb-2.4.0RC2.jar:]
at net.opentsdb.tools.MetaSync$1TSMetaCB$1CounterCB.call(MetaSync.java:304) ~[tsdb-2.4.0RC2.jar:]
at net.opentsdb.tools.MetaSync$1TSMetaCB$1CounterCB.call(MetaSync.java:292) ~[tsdb-2.4.0RC2.jar:]
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1278) [async-1.4.0.jar:na]
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1257) [async-1.4.0.jar:na]
at com.stumbleupon.async.Deferred.callback(Deferred.java:1005) [async-1.4.0.jar:na]
at org.hbase.async.HBaseRpc.callback(HBaseRpc.java:712) [asynchbase-1.8.0.jar:na]
at org.hbase.async.RegionClient.decode(RegionClient.java:1536) [asynchbase-1.8.0.jar:na]
at org.hbase.async.RegionClient.decode(RegionClient.java:88) [asynchbase-1.8.0.jar:na]
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:500) [netty-3.9.4.Final.jar:na]
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:485) [netty-3.9.4.Final.jar:na]
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) [netty-3.9.4.Final.jar:na]
at org.hbase.async.RegionClient.handleUpstream(RegionClient.java:1226) [asynchbase-1.8.0.jar:na]
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [netty-3.9.4.Final.jar:na]
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [netty-3.9.4.Final.jar:na]
at org.jboss.netty.channel.SimpleChannelHandler.messageReceived(SimpleChannelHandler.java:142) [netty-3.9.4.Final.jar:na]
at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:88) [netty-3.9.4.Final.jar:na]
at org.jboss.netty.handler.timeout.IdleStateAwareChannelHandler.handleUpstream(IdleStat
Following the instructions, i get the following error:
644 [INFO] ------------------------------------------------------------------------
645 [INFO] BUILD FAILURE
646 [INFO] ------------------------------------------------------------------------
647 [INFO] Total time: 55.073 s
648 [INFO] Finished at: 2015-10-25T16:33:25-07:00
649 [INFO] Final Memory: 16M/310M
650 [INFO] ------------------------------------------------------------------------
651 [ERROR] Failed to execute goal on project otsdb-elasticsearch: Could not resolve dependencies for project net.opentsdb:otsdb-elasticsearch:jar:2.0.0: Could not find artifact net.opentsdb:opentsdb:jar:2.0.0 in mcaprari-releases (https://github.com/mcaprari/mcaprari-maven-repo/raw/master/releases) -> [Help 1]
652 [ERROR]
653 [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
654 [ERROR] Re-run Maven using the -X switch to enable full debug logging.
655 [ERROR]
ElasticSearch 6.x remove the multi type support, the reason and other information are as follows:
https://www.elastic.co/guide/en/elasticsearch/reference/6.0/removal-of-types.html
So, I just upgrade opentsdb version to 2.4.0 and elasticsearch support to 6.8.0
The plugin address:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.