This project forked from veepee-oss/influxdb-relay
Service to online route, filter, modify and replicate metrics from InfluxDB or prometheus sources to InfluxDB DB's
License: MIT License
Thanks for visiting my GitHub profile, it's great to meet you here! 😊
Hi, I'm Toni Moreno from Barcelona.
GitHub Stats:
we deploy skywalking(https://github.com/apache/skywalking) and use influxdb-srelay for influxdb ha.skywalkig proejct use influxdb-java client throws many errors.
we fix x-influxdb-version error,then find another error
Could be interesting if working with several config files...
/etc/influxdb-srelay/srelay-main.conf
/etc/influxdb-srelay/conf.d/http_server_1.conf
/etc/influxdb-srelay/conf.d/http_server_2.conf
/etc/influxdb-srelay/conf.d/http_server_3.conf
/etc/influxdb-srelay/conf.d/cluster_1.conf
/etc/influxdb-srelay/conf.d/cluster_2.conf
/etc/influxdb-srelay/conf.d/cluster_3.conf
Hi,
InfluxDB-sRelay: 0.4
After changing loglevel on the config file and do a reload it doesn't seems to be reflected on the log.
log-file = "/logs/apps/influxdb-srelay/cloud_cluster.log"
-log-level = "debug"
+log-level = "info"In the following configuration rule, the value_on_unmatch ( db will take the point if not match, the tag namespace) is not working as it doesn't exist
[[http.endpoint.route.rule]]
name="to_namespace_metrics"
action="route_db_from_data"
key="tag"
key_aux="namespace"
match="(.*)"
value="$1"
value_on_unmatch="db_test_1"
to_cluster="cluster_caas"Hello,
The agent panics whenever there is a network connectivity delay or partition and sometimes randomly without valid cause
2019/11/29 12:19:09 starting relays...
2019/11/29 12:19:09 Running relay cympa-ha-relay...
2019/11/29 15:37:48 http: panic serving 127.0.0.1:37094: runtime error: invalid memory address or nil pointer dereference
goroutine 6709 [running]:
net/http.(*conn).serve.func1(0xc4201fa140)
/usr/lib/go-1.10/src/net/http/server.go:1726 +0xd0
panic(0x7fdec0, 0xab7170)
/usr/lib/go-1.10/src/runtime/panic.go:502 +0x229
github.com/toni-moreno/influxdb-srelay/cluster.(*Cluster).handleQuery(0xc4201c4120, 0x8bce20, 0xc4201421c0, 0xc4201ae900, 0xc420179480)
/home/hanine/go/src/github.com/toni-moreno/influxdb-srelay/cluster/influxcluster.go:436 +0x20e
github.com/toni-moreno/influxdb-srelay/cluster.(*Cluster).handleQueryHA(0xc4201c4120, 0x8bce20, 0xc4201421c0, 0xc4201ae900)
/home/hanine/go/src/github.com/toni-moreno/influxdb-srelay/cluster/influxcluster.go:405 +0x1c6
github.com/toni-moreno/influxdb-srelay/cluster.(*Cluster).(github.com/toni-moreno/influxdb-srelay/cluster.handleQueryHA)-fm(0x8bce20, 0xc4201421c0, 0xc4201ae900)
/home/hanine/go/src/github.com/toni-moreno/influxdb-srelay/cluster/influxcluster.go:88 +0x48
github.com/toni-moreno/influxdb-srelay/relay.(*RouteRule).ActionRouteHTTP(0xc420179600, 0x8bce20, 0xc4201421c0, 0xc4201ae900, 0xc420099dd0, 0x0, 0x0, 0xc4201805c8)
/home/hanine/go/src/github.com/toni-moreno/influxdb-srelay/relay/http_route.go:133 +0x470
github.com/toni-moreno/influxdb-srelay/relay.(*RouteRule).ActionRouteHTTP-fm(0x8bce20, 0xc4201421c0, 0xc4201ae900, 0xc420099dd0, 0x9, 0xc420082540, 0x100)
/home/hanine/go/src/github.com/toni-moreno/influxdb-srelay/relay/http_route.go:357 +0x52
github.com/toni-moreno/influxdb-srelay/relay.(*HTTPRoute).ProcessRules(0xc4201725b0, 0x8bce20, 0xc4201421c0, 0xc4201ae900, 0xc420099dd0)
/home/hanine/go/src/github.com/toni-moreno/influxdb-srelay/relay/http_route.go:628 +0x156
github.com/toni-moreno/influxdb-srelay/relay.(*HTTPEndPoint).ProcessRead(0xc4201795c0, 0x8bce20, 0xc4201421c0, 0xc4201ae900, 0xc420099dd0)
/home/hanine/go/src/github.com/toni-moreno/influxdb-srelay/relay/http_endpoint.go:76 +0x372
github.com/toni-moreno/influxdb-srelay/relay.(*HTTPEndPoint).ProcessRead-fm(0x8bce20, 0xc4201421c0, 0xc4201ae900, 0xc420099dd0)
/home/hanine/go/src/github.com/toni-moreno/influxdb-srelay/relay/http_endpoint.go:26 +0x52
github.com/toni-moreno/influxdb-srelay/relay.(*HTTPEndPoint).ProcessInput(0xc4201795c0, 0x8bce20, 0xc4201421c0, 0xc4201ae900, 0x3)
/home/hanine/go/src/github.com/toni-moreno/influxdb-srelay/relay/http_endpoint.go:129 +0x268
github.com/toni-moreno/influxdb-srelay/relay.(*HTTP).processEndpoint(0xc4201c41b0, 0x8bce20, 0xc4201421c0, 0xc4201ae900)
/home/hanine/go/src/github.com/toni-moreno/influxdb-srelay/relay/http.go:182 +0x1c7
github.com/toni-moreno/influxdb-srelay/relay.(*HTTP).bodyMiddleWare.func1(0xc4201c41b0, 0x8bce20, 0xc4201421c0, 0xc4201ae900)
/home/hanine/go/src/github.com/toni-moreno/influxdb-srelay/relay/http_middlewares.go:37 +0x10d
github.com/toni-moreno/influxdb-srelay/relay.(*HTTP).queryMiddleWare.func1(0xc4201c41b0, 0x8bce20, 0xc4201421c0, 0xc4201ae900)
/home/hanine/go/src/github.com/toni-moreno/influxdb-srelay/relay/http_middlewares.go:57 +0x19e
github.com/toni-moreno/influxdb-srelay/relay.(*HTTP).logMiddleWare.func1(0xc4201c41b0, 0x8bce20, 0xc4201421c0, 0xc4201ae900)
/home/hanine/go/src/github.com/toni-moreno/influxdb-srelay/relay/http_middlewares.go:66 +0xad
github.com/toni-moreno/influxdb-srelay/relay.(*HTTP).rateMiddleware.func1(0xc4201c41b0, 0x8bce20, 0xc4201421c0, 0xc4201ae900)
/home/hanine/go/src/github.com/toni-moreno/influxdb-srelay/relay/http_middlewares.go:100 +0xb9
github.com/toni-moreno/influxdb-srelay/relay.(*HTTP).ServeHTTP(0xc4201c41b0, 0x8bce20, 0xc4201421c0, 0xc4201ac300)
/home/hanine/go/src/github.com/toni-moreno/influxdb-srelay/relay/http.go:201 +0x165
net/http.serverHandler.ServeHTTP(0xc4201b24e0, 0x8bce20, 0xc4201421c0, 0xc4201ac300)
/usr/lib/go-1.10/src/net/http/server.go:2697 +0xbc
net/http.(*conn).serve(0xc4201fa140, 0x8bd2e0, 0xc4200b03c0)
/usr/lib/go-1.10/src/net/http/server.go:1830 +0x651
created by net/http.(*Server).Serve
/usr/lib/go-1.10/src/net/http/server.go:2798 +0x27b
/health endpoint?using package 'github.com/influxdata/influxdb-client-go/v2' write data to srelay.
client log: influxdb2client E! Write error: Unexpected status code 400
srelay log: 2021-03-12 03:14:48 INF bk_duration_ms=2.608922 duration_ms=11.53337 latency_ms=8.928223 method=POST referer= returnsize=33 source=100.101.78.0:64221 status=400 trace-route="http:http-influxdbcluster> rt:pass_all> decode:ILP> rule:pass_all> " url=/api/v2/write?bucket=fcs%2Fautogen&org=&precision=ns user=- user-agent="influxdb-client-go/2.2.2 (windows; amd64)" write-points=100 write-size=45588
conf:
` # ILP /api/v2/write Endpoint
[[http.endpoint]]
uri=["/api/v2/write"]
source_format="ILP"
type = "WR"
[[http.endpoint.route]]
name="pass_all"
log-inherit = true
level="data" # any of the following rules will need data inspection
[[http.endpoint.route.filter]]
name="pass_all"
key="db"
match=".*"
[[http.endpoint.route.rule]]
name="pass_all"
action="route"
key="db"
match=".*"
to_cluster="influxcluster"`
On influxdb-relay
# curl -I localhost:9096/health
HTTP/1.1 200 OK
Content-Length: 135
Content-Type: application/json
Date: Mon, 13 May 2019 09:39:41 GMT
# curl localhost:9096/health
{"status":"healthy","healthy":{"influxdb01":"OK. Time taken 755.098µs","influxdb02":"OK. Time taken 830.941µs"}}Now , no response , only HTTP 200..
# curl -I localhost:9096/health
HTTP/1.1 200 OK
Date: Mon, 13 May 2019 09:44:05 GMT
# curl localhost:9096/health
```We are trying out your container for influxdb-srelay and trying to figure out how to set up basic replication and read/write HA proxying.
The sample configuration in /examples seems like it's a little more complex than that? Or are all the http.endpoint definitions required to get that to work?
Finally the rpm has been installed , and it works well but the error message apears on install and also on some systemctl actions.
$> rpm -ivh /root/influxdb-srelay-latest.x86_64.rpm
Preparing... ################################# [100%]
Updating / installing...
1:influxdb-srelay-latest-1 ################################# [100%]
Created symlink from /etc/systemd/system/influxdb-srelay.service to /usr/lib/systemd/system/influxdb-srelay.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/influxdb-srelay.service to /usr/lib/systemd/system/influxdb-srelay.service.
Failed to execute operation: Too many levels of symbolic links
warning: %posttrans(influxdb-srelay-latest-1.x86_64) scriptlet failed, exit status 1
$> systemctl enable influxdb-srelay
Failed to execute operation: Too many levels of symbolic linksa HTTP 400 is shown
2019-06-14 15:39:42 INF bk_duration_ms=0.025887 duration_ms=307.649156 latency_ms=307.627725 method=GET referer=http://xxxxxxxxxx/d/000000165/netscout-infinistream?refresh=5m&orgId=2 returnsize=0 source=XXXXXXXX status=400 trace-route="http:metrics-be-pmere> rule:route-db> handleQuerySingle:cluster_asp15> " url=/query?db=pmere_metrics&epoch=ms&q=SELECT+mean%28%22platformTemperatureValue%22%29+FROM+%22platformTemperatureTable%22+WHERE+%28%22device%22+%3D~+%2F%5ENETSCOUT_InfiniStream_CD1%24%2F%29+AND+time+%3E%3D+now%28%29+-+6h+and+time+%3C%3D+now%28%29+-+2m+GROUP+BY+time%281m%29%2C+%22hardware%22+fill%28null%29 user=pmere_writer user-agent=Grafana/5.3.2 write-points=0 write-size=0
Hi,
Is possible to make HTTP flux queries?
Thank you for developing this project. I understand it is still in beta version, but could we keep the changes in sync with read me? So we can know what are new in this repo vs veepee-moc/influxdb-relay
Cluster logs on info mode has something not human readable , and not usefull.
2019-05-28 16:35:12 INF Content Length BODYBUF: 207
2019-05-28 16:35:12 INF Content Length BODYBUF: 494
2019-05-28 16:35:12 INF Content Length BODYBUF: 184
2019-05-28 16:35:12 INF Content Length BODYBUF: 275
2019-05-28 16:35:12 INF Content Length BODYBUF: 245
2019-05-28 16:35:12 INF Content Length BODYBUF: 184
2019-05-28 16:35:12 INF Content Length BODYBUF: 281
2019-05-28 16:35:12 INF Content Length BODYBUF: 206
2019-05-28 16:35:12 INF Content Length BODYBUF: 254
2019-05-28 16:35:12 INF Content Length BODYBUF: 1083
2019-05-28 16:35:12 INF Content Length BODYBUF: 183
http log has the same problem and it is writting a lot of unusefull info.
2019-05-28 16:39:25 INF Route all_writes Match!!!!
2019-05-28 16:39:25 INF Init Processing Endpoint /write
2019-05-28 16:39:25 INF Route all_writes Match!!!!
2019-05-28 16:39:25 INF Init Processing Endpoint /write
2019-05-28 16:39:25 INF Route all_writes Match!!!!
The influxcluster in rwha-sample.influxdb-srelay.cong seems duplicated info with query-router-endpoint-api?
For the SyncFlux, we can run as a HA Cluster, with 1 master 1 slave.
My question is:
[[influxdb]], but the example show we can also define two endpoints in query-router-endpoint-api, which is the cluster endpoint of SyncFlux. My understanding is we can only have one SyncFlux cluster by the existing two db.hamonitor of SyncFlux works. I tested to write data when one node is down (either master or slave), but I don't see data was synced.Would be nice if we can have for each incoming HTTP request these new fields. (
Cluster could be more than one backend so each cluster query could generate more than one http request in this scenario we also would like to have
When data is send with route_db_from_data_rules rule action, the log shows status=0
2019-05-29 11:21:22 INF duration_ms=308.724797 method=POST referer= returnsize=0 source=127.0.0.1:36791 status=0 trace-route="http:example-http-influxdb2> rt:telegraf> decode:ILP> rule:rename_mem_measurement_to_mem_2> rule:to_namespace_metrics> " url=/write?db=telegraf user=admin user-agent=telegraf write-points=1000 write-size=212922
curl -i -XPOST "http://localhost:9096/write?db= test_1" --data-binary 'mymeas,mytag=1 myfield=90 1463683075'
HTTP/1.1 204 No Content
Content-Type: text/plain
Date: Fri, 03 Apr 2020 01:04:34 GMT
No data added to influxdb
Sometimes this out is shown , detected on 0.5 release
[myserverXX].root:~ # journalctl -fu influxdb-srelay
-- Logs begin at Thu 2019-05-30 09:47:46 CEST. --
Jun 14 15:32:21 myserverXX influxdb-srelay[2357]: 2019/06/14 15:32:21 http: multiple response.WriteHeader calls
Jun 14 15:32:21 myserverXX influxdb-srelay[2357]: 2019/06/14 15:32:21 http: multiple response.WriteHeader calls
Jun 14 15:32:21 myserverXX influxdb-srelay[2357]: 2019/06/14 15:32:21 http: multiple response.WriteHeader calls
Jun 14 15:32:21 myserverXX influxdb-srelay[2357]: 2019/06/14 15:32:21 http: multiple response.WriteHeader calls
Jun 14 15:32:21 myserverXX influxdb-srelay[2357]: 2019/06/14 15:32:21 http: multiple response.WriteHeader calls
Jun 14 15:32:21 myserverXX influxdb-srelay[2357]: 2019/06/14 15:32:21 http: multiple response.WriteHeader calls
Jun 14 15:32:48 myserverXX influxdb-srelay[2357]: 2019/06/14 15:32:48 http: multiple response.WriteHeader calls
Jun 14 15:32:48 myserverXX influxdb-srelay[2357]: 2019/06/14 15:32:48 http: multiple response.WriteHeader calls
Jun 14 15:32:48 myserverXX influxdb-srelay[2357]: 2019/06/14 15:32:48 http: multiple response.WriteHeader calls
Jun 14 15:32:48 myserverXX influxdb-srelay[2357]: 2019/06/14 15:32:48 http: multiple response.WriteHeader calls
Hi, i am trying to query second influxdb node while first is down and i got error (error.txt)
I am using one influxdb-srelay with one syncflux for each influxdb node.
config for first syncflux - syncflux1.txt
config for second syncflux - syncflux2.txt
config for srelay - srelay.txt
I would appreciate if you would find the time to assist me with this matter.
func NewHTTP(cfg *config.HTTPConfig) (*HTTP, error) {
h := &HTTP{}
h.cfg = cfg
h.closing = make(chan bool, 1)
//Log output
h.log = utils.GetConsoleLogFormated(cfg.LogFile, cfg.LogLevel)
//AccessLog Output
h.acclog = utils.GetConsoleLogFormated(cfg.AccessLog, "debug")
h.rp = cfg.DefaultRetentionPolicy
Hey,
So this is not an issue per se, just something that might be helpful for other people considering influx 2.0, and the route of the home-made influx clustering – and slowly realising that influx' own unloved influxdb-relay is now finally properly dead, then finding the veepee's fork, going through the issues and realising that it doesn't look like that the fork will ever support v2 api, and after additional googling finding your repos, and getting ready for disappointment only to find that the v2 api support PR was successfully merged a few months ago.
Also, thanks for your hard work. There's a huge gap between single-node influx installations and needing full-blown enterprise package, and this project fills it very well.
Could be interesting log also de X-Forwarded-For header looking for original IP when query has been redirected.
Hello,
I'm trying to setup influxdb-srelay to send metrics to influxdb and victoriametrics storage backends.
Instead of HA, I want to configure 2 single instances and put up some filters to slowly transition to VM, since I had some problems with HA setup.
It turns out, metrics are only sent to the first route.rule defined backend, while the request is sent to the second, but the content-lenght is 0.
My config:
# InfluxDB Backend InfluxDB01
[[influxdb]]
name = "influxdb"
location = "http://127.0.0.1:8086/"
timeout = "15s"
# InfluxDB Backend InfluxDB02
[[influxdb]]
name = "victoriametrics"
location = "http://127.0.0.1:10000/"
timeout = "15s"
# Cluster for linux Metrics
[[influxcluster]]
name = "influxdb_cluster"
members = ["influxdb"]
log-file = "cluster_influxdb.log"
log-level = "debug"
type = "Single"
rate-limit = 1000
burst-limit = 1000
default-ping-response-code = 200
[[influxcluster]]
name = "victoriametrics_cluster"
members = ["victoriametrics"]
log-file = "cluster_victoriametrics.log"
log-level = "debug"
type = "Single"
rate-limit = 1000
burst-limit = 1000
default-ping-response-code = 200
# HTTP Server
[[http]]
name = "influxdb-ingestor"
bind-addr = "0.0.0.0:9096"
log-file = "http_relay_9096.log"
log-level = "debug"
access-log = "access.log"
rate-limit = 1000000
burst-limit = 2000000
#
# IQL /query Endpoint
#
[[http.endpoint]]
uri=["/query"]
type="RD"
source_format="IQL"
#
#
# IQL /write Endpoint
[[http.endpoint]]
uri=["/write"]
source_format="ILP"
type = "WR"
## WRITE request - windows
[[http.endpoint.route]]
name="any_write"
level="http"
[[http.endpoint.route.filter]]
name="pass_all"
key="db"
match=".*"
## Send to influxdb instance
[[http.endpoint.route.rule]]
name="route_all_influxdb"
action="route"
key="db"
match=".*"
to_cluster="influxdb_cluster"
## Send to victoriametrics cluster
[[http.endpoint.route.rule]]
name="route_all_victoriametrics"
action="route"
key="db"
match=".*"
to_cluster="victoriametrics_cluster"
http relay log:
2019-11-22 07:40:51 DBG Processing [0][RD] endpoint [/query]
2019-11-22 07:40:51 DBG Discarding Input Endpoint /write for endpoints [/query]
2019-11-22 07:40:51 DBG Processing [1][WR] endpoint [/write]
2019-11-22 07:40:51 INF Init Processing Endpoint /write
2019-11-22 07:40:51 DBG Processing WRITE route 0 , any_write
2019-11-22 07:40:51 DBG Processing WRITE route 0 , &{cfg:0xc000192980 DecodeFmt:ILP Type:WR log:0xc00016c8a0 filters:[0xc0001d29e0] rules:[0xc0001770c0 0xc000177100] DecodeData:<nil>}
2019-11-22 07:40:51 DBG ----------------------END bodyMiddleWare------------------------
2019-11-22 07:40:51 DBG ----------------------END queryMiddleWare------------------------
2019-11-22 07:40:51 DBG ----------------------END logMiddleWare------------------------
2019-11-22 07:40:51 DBG ----------------------END rateMiddleware-----------------------
In access log, I can see both rules match: 2019-11-22 07:39:51 INF bk_duration_ms=0.480224 duration_ms=6.147961 latency_ms=5.671812 method=POST referer= returnsize=0 source=[2001:1470:8000:82d::173]:48682 status=204 trace-route="http:influxdb-ingestor> rt:any_write> rule:route_all_influxdb> rule:route_all_victoriametrics> " url=/write?consistency=&db=hw&precision=s&rp=autogen user=test user-agent=snmpcollector write-points=-1 write-size=0
In http_route_any_write I can see both rules match:
2019-11-22 07:41:41 INF ROUTE RULE (ACTION: route_all_influxdb): Key db | Match .* | Value | Matched !!!!
2019-11-22 07:41:41 INF ROUTE RULE (ACTION: route_all_victoriametrics): Key db | Match .* | Value | Matched !!!!
Metrics are correctly sent to the first configured server:
2019-11-22 07:42:10 DBG Content Length BODYBUF: 13082
2019-11-22 07:42:11 DBG Content Length BODYBUF: 36458
2019-11-22 07:42:20 DBG Content Length BODYBUF: 1871
But the second server only receives empty content lenght:
2019-11-22 07:42:50 DBG Content Length BODYBUF: 0
2019-11-22 07:42:50 DBG Content Length BODYBUF: 0
2019-11-22 07:42:51 DBG Content Length BODYBUF: 0
2019-11-22 07:42:51 DBG Content Length BODYBUF: 0
srelay version: influxdb-srelay version 0.6.0
I hope I read example configs correctly and to my understanding this should work.
I am trying to DROP SERIES using the default endpoint, it results as my SERIES being dropped only on one of the influxdb instances.
I don't know how to proceed to DROP SERIES as we need to provide a "database" name and all verbs in admin endpoints do not need any database.
Would you have any solutions (except opening some ingress on each instance).
Thanks
We are using your influxdb-srelay application for Influxdb HA. It is a great application.
I have just one small question, maybe someone knows any python and java library like influxdb-python which provide influxdb-srelay support?
2019/05/17 10:26:59 starting relays...
^C
==================
WARNING: DATA RACE
Read at 0x00c0000c8470 by goroutine 8:
github.com/toni-moreno/influxdb-srelay/relay.(*HTTP).Stop()
/home/developer/src/gospace/src/github.com/toni-moreno/influxdb-srelay/relay/http.go:159 +0x5d
github.com/toni-moreno/influxdb-srelay/relayservice.(*Service).Stop()
/home/developer/src/gospace/src/github.com/toni-moreno/influxdb-srelay/relayservice/relayservice.go:78 +0xec
main.runRelay.func1()
/home/developer/src/gospace/src/github.com/toni-moreno/influxdb-srelay/main.go:44 +0x4f
Previous write at 0x00c0000c8470 by goroutine 9:
github.com/toni-moreno/influxdb-srelay/relay.(*HTTP).Run()
/home/developer/src/gospace/src/github.com/toni-moreno/influxdb-srelay/relay/http.go:145 +0x156
github.com/toni-moreno/influxdb-srelay/relayservice.(*Service).Run.func1()
/home/developer/src/gospace/src/github.com/toni-moreno/influxdb-srelay/relayservice/relayservice.go:66 +0x75
Goroutine 8 (running) created at:
main.runRelay()
/home/developer/src/gospace/src/github.com/toni-moreno/influxdb-srelay/main.go:42 +0x209
main.main()
/home/developer/src/gospace/src/github.com/toni-moreno/influxdb-srelay/main.go:95 +0x752
Goroutine 9 (running) created at:
github.com/toni-moreno/influxdb-srelay/relayservice.(*Service).Run()
/home/developer/src/gospace/src/github.com/toni-moreno/influxdb-srelay/relayservice/relayservice.go:63 +0x1e2
main.runRelay()
/home/developer/src/gospace/src/github.com/toni-moreno/influxdb-srelay/main.go:48 +0x266
main.main()
/home/developer/src/gospace/src/github.com/toni-moreno/influxdb-srelay/main.go:95 +0x752
==================
==================
WARNING: DATA RACE
Read at 0x00c00000e438 by goroutine 8:
net.(*TCPListener).Close()
/usr/local/go/src/net/tcpsock_posix.go:136 +0x2c1
github.com/toni-moreno/influxdb-srelay/relay.(*HTTP).Stop()
/home/developer/src/gospace/src/github.com/toni-moreno/influxdb-srelay/relay/http.go:159 +0x74
github.com/toni-moreno/influxdb-srelay/relayservice.(*Service).Stop()
/home/developer/src/gospace/src/github.com/toni-moreno/influxdb-srelay/relayservice/relayservice.go:78 +0xec
main.runRelay.func1()
/home/developer/src/gospace/src/github.com/toni-moreno/influxdb-srelay/main.go:44 +0x4f
Previous write at 0x00c00000e438 by goroutine 9:
net.(*sysListener).listenTCP()
/usr/local/go/src/net/tcpsock_posix.go:163 +0x155
net.(*ListenConfig).Listen()
/usr/local/go/src/net/dial.go:604 +0x70d
net.Listen()
/usr/local/go/src/net/dial.go:674 +0xb2
github.com/toni-moreno/influxdb-srelay/relay.(*HTTP).Run()
/home/developer/src/gospace/src/github.com/toni-moreno/influxdb-srelay/relay/http.go:128 +0xcc
github.com/toni-moreno/influxdb-srelay/relayservice.(*Service).Run.func1()
/home/developer/src/gospace/src/github.com/toni-moreno/influxdb-srelay/relayservice/relayservice.go:66 +0x75
Goroutine 8 (running) created at:
main.runRelay()
/home/developer/src/gospace/src/github.com/toni-moreno/influxdb-srelay/main.go:42 +0x209
main.main()
/home/developer/src/gospace/src/github.com/toni-moreno/influxdb-srelay/main.go:95 +0x752
Goroutine 9 (running) created at:
github.com/toni-moreno/influxdb-srelay/relayservice.(*Service).Run()
/home/developer/src/gospace/src/github.com/toni-moreno/influxdb-srelay/relayservice/relayservice.go:63 +0x1e2
main.runRelay()
/home/developer/src/gospace/src/github.com/toni-moreno/influxdb-srelay/main.go:48 +0x266
main.main()
/home/developer/src/gospace/src/github.com/toni-moreno/influxdb-srelay/main.go:95 +0x752
==================
developer@snmpcoldev01:~/src/gospace/src/github.com/toni-moreno/influxdb-srelay$ Found 2 data race(s)
Traceroute log field should show the same routing info for both modes.
Write mode
2019-08-26 13:01:05 INF bk_duration_ms=0.607902 duration_ms=0.798357 latency_ms=0.1939 method=POST referer= returnsize=0 source=10.121.30.145:2550 status=204 trace-route="http:metrics-be> rt:xxx_metrics_wr> rule:route_db> " url=/write?consistency=&db=xxx_metrics&precision=s&rp=1y user=xxx user-agent=snmpcollector write-points=-1 write-size=0
Here
http:metrics-be> rt:xxx_metrics_wr> rule:route_db>
Read Mode
2019-08-26 13:01:05 INF bk_duration_ms=2.395945 duration_ms=2.704481 latency_ms=0.312109 method=GET referer= returnsize=267 source=10.121.30.145:4508 status=200 trace-route="http:metrics-be> rt:xxx_metrics_rd> rule:route-db> handleQuerySingle:cluster_TEST> Backend:influx-test01> " url=/query?db=xxx_metrics&epoch=s&q="xadsfasdfasdfasdf" user=xxx_writer user-agent=Grafana write-points=0 write-size=0
Here
http:metrics-be> rt:xxx_metrics_rd> rule:route-db> handleQuerySingle:cluster_TEST> Backend:influx-test01>
Will be interesting applying changes in the least impacting way to the service
Hello @toni-moreno
I am trying to send data from prometheus with the following URL for remote read/write
remoteRead: []
# - url: https://www.influxdata.com/blog/influxdb-now-supports-prometheus-remote-read-write-natively/
# - url: "http://influxdb-relay.monitoring.svc.cluster.local:9096/api/v1/prom/read?u=prometheusrw&p=xxxx&db=prometheus_db"
## The remote_write spec configuration for Prometheus.
## ref: https://github.com/prometheus-operator/prometheus-operator/blob/master/Documentation/api.md#remotewritespec
remoteWrite: []
# - url: https://www.influxdata.com/blog/influxdb-now-supports-prometheus-remote-read-write-natively/
# - url: "http://influxdb-relay.monitoring.svc.cluster.local:9096/api/v1/prom/write?u=prometheusrw&p=xxx&db=prometheus_db"
I have added the following "writes" section in srelay config :
#
# IQL /write Endpoint
#
[[http.endpoint]]
uri=["/write"]
source_format="ILP"
type = "WR"
## WRITE request
[[http.endpoint.route]]
name="any_write"
level="http"
# true => will use the endpoint log as this route log
# false => will use its own log file , if not set the name <logdir>/http_route_<route_name>.log
log-inherit = true
#log-file = "query_route_linux_metrics.log"
log-level = "info"
[[http.endpoint.route.filter]]
name="pass_all"
key="db"
match=".*"
## Send to cluster
[[http.endpoint.route.rule]]
name="route_all"
action="route"
key="db"
match=".*"
to_cluster="cluster_influxdb"
#
# Prometheus /api/v1/prom/write ENDPOINT
#
[[http.endpoint]]
uri=["/api/v1/prom/write"]
source_format="prom-write"
type = "WR"
## WRITE request
[[http.endpoint.route]]
name="prom_any_write"
level="http"
# true => will use the endpoint log as this route log
# false => will use its own log file , if not set the name <logdir>/http_route_<route_name>.log
log-inherit = true
#log-file = "query_route_prometheus_metrics.log"
log-level = "info"
[[http.endpoint.route.filter]]
name="pass_all"
key="db"
match=".*"
## Send to cluster
[[http.endpoint.route.rule]]
name="route_all"
action="route"
key="db"
match=".*"
to_cluster="cluster_influxdb"
But I get HTTP 400 error from prometheus logs.
Is it due to the way the authentification is passed to influxdb-srelay ?
Would you have an simple example with basic auth setup ?
Thanks
Hi,
InfluxDB-sRelay: 0.4
Used the new available value for filtering: key="tagvalue"
[[http.endpoint.route.rule]]
name="to_namespace_metrics"
action="route_db_from_data"
key="tagvalue"
key_aux="kubernetes_namespace"
And after reload config, it thrown:
influxdb-srelay[24062]: Error on reload File [%!s(*string=0xc000050e30)]: ERR: Error in Endpoint [/api/v1/prom/write] ...
influxdb-srelay[24062]: Error in parse Rule to_namespace_metrics: Invalid Key tagvalue
```
With srelay we can do /ping/<cluster_id>, but some tools coudl need test with the influxdb client lib witch needs X-Influxdb-Version header.
Hello, thank you for this repo.
I am trying to create a srelay config that help me to route writes based on measurements on a influxdb-v1 input.
I can correcly duplicate to 2 influxdb in HA-Style this input:
net,host=vclient,interface=ens192,server=client err_in=0i,err_out=0i,drop_in=90i,drop_out=0i,bytes_sent=70922687456i,bytes_recv=161218632749i 1704468720000000000
database_acme_com,dbname=MTEMP_DB_DEV1,host=vclient,server=client,subsystem=writes,system=readandwrite value=39 1704468720000000000
with the following config:
[[influxdb]]
name = "server1"
location = "http://server1:8186/"
timeout = "10s"
[[influxdb]]
name = "server2"
location = "http://server2:8086/"
timeout = "10s"
[[influxcluster]]
name = "ha_cluster"
members = ["server1","server2"]
log-file = "/var/log/ha_cluster.log"
log-level = "debug"
type = "HA"
query-router-endpoint-api = ["http://server1:4090/api/queryactive","http://server2:4090/api/queryactive"]
[[http]]
name = "ha-relay"
bind-addr = "0.0.0.0:8085"
log-file = "/var/log/http_harelay_8085.log"
log-level = "debug"
access-log = "/var/log/ha_access_8085.log"
rate-limit = 1000000
burst-limit = 2000000
[[http.endpoint]]
uri=["/write"]
source_format="ILP"
type = "WR"
[[http.endpoint.route]]
name="any_write"
log-file = "/var/log/write_metrics.log"
level="http"
[[http.endpoint.route.filter]]
name="pass_all"
key="db"
match=".*"
[[http.endpoint.route.rule]]
name="route_all"
action="route"
key="db"
match=".*"
to_cluster="ha_cluster"
host-file:
10.224.90.177 server2
10.224.90.188 server1
network trace, from the relay to the remote hosts:
ngrep -t -qd any -W byline \( port 8186 or port 8086 \) and \( host 10.224.90.177 or host 10.224.90.188 \)
interface: any
filter: ( ( port 8186 or port 8086 ) and ( host 10.224.90.177 or host 10.224.90.188 ) ) and (ip || ip6)
T 2024/01/12 13:05:10.249199 10.224.90.177:36986 -> 10.224.90.177:8086 [AP] #11
POST /write?consistency=any&db=dmz HTTP/1.1.
Host: server2:8086.
User-Agent: influxdb-smart-relay.
Content-Length: 294.
Content-Type: text/plain.
Accept-Encoding: gzip.
.
net,host=vclient,interface=ens192,server=client err_in=0i,err_out=0i,drop_in=90i,drop_out=0i,bytes_sent=70922687456i,bytes_recv=161218632749i 1704468720000000000
database_acme_com,dbname=MTEMP_DB_DEV1,host=vclient,server=client,subsystem=writes,system=readandwrite value=39 1704468720000000000
T 2024/01/12 13:05:10.249241 10.224.90.177:33736 -> 10.224.90.188:8186 [AP] #13
POST /write?consistency=any&db=dmz HTTP/1.1.
Host: server1:8186.
User-Agent: influxdb-smart-relay.
Content-Length: 294.
Content-Type: text/plain.
Accept-Encoding: gzip.
.
net,host=vclient,interface=ens192,server=client err_in=0i,err_out=0i,drop_in=90i,drop_out=0i,bytes_sent=70922687456i,bytes_recv=161218632749i 1704468720000000000
database_acme_com,dbname=MTEMP_DB_DEV1,host=vclient,server=client,subsystem=writes,system=readandwrite value=39 1704468720000000000
T 2024/01/12 13:05:10.358572 10.224.90.188:8186 -> 10.224.90.177:33736 [AP] #15
HTTP/1.1 204 No Content.
Content-Type: application/json.
Request-Id: d578ea57-b142-11ee-8c50-801844f28aa8.
X-Influxdb-Build: OSS.
X-Influxdb-Version: 1.8.10.
X-Request-Id: d578ea57-b142-11ee-8c50-801844f28aa8.
Date: Fri, 12 Jan 2024 12:05:10 GMT.
.
T 2024/01/12 13:05:10.360447 10.224.90.177:8086 -> 10.224.90.177:36986 [AP] #17
HTTP/1.1 204 No Content.
Content-Type: application/json.
Request-Id: d578ecb8-b142-11ee-b781-00262d0017a8.
X-Influxdb-Build: OSS.
X-Influxdb-Version: 1.8.10.
X-Request-Id: d578ecb8-b142-11ee-b781-00262d0017a8.
Date: Fri, 12 Jan 2024 12:05:10 GMT.
.
But what I am trying to do is to rotate the input based on measurements. So, for ie, I would like to rotate:
net,host=vclient,interface=ens192,server=client err_in=0i,err_out=0i,drop_in=90i,drop_out=0i,bytes_sent=70922687456i,bytes_recv=161218632749i 1704468720000000000
to server1
and
database_acme_com,dbname=MTEMP_DB_DEV1,host=vclient,server=client,subsystem=writes,system=readandwrite value=39 1704468720000000000
to server2.
I have tried a simple config like that:
[[influxdb]]
name = "server1"
location = "http://server1:8186/"
timeout = "10s"
[[influxdb]]
name = "server2"
location = "http://server2:8086/"
timeout = "10s"
[[influxcluster]]
name = "server1"
members = ["server1"]
log-file = "/var/log/single_server1.log"
log-level = "debug"
type = "Single"
[[influxcluster]]
name = "server2"
members = ["server2"]
log-file = "/var/log/single_server2.log"
log-level = "debug"
type = "Single"
# HTTP Server
[[http]]
name = "ha-relay"
bind-addr = "0.0.0.0:8085"
log-file = "/var/log/http_harelay_8085.log"
log-level = "debug"
access-log = "/var/log/ha_access_8085.log"
rate-limit = 1000000
burst-limit = 2000000
[[http.endpoint]]
uri=["/write"]
source_format="ILP"
type = "WR"
[[http.endpoint.route]]
name="any_write"
log-file = "/var/log/write_metrics.log"
level="data"
[[http.endpoint.route.filter]]
name="pass_all"
key="db"
match=".*"
[[http.endpoint.route.rule]]
name="database"
action="route"
key="measurement"
match="database.*"
to_cluster="server2"
[[http.endpoint.route.rule]]
name="os"
action="route"
key="measurement"
match="(?:cpu.*|disk.*|mem.*|net.*|processes.*|swap.*|system.*)"
to_cluster="server1"
but the route is not happening, and the logs says:
==> /var/log/http_harelay_8085.log <==
2024-01-12 13:14:34 DBG IN REQUEST:&{Method:POST URL:/write?consistency=any&db=dmz Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 Header:map[Accept:[*/*] Content-Type:[text/plain; charset=utf-8] User-Agent:[Telegraf/1.28.5 Go/1.21.4]] Body:0xc0001e8080 GetBody:<nil> ContentLength:-1 TransferEncoding:[chunked] Close:false Host:influxdb-sv02.acme.com:8085 Form:map[] PostForm:map[] MultipartForm:<nil> Trailer:map[] RemoteAddr:10.126.66.121:39440 RequestURI:/write?consistency=any&db=dmz TLS:<nil> Cancel:<nil> Response:<nil> ctx:0xc0001cc210}
2024-01-12 13:14:34 DBG ----------------------INIT rateMiddleware-----------------------
2024-01-12 13:14:34 DBG ----------------------INIT logMiddleWare------------------------
2024-01-12 13:14:34 DBG ----------------------INIT queryMiddleWare------------------------
2024-01-12 13:14:34 DBG ----------------------INIT bodyMiddleWare------------------------
2024-01-12 13:14:34 DBG Processing [0][WR] endpoint [/write]
2024-01-12 13:14:34 INF Init Processing Endpoint /write
2024-01-12 13:14:34 DBG Processing WRITE route 0 , any_write
2024-01-12 13:14:34 DBG Processing WRITE route 0 , &{cfg:0xc00012c980 DecodeFmt:ILP Type:WR log:0xc0001188a0 filters:[0xc0001783e0] rules:[0xc000153080 0xc0001530c0] DecodeData:0x809140}
==> /var/log/write_metrics.log <==
2024-01-12 13:14:34 INF No HTTP responses found on request route any_write
==> /var/log/http_harelay_8085.log <==
2024-01-12 13:14:34 DBG ----------------------END bodyMiddleWare------------------------
2024-01-12 13:14:34 DBG ----------------------END queryMiddleWare------------------------
==> /var/log/ha_access_8085.log <==
2024-01-12 13:14:34 INF bk_duration_ms=9223372036854.775 duration_ms=1.604681 latency_ms=-9223372036854.775 method=POST referer= returnsize=0 source=10.126.66.121:39440 status=0 trace-route="http:ha-relay> rt:any_write> decode:ILP> rule:database> rule:os> " url=/write?consistency=any&db=dmz user=- user-agent="Telegraf/1.28.5 Go/1.21.4" write-points=2 write-size=294
==> /var/log/http_harelay_8085.log <==
2024-01-12 13:14:34 DBG ----------------------END logMiddleWare------------------------
2024-01-12 13:14:34 DBG ----------------------END rateMiddleware-----------------------
If I change the action from "route" to "route_db_from_data" it is almost ok:
[[influxdb]]
name = "server1"
location = "http://server1:8186/"
timeout = "10s"
[[influxdb]]
name = "server2"
location = "http://server2:8086/"
timeout = "10s"
[[influxcluster]]
name = "server1"
members = ["server1"]
log-file = "/var/log/single_server1.log"
log-level = "debug"
type = "Single"
[[influxcluster]]
name = "server2"
members = ["server2"]
log-file = "/var/log/single_server2.log"
log-level = "debug"
type = "Single"
# HTTP Server
[[http]]
name = "ha-relay"
bind-addr = "0.0.0.0:8085"
log-file = "/var/log/http_harelay_8085.log"
log-level = "debug"
access-log = "/var/log/ha_access_8085.log"
rate-limit = 1000000
burst-limit = 2000000
[[http.endpoint]]
uri=["/write"]
source_format="ILP"
type = "WR"
[[http.endpoint.route]]
name="any_write"
log-file = "/var/log/write_metrics.log"
level="data"
[[http.endpoint.route.filter]]
name="pass_all"
key="db"
match=".*"
[[http.endpoint.route.rule]]
name="database"
action="route_db_from_data"
key="measurement"
match="database.*"
value="dmz"
to_cluster="server2"
[[http.endpoint.route.rule]]
name="os"
action="route_db_from_data"
key="measurement"
match="(?:cpu.*|disk.*|mem.*|net.*|processes.*|swap.*|system.*)"
value="dmz"
to_cluster="server1"
the route is happening, network trace:
ngrep -t -qd any -W byline \( port 8186 or port 8086 \) and \( host 10.224.90.177 or host 10.224.90.188 \)
interface: any
filter: ( ( port 8186 or port 8086 ) and ( host 10.224.90.177 or host 10.224.90.188 ) ) and (ip || ip6)
T 2024/01/12 13:18:30.896576 10.224.90.177:36996 -> 10.224.90.177:8086 [AP] #1
POST /write?chunked=&chunksize=&db=dmz&epoch=&p=&pretty=&q=&u= HTTP/1.1.
Host: server2:8086.
User-Agent: influxdb-smart-relay.
Content-Length: 132.
Content-Type: text/plain.
Accept-Encoding: gzip.
.
database_acme_com,dbname=MTEMP_DB_DEV1,host=vclient,server=client,subsystem=writes,system=readandwrite value=39 1704468720000000000
T 2024/01/12 13:18:31.010373 10.224.90.177:8086 -> 10.224.90.177:36996 [AP] #3
HTTP/1.1 204 No Content.
Content-Type: application/json.
Request-Id: b2b20480-b144-11ee-b7c6-00262d0017a8.
X-Influxdb-Build: OSS.
X-Influxdb-Version: 1.8.10.
X-Request-Id: b2b20480-b144-11ee-b7c6-00262d0017a8.
Date: Fri, 12 Jan 2024 12:18:31 GMT.
.
T 2024/01/12 13:18:31.010813 10.224.90.177:33746 -> 10.224.90.188:8186 [AP] #5
POST /write?chunked=&chunksize=&db=dmz&epoch=&p=&pretty=&q=&u= HTTP/1.1.
Host: server1:8186.
User-Agent: influxdb-smart-relay.
Content-Length: 162.
Content-Type: text/plain.
Accept-Encoding: gzip.
.
net,host=vclient,interface=ens192,server=client err_in=0i,err_out=0i,drop_in=90i,drop_out=0i,bytes_sent=70922687456i,bytes_recv=161218632749i 1704468720000000000
T 2024/01/12 13:18:31.122877 10.224.90.188:8186 -> 10.224.90.177:33746 [AP] #7
HTTP/1.1 204 No Content.
Content-Type: application/json.
Request-Id: b2c37ced-b144-11ee-8826-801844f28aa8.
X-Influxdb-Build: OSS.
X-Influxdb-Version: 1.8.10.
X-Request-Id: b2c37ced-b144-11ee-8826-801844f28aa8.
Date: Fri, 12 Jan 2024 12:18:31 GMT.
.
but the answer that the relay give to the client is HTTP 200 and not HTTP 204, that is the what the client expect:
curl -k -v 'http://influxdb-sv02.acme.it:8085/write?consistency=any&db=dmz' -H 'User-Agent: Telegraf/1.28.5 Go/1.21.4' -H 'Content-Type: text/plain; charset=utf-8' -H 'Transfer-Encoding: chunked' --data-binary @/tmp/post2
* Trying 10.224.90.177:8085...
* Connected to influxdb-sv02.acme.it (10.224.90.177) port 8085 (#0)
> POST /write?consistency=any&db=dmz HTTP/1.1
> Host: influxdb-sv02.acme.it:8085
> Accept: */*
> User-Agent: Telegraf/1.28.5 Go/1.21.4
> Content-Type: text/plain; charset=utf-8
> Transfer-Encoding: chunked
>
* Mark bundle as not supporting multiuse
**< HTTP/1.1 200 OK**
< Date: Fri, 12 Jan 2024 12:18:31 GMT
< Content-Length: 0
<
* Connection #0 to host influxdb-sv02.acme.it left intact
That HTTP 204 is correctly answered when I use the first HA-Style srelay config.
Here the logs in debug in that latest case:
==> /var/log/http_harelay_8085.log <==
2024-01-12 13:21:12 DBG IN REQUEST:&{Method:POST URL:/write?consistency=any&db=dmz Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 Header:map[Accept:[*/*] Content-Type:[text/plain; charset=utf-8] User-Agent:[Telegraf/1.28.5 Go/1.21.4]] Body:0xc00023c5c0 GetBody:<nil> ContentLength:-1 TransferEncoding:[chunked] Close:false Host:influxdb-sv02.acme.it:8085 Form:map[] PostForm:map[] MultipartForm:<nil> Trailer:map[] RemoteAddr:10.126.66.121:41886 RequestURI:/write?consistency=any&db=dmz TLS:<nil> Cancel:<nil> Response:<nil> ctx:0xc000231380}
2024-01-12 13:21:12 DBG ----------------------INIT rateMiddleware-----------------------
2024-01-12 13:21:12 DBG ----------------------INIT logMiddleWare------------------------
2024-01-12 13:21:12 DBG ----------------------INIT queryMiddleWare------------------------
2024-01-12 13:21:12 DBG ----------------------INIT bodyMiddleWare------------------------
2024-01-12 13:21:12 DBG Processing [0][WR] endpoint [/write]
2024-01-12 13:21:12 INF Init Processing Endpoint /write
2024-01-12 13:21:12 DBG Processing WRITE route 0 , any_write
2024-01-12 13:21:12 DBG Processing WRITE route 0 , &{cfg:0xc000130b00 DecodeFmt:ILP Type:WR log:0xc0001388a0 filters:[0xc000192480] rules:[0xc00016f100 0xc00016f140] DecodeData:0x809140}
==> /var/log/write_metrics.log <==
2024-01-12 13:21:12 INF processing output for db dmz : # 1 Points
2024-01-12 13:21:12 INF Handle DB route Write to dmz.....
2024-01-12 13:21:12 INF processing output for db dmz : # 1 Points
2024-01-12 13:21:12 INF Handle DB route Write to dmz.....
Please, can you help me to build the right config ?
I am using the 0.6.0 version:
/root/go/bin/influxdb-srelay -version
influxdb-srelay version 0.6.0
Thanks
Aldo
Hello, I'm testing with RPM installed.
but, access logs are piling up continuously.
Line 76 in 0dab707
I wonder if there is any way to adjust the log_level or stack the access log gently.
Hi,
we (me and @Zottel92) set up influxdb-srelay in a HA setup like in the example here: rwha-sample.influxdb-srelay.conf
There are only small changes in the configuration of the backends:
# InfluxDB Backend InfluxDB01
[[influxdb]]
name = "influx1"
location = "http://influx1:8086/"
timeout = "10s"
buffer-size-mb = 1024
max-batch-kb = 512
max-delay-interval = "10s"
# InfluxDB Backend InfluxDB02
[[influxdb]]
name = "influx2"
location = "http://influx2:8086/"
timeout = "10s"
buffer-size-mb = 1024
max-batch-kb = 512
max-delay-interval = "10s"
If both backends are healthy it works as expected.
The data always looks like this while x = 0..5:
mymeas,mytag=1 myfield=5 16346609x1
mymeas,mytag=1 myfield=5 16346609x2
mymeas,mytag=2 myfield=5 16346609x3
mymeas,mytag=2 myfield=5 16346609x4
influx1:Oct 19 19:58:37 influx1 influxd-systemd-start.sh[93646]: ts=2021-10-19T17:58:37.538938Z lvl=info msg="Write body received by handler" log_id=0XIP0jkW000 service=httpd body="mymeas,mytag=1 myfield=5 1634660901\nmymeas,mytag=1 myfield=5 1634660902\nmymeas,mytag=2 myfield=5 1634660903\nmymeas,mytag=2 myfield=5 1634660904"
influx1.influx2 but not on influx1.influx1 can recover.influx1:Oct 19 19:59:51 influx1 influxd-systemd-start.sh[93646]: ts=2021-10-19T17:59:51.545892Z lvl=info msg="Write body received by handler" log_id=0XIP0jkW000 service=httpd body="mymeas,mytag=1 myfield=5 1634660911\nmymeas,mytag=1 myfield=5 1634660912\nmymeas,mytag=2 myfield=5 1634660913\nmymeas,mytag=2 myfield=5 1634660914"
Oct 19 19:59:51 influx1 influxd-systemd-start.sh[93646]: ts=2021-10-19T17:59:51.549231Z lvl=info msg="Write body received by handler" log_id=0XIP0jkW000 service=httpd body="mymeas,mytag=1 myfield=5 1634660941\nmymeas,mytag=1 myfield=5 1634660942\nmymeas,mytag=2 myfield=5 1634660943\nmymeas,mytag=2 myfield=5 1634660944mymeas,mytag=1 myfield=5 1634660941\nmymeas,mytag=1 myfield=5 1634660942\nmymeas,mytag=2 myfield=5 1634660943\nmymeas,mytag=2 myfield=5 1634660944mymeas,mytag=1 myfield=5 1634660941\nmymeas,mytag=1 myfield=5 1634660942\nmymeas,mytag=2 myfield=5 1634660943\nmymeas,mytag=2 myfield=5 1634660944"
Oct 19 19:59:57 influx1 influxd-systemd-start.sh[93646]: ts=2021-10-19T17:59:57.017071Z lvl=info msg="Write body received by handler" log_id=0XIP0jkW000 service=httpd body="mymeas,mytag=1 myfield=5 1634660911\nmymeas,mytag=1 myfield=5 1634660912\nmymeas,mytag=2 myfield=5 1634660913\nmymeas,mytag=2 myfield=5 1634660914"
Oct 19 20:00:07 influx1 influxd-systemd-start.sh[93646]: ts=2021-10-19T18:00:07.256977Z lvl=info msg="Write body received by handler" log_id=0XIP0jkW000 service=httpd body="mymeas,mytag=1 myfield=5 1634660911\nmymeas,mytag=1 myfield=5 1634660912\nmymeas,mytag=2 myfield=5 1634660913\nmymeas,mytag=2 myfield=5 1634660914"
or in short:
mymeas,mytag=1 myfield=5 1634660911\nmymeas,mytag=1 myfield=5 1634660912\nmymeas,mytag=2 myfield=5 1634660913\nmymeas,mytag=2 myfield=5 1634660914"
mymeas,mytag=1 myfield=5 1634660941\nmymeas,mytag=1 myfield=5 1634660942\nmymeas,mytag=2 myfield=5 1634660943\nmymeas,mytag=2 myfield=5 1634660944mymeas,mytag=1 myfield=5 1634660941\nmymeas,mytag=1 myfield=5 1634660942\nmymeas,mytag=2 myfield=5 1634660943\nmymeas,mytag=2 myfield=5 1634660944mymeas,mytag=1 myfield=5 1634660941\nmymeas,mytag=1 myfield=5 1634660942\nmymeas,mytag=2 myfield=5 1634660943\nmymeas,mytag=2 myfield=5 1634660944"
mymeas,mytag=1 myfield=5 1634660911\nmymeas,mytag=1 myfield=5 1634660912\nmymeas,mytag=2 myfield=5 1634660913\nmymeas,mytag=2 myfield=5 1634660914"
mymeas,mytag=1 myfield=5 1634660911\nmymeas,mytag=1 myfield=5 1634660912\nmymeas,mytag=2 myfield=5 1634660913\nmymeas,mytag=2 myfield=5 1634660914"
Oct 19 20:01:04 influx1 influxd-systemd-start.sh[93646]: ts=2021-10-19T18:01:04.849600Z lvl=info msg="Write body received by handler" log_id=0XIP0jkW000 service=httpd body="mymeas,mytag=1 myfield=5 1634660951\nmymeas,mytag=1 myfield=5 1634660952\nmymeas,mytag=2 myfield=5 1634660953\nmymeas,mytag=2 myfield=5 1634660954"
As you can see in step 8 srelaywrites the data for x=1, then for x=4 four times in a row and not split by \n. Then it writes the data for x=1 again twice. The data for x=2 and x=3 is missing completely.
This is a serious issue for us and we hope this can be fixed soon.
Do you need further information, then please ask.
Hi,
InfluxDB-sRelay: 0.4
After checking the logs, the following entries seems to have bad records on bk_duration_ms and latency_ms:
Example:
... INF bk_duration_ms=9223372036854.775 duration_ms=0.519287 latency_ms=-9223372036854.775 method=GET referer= returnsize=4 source=foo status=200 trace-route="bar" url=/health user=- user-agent=XX/1.0 write-points=0 write-size=0
Hi,
While trying to redirect /query to active endpoints, I see that it is always redirected to the first specified backend.
On this line
influxdb-srelay/cluster/influxcluster.go
Line 322 in 0dab707
queryRouterEndpointAPI is an empty list, and does not seem to be populated anywhere.
Could you please go through the code and explain how this variable works?
Could be nice if we can generate self resource use with the runtime package and send it to any configured cluster
Also consolidated access.log events would be nice to sent to any configured Influxcluster
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.