performance issues with 500k of influxdb data. about grafana-statusmap HOT 7 OPEN

from grafana-statusmap.

Thank you for bug report. In order to reproduce the problem, please, send versions of grafana and statusmap and also statusmap panel options as exported panel json or as screenshots.

from grafana-statusmap.

Perhaps the most helpful tip is this:
The sample influx response I posted above has timestamps from 12:01am to 4am-ish on May 9, 2019.
The problem still happens if you set a Grafana timeframe OUTSIDE of those 4 hours.

You'll know you've reproduced this when the UI becomes unresponsive....barely alive.
Occasionally, it will respond a little, but if you switch to another app and then back, it will go dead again.

StatusMap version: v0.1.1
Grafana version: 5.4.3
statusMapPanelJson.zip

from grafana-statusmap.

brand spanking new to this code, but all this time in addHeatmap() in rendering.js looks suspicious.
What do ya think?

from grafana-statusmap.

Got some time to investigate this issue. It looks like some kind of overhead from d3. One way to eleminate a bottleneck is to precalculate x, y, width, height, and especially a color before render — this can help to avoid multiple scaling operations. So.. Sorry, no quick fix here :(
Refactor of render algorithms is planned in #53

from grafana-statusmap.

Thanks for having a look.
I can't help but think all of the nulls in my influx response are part of what's choking up d3.
The influx response is .5 mb of json data -- and if you zip it up, it's just 8k. That's because 99% of everything in the json looks like this block of nulls below.

Any suggestions (you've had a few good influx sql suggestions for me already!!) on tweaking my influx to eliminate these nulls? Would that help anything?

--Erik

null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null]

from grafana-statusmap.

Hey,

I'm struggling a bit with the performance as well. My approach was to divide the received data into multiple panels to benefit from Grafana's lazy loading.

However, when every panel is loaded, my dashboard takes up about 3 gb RAM. My idea was to implement virtual scrolling under the use of Intersection Observers, i.e. the heatmap is only rendered when it is inside of the viewport.

Could this be a feature for this plugin? Maybe some "performance-mode"-option, which deactivates hovering effects and activates virtual scrolling. I could provide an implementation for it!

from grafana-statusmap.