glenn-m / promr Goto Github PK
View Code? Open in Web Editor NEWR library for reading/analyzing metrics stored on a Prometheus server
License: MIT License
R library for reading/analyzing metrics stored on a Prometheus server
License: MIT License
devtools::check()
returns a warning about missing documentation
❯ checking for missing documentation entries ... WARNING
Undocumented S4 classes:
‘Prometheus’
All user-level objects in a package (including S4 classes and methods)
should have documentation entries.
See chapter ‘Writing R documentation files’ in the ‘Writing R
Extensions’ manual.
Create a vignette with sample use cases for the available queries.
httptest
offers a dedicated environment for testing functions that wrap APIs. I reckon we should consider this instead of faffing around trying to mock/wrap httr::GET
. It seems that a substantial effort was put in that package to facilitate the development of testing and vignette generation. We would, in effect, swap one dependency for another mockery
-> httptest
. It just feels cleaner to me as it was developed for that purpose and looks like a more mature development.
httptest makes it easy to write tests for code and packages that wrap web APIs. Testing code that communicates with remote servers can otherwise be painful: things like authentication, server state, and network flakiness can make testing seem too costly to bother with. The httptest package enables you to test all of the logic on the R sides of the API in your package without requiring access to the remote service.
FYI @glenn-m
Currently, it seems the value column is being cast as a character
type, meaning users have to cast to complex
or numeric
.
library(promR)
library(ggplot2)
prom <- Prometheus$new(host = "http://demo.robustperception.io", port = 9090)
metrics_instant <- prom$query(query = 'go_goroutines', time = as.POSIXct(Sys.time() - 60))
m <- ggplot(metrics_instant, aes(metrics_instant$job)) +
geom_bar(aes(weight=as.complex(metrics_instant$value))) +
xlab("Prometheus Jobs") +
ylab("Value")
plot(m)
Ideally this should be handled for the user as the value column will always be a numerical value.
However, some Prometheus metrics will return scientific notation when the value is large enough.
For example:
alertmanager_silences_query_duration_seconds_bucket{le="2.5"} 4.766468e+06
Is there any reason we shouldn't just cast everything to the complex
type? Or would it be better to default to numeric
unless complex
is required? What's your opinion @konradzdeb?
On that
Lines 32 to 39 in d5c1de0
timout
in seconds? If so maybe it's good to add checkmate::assertInt
to avoid people passing -10
as is.null
won't protect from not sure. Not sure how defensive1 you want that script to be and whether it's worth the effort...
1checkmate: Fast Argument Checks for Defensive R Programming
With respect to the object that is returned here:
Lines 134 to 142 in aad5701
Is the generated data frame of similar format that is handled by the function that formats metrics for ranges and instant query? If so a potential solution could be to devise one generic function that formats all data frame outputs returned by Prometheus, irrespectively of the query. If you haven’t started on that, I’m happy to progress this (assuming that this is the right approach, of course)
Lines 28 to 65 in aad5701
Consider exporter that exposes following metrics:
tcp_queue{state="LISTEN", dir="rx", addr="0.0.0.0", port="6432"} 0
tcp_queue{state="LISTEN", dir="rx", addr="127.0.0.1", port="5432"} 0
This is what prom$rangeQuery(query = 'tcp_queue', ...)$port
returns instead of expected values:
[1] 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345
[13] 12345 12345
where 12345
actually comes from instance="127.0.0.1:12345"
It could be useful for objects returned by promR to extend the existing as.xts
and as.ts
methods.
library(promR)
prom <- Prometheus$new(host = "http://demo.robustperception.io", port = 9090)
# Timestamps can be in rfc3339 or unix format
metrics_instant <- prom$query(query = "go_goroutines",
time = as.POSIXct(Sys.time() - 60))
metrics_instant_xts <- xts::as.xts(metrics_instant)
class(metrics_instant_xts)
# [1] "xts" "zoo"
xts::as.xts(metrics_instant)
Error in
as.POSIXlt.character(x, tz, ...)
: character string is not in a standard unambiguous format
May be worth reflecting whether there is merit in:
Providing zoo::as.zoo
. Properly implemented as.xts
method should return an object usable by zoo compatible methods. From ?as.xts
documentation:
(...) The returned object will preserve all relevant attribute/slot data within itself, allowing for temporary conversion to use zoo and xts compatible methods. (...)
as.ts
method is likely, not useful in the current implementation, where metrics objects return data for various categories, as visible in table(metrics_instant$job)
.
The when running sample code:
Lines 32 to 34 in 13faf82
metrics_current <- prom$query(query = "go_goroutines",
time = format(Sys.time(), "%Y-%m-%dT%H:%M:%SZ"))
# Error in format_metrics_current_data(metrics) :
# Assertion on 'x' failed: Must have at least 1 rows, but has 0 rows.
metrics_current <- prom$query(query = "go_goroutines",
time = format(Sys.time() - 10, "%Y-%m-%dT%H:%M:%SZ"))
# Error in format_metrics_current_data(metrics) :
# Assertion on 'x' failed: Must have at least 1 rows, but has 0 rows.
for:
Sys.time()
# [1] "2019-04-01 07:22:00 BST"
the data would be produced for Sys.time() - 3600
metrics_current <- prom$query(query = "go_goroutines",
time = format(Sys.time() - 3600, "%Y-%m-%dT%H:%M:%SZ"))
It's unclear whether:
07:22
but there is for an hour earlier (?) Sys.time() - 3600
If possible to solve through #11 if not consider running curl::has_internet()
before query, return stop
if (!curl::has_internet()) stop("This machine is not connected to the Internet :)")
Shall return something like (?):
The query completed but returned no data....
Potentially related to #11
Currently we have few tests around the formatting of the metrics output from Prometheus, this should be improved.
We should also avoid having a reliance on external systems for the tests. Perhaps we can create a data set for use in testing?
Seems like formatting function wasn’t applied to this output as it still has X__name__ in column name.
Line 102 in a111df4
The current error messages raised when we receive a non-200 status code are very basic.
We should be parsing JSON object returned by Prometheus to enrich our error messages.
Currently we have differing column names depending on query, we should really make it more consistent. Mainly the name
column name.
e.g.
An instant query results in:
X__name__ | instance | job | timestamp | value | port |
---|---|---|---|---|---|
go_goroutines | demo.robustperception.io | prometheus | 1554235843.452 | 87 | 9090 |
go_goroutines | demo.robustperception.io | pushgateway | 1554235843.452 | 40 | 9091 |
go_goroutines | demo.robustperception.io | alertmanager | 1554235843.452 | 34 | 9093 |
go_goroutines | demo.robustperception.io | node | 1554235843.452 | 8 | 9100 |
where as range query results in:
name | instance | job | port | timestamp | value |
---|---|---|---|---|---|
go_goroutines | demo.robustperception.io | prometheus | 9090 | 1554235303.644 | 85 |
go_goroutines | demo.robustperception.io | prometheus | 9090 | 1554235313.644 | 85 |
go_goroutines | demo.robustperception.io | prometheus | 9090 | 1554235323.644 | 85 |
go_goroutines | demo.robustperception.io | prometheus | 9090 | 1554235333.644 | 85 |
go_goroutines | demo.robustperception.io | prometheus | 9090 | 1554235343.644 | 85 |
go_goroutines | demo.robustperception.io | prometheus | 9090 | 1554235353.644 | 85 |
tl;dr X__name__
should be name
instead
Error in metrics <- format_metrics_range_data(metrics) (main.R#85): Assertion on 'x' failed: Must have at least 1 rows, but has 0 rows.
It would be nice though, if it would just work (and return an empty result?).
Hi All,
Firstly thanks for taking the time to support this project.
Would there be an appetite to pass basic auth creds? For instance if Prometheus is hosted in Openshift/OKD it will sits behind an auth proxy.
The thinking would be if someone were to pass a username/password then try and authenticate, if these aren't present then don't use them.
Any thoughts?
Thanks
Matt
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.