Giter VIP home page Giter VIP logo

tsl's Introduction

Time Series Language (TSL)

TSL is a HTTP proxy which generate WarpScript or a PromQl script based on a TSL query, then execute it on a Warp 10 or a Prometheus backend. To get more information about a TSL query, you can have a look at our initial spec and our query doc. Our ANTLR grammar definition is available in the antlr folder here.

TSL: a Time Series Language

Pre-install

To build and install Tsl you need:

  • A working go install, with the GOROOTand GOPATH variables correctly set
  • make in order to use the Makefile

Install

  1. Install the tooling

    make init

  2. Grab the dependencies

    make dep

  3. Local build of TSL

    make dev

  4. Release build of TSL

    make release

Binary is now available in the directory build under the name tsl.

Configure

TSL needs a YAML configuration file with two entries:

  • tsl.default.endpoint
  • tsl.default.type

This is an example with a Warp 10 backend.

tsl:
  default:
    endpoint: http//example.com
    type: 'warp10'

TSL will look for a config.yml file on:

  • /etc/tsl/
  • $HOME/.tsl
  • the current path

You can also use the -c to set a configuration path for a TSL instance.

Without a configuaration file, TSL will use http://127.0.0.1:9090 as default endpoint.

To use TSL on several backends you can specify the following optional parameters:

tsl:
  warp10:
    endpoints:
      - http://127.0.0.1:8080
      - http://127.0.0.1:8081

  promQL:
    endpoints:
      - http://127.0.0.1:9090
      - http://127.0.0.1:9091

Run TSL

You can simply run the TSL binary, ./build/tsl.

By default, TSL listens on 127.0.0.1:8080.

$ ./build/tsl --config path/to/config.yml
INFO[0000] Start tsl server on 127.0.0.1:8080

Query TSL

To send a TSL request, you can use the v0/query api endpoint to send TSL queries.

A TSL query can be:

select("sys.cpu.nice").where("host=web01").from(1346846400000,to=1346847000005)

This is a first TSL query, with the following methods used:

  • select to specify a metric name to retrieve (or pattern name).
  • where to set labels that the current Time-series must have.
  • from to select data between two dates.

You can send this request through an HTTP Post with the cURL Command line or any other HTTP tools like Insomnia or PostMan.

As example, we can write a new TSL file containing the previous script and send the cURL command below:

curl -v --data-binary @select.tsl 'https://user:[email protected]:8080/v0/query'

This command provides as result a time-series JSON list.

TSL implements a lot of diffetent methods and you can find a more details in the spec folder.

Usage

If you need more complex options, use ./build/tsl --help:

$ ./build/tsl --help
A proxy that translates queries for a TSDB backend

Usage:
  tsl [flags]
  tsl [command]

Available Commands:
  help        Help about any command
  version     Print the version number

Flags:
  -c, --config string   config file (default is $HOME/.tsl.yaml)
  -h, --help            help for tsl
  -l, --listen string   listen address (default "127.0.0.1:8080")
  -v, --verbose         verbose output

Use "tsl [command] --help" for more information about a command.

Use TSL with WebAssembly

NOTE: A Go 1.11 (> go1.11.1) version at least is needed. Building tsl.wasm works with go 1.12.5.

You can run the TSL to WarpScript interpreter in a WebAssembly program. The tsl.wasm file export the method tslToWarpScript callable from JavaScript running in a browser or NodeJS.

To compile the tsl package for the web:

$ make wasm

As example, you can call the tslToWarpScript from a JavaScript program like that:

tslToWarpScript("select(\"sys.cpu.nice\").where(\"host=web01\").from(1346846400000,to=1346847000005)", "", false, (err, res) => {
	console.log(res) // Output the tsl query in a WarpScript syntax
})

To integrate the tsl.wasm file in your program, you should follow the Golang WebAssembly wiki.

Build TSL .so file to use TSL in other language

You can build TSL so library files, usable in other progamming language as describe in this github tutorial.

To compile TSL .so libray simply run

make so

In the spec folder, you can find a JAVA example of the .so TSL library.

## License

TSL is released under a 3-BSD clause license.

Get in touch

Gitter: gitter.im/ovh/metrics-TSL

Twitter: @AurrelH95

tsl's People

Contributors

aurrelhebert avatar florentindubois avatar gillesbiannic avatar miton18 avatar notbad4u avatar pierrez avatar stevenleroux avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

tsl's Issues

Create a npm package for TSL with wasm

Following #43, that could be nice to provide an npm package to integrate this project easily in a frontend project.

According to creating-node-js-modules from npm documentation, creating a package seem to be simple.
The requirements are:

  • Create a package.json file with a main entry
  • Propose a js file that will load the wasm ... but maybe isn't a good idea and we should let people this "custom" task and just provide an empty main.js
  • A TravisCI or whatever to recreate the package after each new version published.

TSL implementation on python pandas

This is more a question rather than an issue.

I have seen in the roadmap that an implementation independent from the back-end is foreseen. Has an implementation on top of python pandas been considered? Would it make senses according to you?

Thanks in anticipation for your response.

Ability to create a new series withe defined values

When I draw a graph with the result of a TSL query, I often need to add a custom series which can be for example an upper limit.

Is it possible to do it in TSL to perform the two operations at the same time

Exemple:

create(series('metrics.service.mads').setLabels('service=abcd').addValue(now, 150).addValue(now - 1h, 150)...) 

ANTLR grammar does not reflect the current state of TSL

It seems the ANTLR grammar file is not used by the project which instead uses a handcrafted parser, as a consequence, the grammar file does not seem to be updated whenever the features of the language evolve thus preventing implementations which rely on the ANTLR4 grammar to be up to date.

As an example, functions such as fill and fillValue are not present in the ANTLR files.

Are there any plans to actually use the grammar file for the go implementation so it would be kept up to date?

Rename a series with a custom template

TSL already have .renameBy() function which allow us to rename a series with its labels.

For some use case, it could be interesting have template the name with a very custom string.

Can we extend .renameBy() function with the following behaviour ?

select('os.cpu').renameBy('${this.labels.hostname} (${this.labels.datacenter})' )
// produce a series named: "1.server.front (GRA)"

Sort series by label or name

Hello,

when I get mutiple series, I cannot predict how the list of series will be sorted. I would like to be able to sort the list of series based on the name or a label of the series.
This way I am sure series will always be displayed in the same order in Grafana.

Thank you,
Tristan

[Bug] "to" keyword parsing seems to be broken

curl -d "select('requests_latency_seconds_count').where('role=0').from('2019-01-31T00:00:00+01:00',to='2019-02-01T00:00:00+01:00').finite().sampleBy(1h,last).group(sum).window(delta, 0, 1)" https://DESC:[email protected]/v0/query

{"error":"Cannot parsed query: Error when parsing "to" in "from" function, this variable isn't declared at line 1, char 91"}

quick fix resolution (removing the to= instruction) see below:
curl -d "select('requests_latency_seconds_count').where('role=0').from('2019-01-31T00:00:00+01:00', '2019-02-01T00:00:00+01:00').finite().sampleBy(1h,last).group(sum).window(delta, 0, 1)" https://DESC:[email protected]/v0/query

Implement setLabelFromName & extractLabelFromName() functions

I'd like to be able to set a new label from series names so I can use it in the query (for example with groupBy)

Example :

select(my.awesome.series.name.*.count)
  .where(...)
  .setLabelFromName("verb")
  .groupBy(verb, mean)

select(my.awesome.series.name.*.count)
  .where(...)
  .extractLabelFromName("verb","~MyRegex")
  .groupBy(verb, mean)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.