Giter VIP home page Giter VIP logo

redis-time-series's Introduction

RSpec Gem Version Documentation Maintainability Test Coverage

RedisTimeSeries

A Ruby adapter for the RedisTimeSeries module.

This doesn't work with vanilla Redis, you need the time series module compiled and installed. Try it with Docker, and see the module setup guide for additional options.

docker run -p 6379:6379 -it --rm redislabs/redistimeseries

TL;DR

require 'redis-time-series'
ts = Redis::TimeSeries.new('foo')
ts.add 1234
=> #<Redis::TimeSeries::Sample:0x00007f8c0d2561d8 @time=2020-06-25 23:23:04 -0700, @value=0.1234e4>
ts.add 56
=> #<Redis::TimeSeries::Sample:0x00007f8c0d26c460 @time=2020-06-25 23:23:16 -0700, @value=0.56e2>
ts.add 78
=> #<Redis::TimeSeries::Sample:0x00007f8c0d276618 @time=2020-06-25 23:23:20 -0700, @value=0.78e2>
ts.range (Time.now.to_i - 100)..Time.now.to_i * 1000
=> [#<Redis::TimeSeries::Sample:0x00007f8c0d297200 @time=2020-06-25 23:23:04 -0700, @value=0.1234e4>,
 #<Redis::TimeSeries::Sample:0x00007f8c0d297048 @time=2020-06-25 23:23:16 -0700, @value=0.56e2>,
 #<Redis::TimeSeries::Sample:0x00007f8c0d296e90 @time=2020-06-25 23:23:20 -0700, @value=0.78e2>]

Installation

Add this line to your application's Gemfile:

gem 'redis-time-series'

And then execute:

$ bundle

Or install it yourself as:

$ gem install redis-time-series

Usage

Check out the Redis Time Series command documentation first. Should be able to do most of that.

Configuring

You can set the default Redis client for class-level calls operating on multiple series, as well as series created without specifying a client.

Redis::TimeSeries.redis = Redis.new(url: ENV['REDIS_URL'], timeout: 1)

Creating a Series

Create a series (issues TS.CREATE command) and return a Redis::TimeSeries object for further use. Key param is required, all other arguments are optional.

ts = Redis::TimeSeries.create(
  'your_ts_key',
  labels: { foo: 'bar' },
  retention: 600,
  uncompressed: false,
  redis: Redis.new(url: ENV['REDIS_URL']) # defaults to Redis::TimeSeries.redis
)

You can also call .new instead of .create to skip the TS.CREATE command.

ts = Redis::TimeSeries.new('your_ts_key')

Adding Data to a Series

Add a single value

ts.add 1234
=> #<Redis::TimeSeries::Sample:0x00007f8c0ea7edc8 @time=2020-06-25 23:41:29 -0700, @value=0.1234e4>

Add a single value with a timestamp

ts.add 1234, 3.minutes.ago # Used ActiveSupport here, but any Time object works fine
=> #<Redis::TimeSeries::Sample:0x00007fa6ce05f3f8 @time=2020-06-25 23:39:54 -0700, @value=0.1234e4>

# Optionally store data uncompressed
ts.add 5678, uncompressed: true
=> #<Redis::TimeSeries::Sample:0x00007f93f43cdf68 @time=2020-07-18 23:15:29 -0700, @value=0.5678e4>

Add multiple values with timestamps

ts.madd(2.minutes.ago => 12, 1.minute.ago => 34, Time.now => 56)
=> [1593153909466, 1593153969466, 1593154029466]

Increment or decrement the most recent value

ts.incrby 2
=> 1593154222877
ts.decrby 1
=> 1593154251392
ts.increment # alias of incrby
=> 1593154255069
ts.decrement # alias of decrby
=> 1593154257344

# Optionally store data uncompressed
ts.incrby 4, uncompressed: true
=> 1595139299769
ts.get
=> #<Redis::TimeSeries::Sample:0x00007fa25f17ed88 @time=2020-06-25 23:50:57 -0700, @value=0.57e2>
ts.increment
=> 1593154290736
ts.get
=> #<Redis::TimeSeries::Sample:0x00007fa25f199480 @time=2020-06-25 23:51:30 -0700, @value=0.58e2>

Add values to multiple series

# Without timestamp (series named "foo" and "bar")
Redis::TimeSeries.madd(foo: 1234, bar: 5678)
=> [#<Redis::TimeSeries::Sample:0x00007ffb3aa32ae0 @time=2020-06-26 00:09:15 -0700, @value=0.1234e4>,
 #<Redis::TimeSeries::Sample:0x00007ffb3aa326d0 @time=2020-06-26 00:09:15 -0700, @value=0.5678e4>]
# With a timestamp
Redis::TimeSeries.madd(foo: { 1.minute.ago => 1234 }, bar: { 1.minute.ago => 2345 })
=> [#<Redis::TimeSeries::Sample:0x00007fb102431f88 @time=2020-06-26 00:10:22 -0700, @value=0.1234e4>,
 #<Redis::TimeSeries::Sample:0x00007fb102431d80 @time=2020-06-26 00:10:22 -0700, @value=0.2345e4>]

Querying a Series

Get the most recent value

ts.get
=> #<Redis::TimeSeries::Sample:0x00007fa25f1b78b8 @time=2020-06-25 23:51:30 -0700, @value=0.58e2>

Get a range of values

# Time range as an argument
ts.range(10.minutes.ago..Time.current)
=> [#<Redis::TimeSeries::Sample:0x00007fa25f13fc28 @time=2020-06-25 23:50:51 -0700, @value=0.57e2>,
    #<Redis::TimeSeries::Sample:0x00007fa25f13db58 @time=2020-06-25 23:50:55 -0700, @value=0.58e2>,
    #<Redis::TimeSeries::Sample:0x00007fa25f13d900 @time=2020-06-25 23:50:57 -0700, @value=0.57e2>,
    #<Redis::TimeSeries::Sample:0x00007fa25f13d680 @time=2020-06-25 23:51:30 -0700, @value=0.58e2>]

# Time range as keyword args
ts.range(from: 10.minutes.ago, to: Time.current)
=> [#<Redis::TimeSeries::Sample:0x00007fa25dc01f00 @time=2020-06-25 23:50:51 -0700, @value=0.57e2>,
    #<Redis::TimeSeries::Sample:0x00007fa25dc01d20 @time=2020-06-25 23:50:55 -0700, @value=0.58e2>,
    #<Redis::TimeSeries::Sample:0x00007fa25dc01b68 @time=2020-06-25 23:50:57 -0700, @value=0.57e2>,
    #<Redis::TimeSeries::Sample:0x00007fa25dc019b0 @time=2020-06-25 23:51:30 -0700, @value=0.58e2>]

# Limit number of results with count argument
ts.range(10.minutes.ago..Time.current, count: 2)
=> [#<Redis::TimeSeries::Sample:0x00007fa25dc01f00 @time=2020-06-25 23:50:51 -0700, @value=0.57e2>,
    #<Redis::TimeSeries::Sample:0x00007fa25dc01d20 @time=2020-06-25 23:50:55 -0700, @value=0.58e2>]

# Apply aggregations to the range
ts.range(from: 10.minutes.ago, to: Time.current, aggregation: [:avg, 10.minutes])
=> [#<Redis::TimeSeries::Sample:0x00007fa25dc01f00 @time=2020-06-25 23:50:00 -0700, @value=0.575e2>]

Get info about the series

ts.info
=> #<struct Redis::TimeSeries::Info
 series=
  #<Redis::TimeSeries:0x00007ff46da9b578 @key="ts3", @redis=#<Redis client v4.2.1 for redis://127.0.0.1:6379/0>>,
 total_samples=3,
 memory_usage=4264,
 first_timestamp=1595187993605,
 last_timestamp=1595187993629,
 retention_time=0,
 chunk_count=1,
 max_samples_per_chunk=256,
 labels={"foo"=>"bar"},
 source_key=nil,
 rules=
  [#<Redis::TimeSeries::Rule:0x00007ff46db30c68
    @aggregation=#<Redis::TimeSeries::Aggregation:0x00007ff46db30c18 @duration=3600000, @type="avg">,
    @destination_key="ts1",
    @source=
     #<Redis::TimeSeries:0x00007ff46da9b578
      @key="ts3",
      @redis=#<Redis client v4.2.1 for redis://127.0.0.1:6379/0>>>]>

# Each info property is also a method on the time series object
ts.memory_usage
=> 4208
ts.labels
=> {"foo"=>"bar"}
ts.total_samples
=> 3

# Total samples also available as #count, #length, and #size
ts.count
=> 3
ts.length
=> 3
ts.size
=> 3

Find series matching specific label(s)

Redis::TimeSeries.query_index('foo=bar')
=> [#<Redis::TimeSeries:0x00007fc115ba1610
  @key="ts3",
  @redis=#<Redis client v4.2.1 for redis://127.0.0.1:6379/0>,
  @retention=nil,
  @uncompressed=false>]
# Note that you need at least one "label equals value" filter
Redis::TimeSeries.query_index('foo!=bar')
=> RuntimeError: Filtering requires at least one equality comparison
# query_index is also aliased as .where for fluency
Redis::TimeSeries.where('foo=bar')
=> [#<Redis::TimeSeries:0x00007fb8981010c8
  @key="ts3",
  @redis=#<Redis client v4.2.1 for redis://127.0.0.1:6379/0>,
  @retention=nil,
  @uncompressed=false>]

Querying Multiple Series

Get all samples from matching series over a time range with mrange

[4] pry(main)> result = Redis::TimeSeries.mrange(1.minute.ago.., filter: { foo: 'bar' })
=> [#<struct Redis::TimeSeries::Multi::Result
  series=
   #<Redis::TimeSeries:0x00007f833e408ad0
    @key="ts3",
    @redis=#<Redis client v4.2.5 for redis://127.0.0.1:6379/0>>,
  labels=[],
  samples=
   [#<Redis::TimeSeries::Sample:0x00007f833e408a58
     @time=2021-06-17 20:58:33 3246391/4194304 -0700,
     @value=0.1e1>,
    #<Redis::TimeSeries::Sample:0x00007f833e408850
     @time=2021-06-17 20:58:33 413139/524288 -0700,
     @value=0.3e1>,
    #<Redis::TimeSeries::Sample:0x00007f833e408670
     @time=2021-06-17 20:58:33 1679819/2097152 -0700,
     @value=0.2e1>]>]
[5] pry(main)> result.keys
=> ["ts3"]
[6] pry(main)> result['ts3'].values
=> [0.1e1, 0.3e1, 0.2e1]

Order them from newest to oldest with mrevrange

[8] pry(main)> Redis::TimeSeries.mrevrange(1.minute.ago.., filter: { foo: 'bar' }).first.values
=> [0.2e1, 0.3e1, 0.1e1]

Filter DSL

You can provide filter strings directly, per the time series documentation.

Redis::TimeSeries.where('foo=bar')
=> [#<Redis::TimeSeries:0x00007fb8981010c8...>]

There is also a hash-based syntax available, which may be more pleasant to work with.

Redis::TimeSeries.where(foo: 'bar')
=> [#<Redis::TimeSeries:0x00007fb89811dca0...>]

All six filter types are represented in hash format below.

{
  foo: 'bar',          # label=value  (equality)
  foo: { not: 'bar' }, # label!=value (inequality)
  foo: true,           # label=       (presence)
  foo: false,          # label!=      (absence)
  foo: [1, 2],         # label=(1,2)  (any value)
  foo: { not: [1, 2] } # label!=(1,2) (no values)
}

Note the special use of true and false. If you're representing a boolean value with a label, rather than setting its value to "true" or "false" (which would be treated as strings in Redis anyway), you should add or remove the label from the series.

Values can be any object that responds to .to_s:

class Person
  def initialize(name)
    @name = name
  end

  def to_s
    @name
  end
end

Redis::TimeSeries.where(person: Person.new('John'))
#=> TS.QUERYINDEX person=John

Compaction Rules

Add a compaction rule to a series.

# Destintation time series needs to be created before the rule is added.
other_ts = Redis::TimeSeries.create('other_ts')

# Aggregation buckets are measured in milliseconds
ts.create_rule(dest: other_ts, aggregation: [:count, 60000]) # 1 minute

# Can provide a string key instead of a time series object
ts.create_rule(dest: 'other_ts', aggregation: [:avg, 120000])

# If you're using Rails or ActiveSupport, you can provide an
# ActiveSupport::Duration instead of an integer
ts.create_rule(dest: other_ts, aggregation: [:avg, 2.minutes])

# Can also provide an Aggregation object instead of an array
agg = Redis::TimeSeries::Aggregation.new(:avg, 120000)
ts.create_rule(dest: other_ts, aggregation: agg)

# Class-level method also available
Redis::TimeSeries.create_rule(source: ts, dest: other_ts, aggregation: ['std.p', 150000])

Get existing compaction rules

ts.rules
=> [#<Redis::TimeSeries::Rule:0x00007ff46e91c728
  @aggregation=#<Redis::TimeSeries::Aggregation:0x00007ff46e91c6d8 @duration=3600000, @type="avg">,
  @destination_key="ts1",
  @source=
   #<Redis::TimeSeries:0x00007ff46da9b578 @key="ts3", @redis=#<Redis client v4.2.1 for redis://127.0.0.1:6379/0>>>]

# Get properties of a rule too
ts.rules.first.aggregation
=> #<Redis::TimeSeries::Aggregation:0x00007ff46d146d38 @duration=3600000, @type="avg">
ts.rules.first.destination
=> #<Redis::TimeSeries:0x00007ff46d8a3d60 @key="ts1", @redis=#<Redis client v4.2.1 for redis://127.0.0.1:6379/0>>

Remove an existing compaction rule

ts.delete_rule(dest: 'other_ts')
ts.rules.first.delete
Redis::TimeSeries.delete_rule(source: ts, dest: 'other_ts')

TODO

Development

After checking out the repo, run bin/setup. You need the docker daemon installed and running. This script will:

  • Install gem dependencies
  • Pull the latest redislabs/redistimeseries image
  • Start a Redis server on port 6379
  • Seed three time series with some sample data
  • Attach to the running server and print logs to STDOUT

With the above script running, or after starting a server manually, you can run bin/console to interact with it. The three series are named ts1, ts2, and ts3, and are available as instance variables in the console.

If you want to see the commands being executed, run the console with DEBUG=true bin/console and it will output the raw command strings as they're executed.

[1] pry(main)> @ts1.increment
DEBUG: TS.INCRBY ts1 1
=> 1593159795467
[2] pry(main)> @ts1.get
DEBUG: TS.GET ts1
=> #<Redis::TimeSeries::Sample:0x00007f8e1a190cf8 @time=2020-06-26 01:23:15 -0700, @value=0.4e1>

Use rake spec to run the test suite.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/dzunk/redis-time-series.

License

The gem is available as open source under the terms of the MIT License.

redis-time-series's People

Contributors

dzunk avatar fffx avatar gkorland avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

redis-time-series's Issues

Fix argument parsing for `TS.MADD` with multiple values

There's duplication between the class-level and instance-level madd methods, particularly around argument parsing. Ideally the instance-level one would call the class-level one, or at least some shared argument parser.

Also, there was something weird going on with timestamp parsing, might be fixed in #32, if not reproduce, add test, and fix parser.

Add support for beginless and endless range queries

It would be cool to add support for beginless and endless ranges in the TS.*RANGE queries, since the correct values can be determined in both cases. The TS.INFO call will return the first and last timestamps in a series, which would let us auto-complete the other side of the range.

Relax dependency restrictions and add build matrix

I've been developing for latest Ruby and Redis (4+), but if we can support Ruby 2.5.x and Redis 3.x, I don't see any reason not to do so.

Ruby 2.4 and below are EOL, so probably not going to worry about them.

Whichever combinations of versions make sense to support, need to add a build matrix to run specs against all of them, instead of just the latest ones.

May also want to pin the redis-time-series docker container version, or include that as part of the matrix as well.

Fix setting labels on `TS.CREATE` and `TS.ALTER`

I think I've used the wrong format for the command - the params need to be passed in an array instead of prebuilt into a string. May also be a good time to extract the label parsing functionality into a module, so it can be re-used in the TS.QUERYINDEX method.

Support redis-client gem

The new https://github.com/redis-rb/redis-client gem is a much simpler and lighter-weight implementation of a Ruby redis client, and drops support for older versions. This module already relies heavily on the low-level .call interface of the redis gem, because all the time-series commands are custom anyway.

Refactoring the gem to support multiple pluggable Redis clients will improve the architecture and leave the client choice in the hands of the end-user.

Add custom method for converting objects into labels

Right now, any object responding to .to_s can be used as a time series label, and while that implicitly means any object at all can be used by virtue of Object#to_s, the labels in question would be useless. Also, it's not unreasonable to think that you might not want to (re)define .to_s to make your object time-series compatible.

Add a new interface #time_series_label that will be used instead of .to_s on an object being cast to a label value. Raise a Redis::TimeSeries::LabelCoercionError if given an object that doesn't implement this interface.

class Person < ApplicationRecord
  def time_series_label
    "#{self.class.name.underscore}:#{id}"
  end
end
Redis::TimeSeries.where(person: Person.find(123))
# TS.QUERYINDEX person=person:123

Fix info struct for RedisTimeSeries 1.4.6

RTS 1.4.6 added a new chunkType attribute to the series info hash. Update the Info struct to support this attribute, and also don't break things when new values are added.

Redis::TimeSeries.madd fails when multiple keys are provided

Steps to reproduce the issue:

$ docker run --network host --name redis -it -d --rm redislabs/redistimeseries:latest
$ docker exec -it redis redis-cli
127.0.0.1:6379> ts.create foo
OK
127.0.0.1:6379> ts.create bar
OK
$ docker run --rm --name ruby -it  --network host ruby:3-alpine /bin/sh
/ # gem install redis redis-time-series
Fetching redis-4.6.0.gem
Successfully installed redis-4.6.0
Fetching redis-time-series-0.7.0.gem
Successfully installed redis-time-series-0.7.0
2 gems installed
/ # irb
irb(main):001:0> require 'redis'
=> true
irb(main):002:0> require 'redis-time-series'
=> true
irb(main):003:0> Redis::TimeSeries.madd(foo: { 1 => 1234 }, bar: { 2 => 2345 })
`Redis.current=` is deprecated and will be removed in 5.0. (called from: /usr/local/bundle/gems/redis-time-series-0.7.0/lib/redis/time_series/client.rb:47:in `redis')
/usr/local/bundle/gems/redis-time-series-0.7.0/lib/redis/time_series/sample.rb:19:in `BigDecimal': can't convert nil into BigDecimal (TypeError)
        from /usr/local/bundle/gems/redis-time-series-0.7.0/lib/redis/time_series/sample.rb:19:in `initialize'
        from /usr/local/bundle/gems/redis-time-series-0.7.0/lib/redis/time_series.rb:115:in `new'
        from /usr/local/bundle/gems/redis-time-series-0.7.0/lib/redis/time_series.rb:115:in `block (2 levels) in madd'
        from /usr/local/bundle/gems/redis-time-series-0.7.0/lib/redis/time_series.rb:114:in `each'
        from /usr/local/bundle/gems/redis-time-series-0.7.0/lib/redis/time_series.rb:114:in `each_with_index'
        from /usr/local/bundle/gems/redis-time-series-0.7.0/lib/redis/time_series.rb:114:in `each'
        from /usr/local/bundle/gems/redis-time-series-0.7.0/lib/redis/time_series.rb:114:in `map'
        from /usr/local/bundle/gems/redis-time-series-0.7.0/lib/redis/time_series.rb:114:in `block in madd'
        from <internal:kernel>:124:in `then'
        from /usr/local/bundle/gems/redis-time-series-0.7.0/lib/redis/time_series.rb:113:in `madd'
        from (irb):3:in `<main>'
        from /usr/local/lib/ruby/gems/3.1.0/gems/irb-1.4.1/exe/irb:11:in `<top (required)>'
        from /usr/local/bin/irb:25:in `load'
        from /usr/local/bin/irb:25:in `<main>'

The time series are updated, the error seems to be thrown when reply from redis is decoded

127.0.0.1:6379> ts.range foo - +
1) 1) (integer) 1
   2) 1234
127.0.0.1:6379> ts.range bar - +
1) 1) (integer) 2
   2) 2345

The same issue happens when issuing madd with multiple keys and timestamp/value pairs:

Redis::TimeSeries.madd(foo: { 1 => 1234, 10 => 12340 }, bar: { 2 => 2345, 20 => 23450 })

Get rid of `Time` monkey-patch

It was the easiest way to get Time objects converted to milliseconds, but monkey-patching stdlib is always a bad idea. Figure out a better way to handle timestamp conversion to milliseconds, either with refinements somehow, or a param parser for the commands that will auto-wrap Time objects.

Add `Rule` object for representing rules in info hash

The ts.info command (or the ts.rules alias) returns an array of compaction rules, which are basically a combination of a time series key and an aggregration.

Should have a Rule object that wraps those and provides accessor methods for getting the destination series and configured aggregation.

Remove deprecated usage of Redis.current

Redis.current and Redis.current= will be deprecated in redis-rb 5.0. Need to remove that from the default client handling, and replace it with... something? Redis.new I suppose? Or just require explicit client configuration.

Add `uncompressed` param to TS.ADD, TS.INCRBY, TS.DECRBY

Per the docs, the TS.ADD, TS.INCRBY, and TS.DECRBY commands can change the series' storage format from compressed to uncompressed. Currently, the gem only supports setting the value on create.

Add an optional uncompressed parameter to the add, incrby, and decrby methods that appends UNCOMPRESSED to the command string if truthy.

Nightly build against `redislabs/redistimeseries:edge`

The latest tag on the redislabs/redistimeseries image is tracking the current stable release, which right now is 1.2.7 released a little over a month ago. There have been two 1.4.x releases since then, and while it's not generally available yet, we should still be running tests agains the new version to proactively catch breaking changes.

There's another tag redislabs/redistimeseries:edge that seems to track against the master branch, so we should set up nightlies for that tag as well. Potentially a good opportunity to try out build matrices in Github actions.

Related to #23

Relax Redis version constraint

The redis dependency is currently pinned at >= 3.3, < 5. The greater than 3.3 requirement is because this gem depends on the Redis#call interface to pass through otherwise unknown commands to the Redis client.

The less than 5 requirement, however, was just arbitrary to defend against potential breaking changes in a theoretical future version of redis-rb. This isn't best practice, as it prevents installation alongside that version and the very ability to make those compatibility tests, if/when version 5 does come out.

Should relax the version constraint to only >= 3.

Clean up connection handling and debugging

Kind of threw together the class-level connection handling code when adding .madd. This ought to be refactored to work with the instance-level client, and avoid copypasta'ing debug statements and redis.call everywhere.

Support connection pools

The default redis client is a single instance of a single-threaded client, so using multiple time-series in parallel would be blocking. Should add support for the ConnectionPool gem to yield a client to a block if a pool is available.

Add filter DSL

https://oss.redislabs.com/redistimeseries/commands/#filtering

Easy route would be to pass strings along, complex but more OOP route would probably be a Redis::TimeSeries::Filter class that parses the various options, or provides a chainable DSL.

l=v  # label equals value
l!=v # label doesn't equal value
l=   # key does not have the label l
l!=  # key has label l
l=(v1,v2,...)  # key with label l that equals one of the values in the list
l!=(v1,v2,...) # key with label l that doesn't equals to the values in the list

Ideas

Redis::TimeSeries::Filter.parse('l=val1 l!=val2') # parse a raw string
Redis::TimeSeries::Filter.new(
  label: 'value',  # l=v
  label: nil,      # l= option 1
  label: false,    # l= option 2
  label: '*',      # l!= option 1
  label: :any,     # l!= option 2
  label: true,     # l!= option 3
  label: [1, 2, 3] # l=(1,2,3)
  not: {
    # This would mean you couldn't have a label named `not`...
    label: 'value', # l!=v
    label: [1, 2]   # l!=(1, 2)
  },
  label: { not: 'value' },
  label: { not: [1, 2] }
)
Redis::TimeSeries::Filter
  .with(label: 'value')
  .without(other_label: 'other_value')

Redis::TimeSeries::Filter
  .where(label: 'value')
  .where_not(other_label: 'other_value')

Is there a mocking library?

This is a really great addition to the ruby eco system - thanks! Is there a way to mock the library from rspecs?

undefined method ts_msec

Hey thanks so much for this gem!

I noticed that ts_msec has been defined for Time but not for ActiveSupport::TimeWithZone and that is why I got this error:

irb(main):002:0> test_ts = Redis::TimeSeries.new('test', redis: $redis_enterprise)
=> #<Redis::TimeSeries:0x000000010c1c7b88 @key="test", @redis=#<Redis client v4.5.1 for redis://127.0.0.1:6379/0>>
irb(main):003:0> test_ts.add(1234, Time.current)
/Users/thushara/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/bundler/gems/rails-fc4d83f43cd3/activesupport/lib/active_support/time_with_zone.rb:535:in `method_missing': undefined method `ts_msec' for Thu, 21 Jul 2022 17:41:05.380072000 UTC +00:00:Time (NoMethodError)

      wrap_with_time_zone time.__send__(sym, *args, &block)
                              ^^^^^^^^^
/Users/thushara/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/bundler/gems/rails-fc4d83f43cd3/activesupport/lib/active_support/time_with_zone.rb:535:in `method_missing': undefined method `ts_msec' for 2022-07-21 17:41:05.380072 UTC:Time (NoMethodError)

      wrap_with_time_zone time.__send__(sym, *args, &block)
                              ^^^^^^^^^
irb(main):004:0>

Wondering if there is an easy fix/work-around?

Cast label values to integers

This is more for convenience, but if a label's value is cast-able to an integer or a boolean, it would be nice to return it as such instead of the string value.

Instead of:

{
  "foo" => "bar",
  "baz" => "1",
  "plugh" => "true"
}

#labels would return:

{
  "foo" => "bar",
  "baz" => 1,
  "plugh" => true
}

Support Ruby 3.0

  • Add ruby 3.0 to build matrix
  • Ensure argument forwarding is set up correctly where necessary
  • Update any version constraints (not sure there are any?)
  • Other items here?

Support TS.MRANGE/MREVRANGE GROUPBY <label> REDUCE <reducer>

The following PR RedisTimeSeries/RedisTimeSeries#617 added support for multi-series aggregations to RedisTimeSeries.

It adds GROUPBY <label> REDUCE <reducer> to TS.MRANGE/TS.MREVRANGE. It accepts SUM, MIN, MAX reducers and works with unaligned series.

Sample request:

TS.MRANGE 1451679382646 1451682982646 WITHLABELS 
AGGREGATION MAX 60000 
FILTER measurement=cpu 
      fieldname=usage_user 
      hostname=(host_9,host_3,host_5,host_1,host_7,host_2,host_8,host_4)
GROUPBY hostname REDUCE MAX

Reply labels array structure

Labels:

  • <label>=<groupbyvalue>
  • __reducer__=<reducer>
  • __source__=key1,key2,key3

Sample output:

127.0.0.1:6379> ts.add ts1 1 90 labels metric cpu name system
(integer) 1
127.0.0.1:6379> ts.add ts1 2 45 labels metric cpu name system
(integer) 2
127.0.0.1:6379> ts.add ts2 2 99 labels metric cpu name user
(integer) 2
127.0.0.1:6379> ts.add ts3 2 02 labels metric cpu name system
(integer) 2

MAX reducer sample output

127.0.0.1:6379> ts.mrange - + withlabels filter metric=cpu groupby name reduce max
1) 1) "name=system"
   2) 1) 1) "name"
         2) "system"
      2) 1) "__reducer__"
         2) "max"
      3) 1) "__source__"
         2) "ts1,ts3"
   3) 1) 1) (integer) 1
         2) 90
      2) 1) (integer) 2
         2) 45
2) 1) "name=user"
   2) 1) 1) "name"
         2) "user"
      2) 1) "__reducer__"
         2) "max"
      3) 1) "__source__"
         2) "ts2"
   3) 1) 1) (integer) 2
         2) 99

MIN reducer sample output

127.0.0.1:6379> ts.mrange - + withlabels filter metric=cpu groupby name reduce min
1) 1) "name=system"
   2) 1) 1) "name"
         2) "system"
      2) 1) "__reducer__"
         2) "min"
      3) 1) "__source__"
         2) "ts1,ts3"
   3) 1) 1) (integer) 1
         2) 90
      2) 1) (integer) 2
         2) 2
2) 1) "name=user"
   2) 1) 1) "name"
         2) "user"
      2) 1) "__reducer__"
         2) "min"
      3) 1) "__source__"
         2) "ts2"
   3) 1) 1) (integer) 2
         2) 99

Add `Result` wrapper when querying

Right now, the result of queries like TS.RANGE is an array of value objects. To make this more user-friendly for using the raw data (e.g. displaying on a chart), it would be nice to return a Redis::TimeSeries::Result wrapper to hold those value objects, and provide convenience wrappers for converting them into hashes and arrays.

[9] pry(main)> result
=> [#<Redis::TimeSeries::Sample:0x00007f803088b7f8 @time=2020-06-29 23:43:12 -0700, @value=0.1e1>,
 #<Redis::TimeSeries::Sample:0x00007f803088b640 @time=2020-06-29 23:43:12 -0700, @value=0.2e1>,
 #<Redis::TimeSeries::Sample:0x00007f803088b488 @time=2020-06-29 23:43:12 -0700, @value=0.3e1>]

# Could return 2d array/hash of times and values (Time object probably more useful elsewhere in a Ruby app)
[10] pry(main)> result.map { |r| [r.time, r.value] }
=> [[2020-06-29 23:43:12 -0700, 0.1e1], [2020-06-29 23:43:12 -0700, 0.2e1], [2020-06-29 23:43:12 -0700, 0.3e1]]
[11] pry(main)> result.to_h { |r| [r.time, r.value] }
=> {2020-06-29 23:43:12 -0700=>0.1e1, 2020-06-29 23:43:12 -0700=>0.2e1, 2020-06-29 23:43:12 -0700=>0.3e1}

# Or, convert to raw milliseconds first (closer to raw reply from Redis)
[12] pry(main)> result.map { |r| [r.time.ts_msec, r.value] }
=> [[1593499392263, 0.1e1], [1593499392276, 0.2e1], [1593499392298, 0.3e1]]
[13] pry(main)> result.to_h { |r| [r.time.ts_msec, r.value] }
=> {1593499392263=>0.1e1, 1593499392276=>0.2e1, 1593499392298=>0.3e1}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.