mheffner / rust-sketches-ddsketch Goto Github PK
View Code? Open in Web Editor NEWRust port of the Go implementation of DDSketch
Home Page: https://docs.rs/sketches-ddsketch
License: Apache License 2.0
Rust port of the Go implementation of DDSketch
Home Page: https://docs.rs/sketches-ddsketch
License: Apache License 2.0
I've been working on a feature for the metrics crate (metrics-rs/metrics#306) that uses the DDSketch::merge function, but under load I am frequently getting panics:
index out of bounds: the len is 10939 but the index is 10939', /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/sketches-ddsketch-0.1.3/src/store.rs:155:22
I have written a randomized test program:
use rand::prelude::*;
use rand::rngs::StdRng;
use sketches_ddsketch::{Config, DDSketch};
fn main() {
let config = Config::new(0.0001, 32_768, 1.0e-9);
let mut sketches = vec![
DDSketch::new(config),
DDSketch::new(config),
DDSketch::new(config)];
let mut rng: StdRng = rand::SeedableRng::seed_from_u64(42);
let mut i = 1;
loop {
let samples = rng.gen_range(0..20000);
sketches[0] = DDSketch::new(config);
for _ in 0..samples {
let rv: i64 = rng.gen_range(0..8000000000);
sketches[0].add(rv as f64);
}
let mut merged = DDSketch::new(config);
for sketch in &sketches {
merged.merge(sketch).unwrap();
}
sketches.rotate_right(1);
if i % 1000 == 0 {
print!(".");
}
i += 1;
}
}
For me, when run in release mode, this program crashes after a few seconds with:
thread 'main' panicked at 'index out of bounds: the len is 32768 but the index is 18446744073709551613', /home/dbn/.cargo/registry/src/github.com-1ecc6299db9ec823/sketches-ddsketch-0.1.3/src/store.rs:170:21
This is a different line than the panics in my original program, so I'm not sure if it is the same issue.
There appears to be a regression in v0.2 for single-value sketches... quick testing shows the issue with any value, and any quantile:
let mut dd = DDSketch::new(Config::defaults());
dd.add(1234f64);
assert_eq!(1234f64, dd.quantile(0.5).unwrap().unwrap());
The above works (produces 1234.0) in v0.1.3, but produces 1224.3764974385315 in v0.2.0/master.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.