Giter VIP home page Giter VIP logo

rust-sketches-ddsketch's Issues

Merge may result in index out of bounds panic

I've been working on a feature for the metrics crate (metrics-rs/metrics#306) that uses the DDSketch::merge function, but under load I am frequently getting panics:

index out of bounds: the len is 10939 but the index is 10939', /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/sketches-ddsketch-0.1.3/src/store.rs:155:22

I have written a randomized test program:

use rand::prelude::*;
use rand::rngs::StdRng;
use sketches_ddsketch::{Config, DDSketch};

fn main() {
    let config = Config::new(0.0001, 32_768, 1.0e-9);
    let mut sketches = vec![
        DDSketch::new(config),
        DDSketch::new(config),
        DDSketch::new(config)];


    let mut rng: StdRng = rand::SeedableRng::seed_from_u64(42);
    let mut i = 1;
    loop {
        let samples = rng.gen_range(0..20000);
        sketches[0] = DDSketch::new(config);
        for _ in 0..samples {
            let rv: i64 = rng.gen_range(0..8000000000);
            sketches[0].add(rv as f64);
        }

        let mut merged = DDSketch::new(config);
        for sketch in &sketches {
            merged.merge(sketch).unwrap();
        }
        sketches.rotate_right(1);

        if i % 1000 == 0 {
            print!(".");
        }
        i += 1;
    }
}

For me, when run in release mode, this program crashes after a few seconds with:

thread 'main' panicked at 'index out of bounds: the len is 32768 but the index is 18446744073709551613', /home/dbn/.cargo/registry/src/github.com-1ecc6299db9ec823/sketches-ddsketch-0.1.3/src/store.rs:170:21

This is a different line than the panics in my original program, so I'm not sure if it is the same issue.

v0.2 Regression in Single-Value Quantile

There appears to be a regression in v0.2 for single-value sketches... quick testing shows the issue with any value, and any quantile:

    let mut dd = DDSketch::new(Config::defaults());

    dd.add(1234f64);

    assert_eq!(1234f64, dd.quantile(0.5).unwrap().unwrap());

The above works (produces 1234.0) in v0.1.3, but produces 1224.3764974385315 in v0.2.0/master.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.