derrickburns / tdigest Goto Github PK

The following test case reproduces the problem:

TEST_F(TDigestTest, CrashAfterMerge) {
  tdigest::TDigest digest(1000);
  std::uniform_real_distribution<> reals(0.0, 1.0);
  std::random_device gen;
  for (int i = 0; i < 100000; i++) {
    digest.add(reals(gen));
  }
  digest.compress();

  tdigest::TDigest digest2(1000);
  digest2.merge(&digest);
  digest2.quantile(0.5);
}

From my debugging, it seems the crash happens in line 338, because the cumulative_ structure is empty after the merge. Adding a call to updateCumulative() at the end of merge() fixes the crash.

Some background: I was using your implementation in a distributed setting, were partial results are serialized, and sent to a central node, where they are deserialized and merged into one result. The above test case simulates this and also triggers the crash (so the serialization is not the problem). Maybe the call to compress() is just wrong, but it instinctively seemed correct to compress before serialization ;-)

Would it be possible to choose a license for this project and add a LICENSE file?

Hi!
I would like to compute quantiles for data of mine and stumbled over this. I browsed through the code but have really no idea, how I can compute quantiles. Would you have a quick guide, what functions to call or classes to instantiate to get a number of quantiles from a data set?

This should be done in the following line:

I don't know if you want anything we have in the Java code, but this is a tickler to have a look.

I just found some insidious issues relative to repeated counts that you may care about. If you have a stable sort, these may not have impacted you.