perf: fractional cascading on seeks about pebble HOT 6 CLOSED

We have marked this issue as stale because it has been inactive for
18 months. If this issue is still relevant, removing the stale label
or adding a comment will keep it active. Otherwise, we'll close it
in 10 days to keep the issue queue tidy. Thank you for your
contribution to Pebble!

from pebble.

This still seems legit.

from pebble.

I wonder how RocksDB maintains the index for fractional cascading? Each compaction shifts all the relative positions within the level. We've seen anything O(# files in a level) show up as a bottleneck. Updating the relative positions of all files within the touched levels seems prohibitively expensive.

from pebble.

My recollection is that RocksDB rebuilds the affected levels and updates the index for fractional cascading on each compaction. One of the big optimizations for large data sizes you made in Pebble was to avoid such full rebuilds (i.e. the per-level btree).

It strikes me that a b-epsilon tree gets something like fractional cascading because the metadata is in a single b-tree. I don't see how we could achieve that in Pebble, but thought I'd mention it.

from pebble.

Once we have the very large LSMs for compaction benchmarking, we could do some iteration benchmarks on them to see how much time is actually spent in this binary search.

from pebble.

We have marked this issue as stale because it has been inactive for
18 months. If this issue is still relevant, removing the stale label
or adding a comment will keep it active. Otherwise, we'll close it
in 10 days to keep the issue queue tidy. Thank you for your
contribution to Pebble!

from pebble.