Snapshot optimization about speedb HOT 5 CLOSED

Performance tests pass, the results are the same as current main.

from speedb.

An experimental implementation is in https://github.com/speedb-io/speedb/tree/exp/snapshot-reuse
Note that this implementation is incomplete and suffers from at least the following two bugs (rephrased from Jira):

last_snapshot_ is incorrectly cached

A caching of the last taken snapshot was implemented in order to facilitate faster snapshot taking in read-mostly workloads, such as pure select MySQL benchmarks, where MySQL is taking a snapshot for each query, and the global DB mutex serializes the reads so it effectively limits the amount of parallelism possible.

That work allows avoiding taking the mutex in read-only workloads and taking the mutex for much shorter periods in workloads that include writes by reusing the last snapshot if it’s still valid. However, in order to ensure that we always hold a valid reference, the DB instance also takes a reference on that snapshot and keeps it alive until the next snapshot is taken, even if multiple writes happened in between, rendering that snapshot useless (because a snapshot is based on the DB’s sequence number, and every write increments it). Moreover, having a live obsolete snapshot (after everyone who needed it released it) prevents the compaction from discarding old keys, increasing space and write amplification.

We need to release the cached snapshot on writes (at least some part of the write is done under the mutex, so it’s valid to release the snapshot during that phase). However, there is a need to be careful about when to release the snapshot, since in the case of transactions LastPublishedSequenceNumber() is used rather than LastSequenceNumber(), so we only want to release the cached snapshot after the former is incremented. The fact that multiple write flows are currently implemented in RocksDB (single queue, two queues, unordered writes, etc.), with our own addition thrown into the mix with #23, this is a task that potentially involves touching many parts of the code.

snapshots: min_uncommitted_ may be overwritten by other transactions when providing last_snapshot_

When the last snapshot matches the current sequence number, with the snapshot optimisation we return the same SnapshotImpl instance that we provide to other callers. However, WritePrepared and WriteUnprepared transactions also make use of the min_uncommitted_ field in SnapshotImpl to store their own information about the state of the transaction. This may conflict between different transactions, leading to unexpected results.

We need to check all usages of min_uncommitted_ to see if that’s indeed an issue, and if so, we need to fix it before merging it, as it may cause a correctness issue in those cases.

from speedb.

New WIP branch that tries to tackle these issues, but is currently incomplete and experiences data races between reference subtraction and SnapshotRecord release: isaac/snapshot.

from speedb.

@erez-speedb - Please run standard tests.

from speedb.

This Feature must have folly library installed.

How to install folly on Ubuntu:

sudo apt install libssl-dev libfmt-dev
git clone https://github.com/facebook/folly
cd folly
sudo ./build/fbcode_builder/getdeps.py install-system-deps --recursive
mkdir build_
cd build_
cmake .. -DBUILD_SHARED_LIBS=ON
make -j $(nproc) install

If there is an error with the compile process above that is related with fmt,
Please run:

 sed -i "s/format_to/fmt::format_to/g" /usr/include/fmt/chrono.h

To compile speedb with this enhancement please compile it using cmake:

git clone https://github.com/speedb-io/speedb
cd speedb
mkdir build
cd build
cmake .. -DWITH_SNAP_OPTIMIZATION=ON -DBUILD_SHARED_LIBS=ON
make -j $(nproc)

from speedb.