perf: faster ingestion about pebble HOT 3 CLOSED

Would we need to create a new memtable on every ingestion so that iterators may respect sequence order within the flushables?

from pebble.

Would we need to create a new memtable on every ingestion so that iterators may respect sequence order within the flushables?

Yes. That could leave a significant amount of wasted space in the memtables, but I think that memory is accounted for and will result in a flush eventually occurring.

As mentioned in cockroachdb/cockroach#62700, if the ingested sstables are small we could convert the ingestions into write batches. Same idea as above that the WAL entry would point to the sstable on disk, but rather than appending the sstable to the memtable list we loop over the contents of the sstable and insert it into the memtable.

from pebble.

DB.Ingest currently experiences a hiccup if the table being ingested overlaps with a memtable: ingestion needs to wait for the memtable to be flushed.

My understanding is that there is also a hiccup for concurrent normal writes that are assigned a seqnum after this ingest. They will need to wait until their seqnum becomes visible, which is blocked behind the ingest waiting for the memtable(s) to be flushed so that it can then update the manifest.

from pebble.