Using BOTTOM priority threadpool to schedule GC will influence bottom level compaction about titan HOT 10 CLOSED

@yiwu-arbug @Connor1996

from titan.

Interesting finding. I misunderstood what bottom-pri mean. I thought it's kind of a lowest-pri threadpool where bottommost level will use the normal low-pri pool if the bottom-pri pool is fully utilized.

from titan.

@yiwu-arbug How about using Priority::USER instead for Titan GC?

from titan.

Seems we can leverage the bottom-pri pool to reduce the write stall possibility when level-merge/range-merge enabled. See facebook/rocksdb#3835

Also, I'm curious how you find this problem. As expected, we will disable Titan background GC when level-merge enabled.

from titan.

Also, I'm curious how you find this problem. As expected, we will disable Titan background GC when level-merge enabled.

Because I found max_background_gc will influence space amplification significanttly (very!) even if BackgroundGCCall is bypassed by level_merge, same as actual size ratio among LSM-Tree levels.

from titan.

@yiwu-arbug How about using Priority::USER instead for Titan GC?

I have tried, I think it's ok.

from titan.

@yiwu-arbug How about using Priority::USER instead for Titan GC?

Let's do that for now.

from titan.

Because I found max_background_gc will influence space amplification significantly (very!) even if BackgroundGCCall is bypassed by level_merge, same as actual size ratio among LSM-Tree levels.

Wait a minute. When Titan background GC is disabled, the BOTTOM is not set no matter what's the value of max_background_gc. why it will influence still?

from titan.

Because I found max_background_gc will influence space amplification significantly (very!) even if BackgroundGCCall is bypassed by level_merge, same as actual size ratio among LSM-Tree levels.

Wait a minute. When Titan background GC is disabled, the BOTTOM is not set no matter what's the value of max_background_gc. why it will influence still?

we did not disable GC, just bypassed MaybeScheduleGC() call after compactoin.

from titan.

oh, so you didn't set disable_background_gc explicitly. got it.

from titan.