Many disks are shutdown due to heavy workload about juicefs HOT 7 CLOSED

#4749
In this PR, we have added a fault tolerance range to reduce misjudgments, which aligns with your scenario. If needed, maxIOErrors can be adjusted as configurable.

from juicefs.

Thanks, but we already included this commit in our build:

from juicefs.

In this writing scenario, maybe turn on maxStageWrite is better

from juicefs.

When one disk cache store is shutdown. Will we lose staged files in that store?

from juicefs.

files still exist.

from juicefs.

Conclusion: There are two scenarios to consider. One is long-term high-load writing, for which writing directly to object storage is recommended (resource issue: disk-cache or object storage bandwidth). The other is short-term high-load writing, for which the concurrency limit can be lifted to redirect and distribute writes to the object storage. This issue will be temporarily closed for now.

from juicefs.

We should enable max-stage-write for this issue, maybe 10 as the default value.

from juicefs.