When flow control is enabled:

• If level0-slowdown-writes-trigger and level0-stop-writes-trigger are explicitly set, l0-file-threshold overrides them only when it is smaller. --> This behavior is likely intend for upgrade compatiability. Refer to this comment.
• If they are not set, it overrides them unconditionally. --> This is required for correctness on SST ingestion flow control. In current implementation ingest_maybe_stall relies on the level0-slowdown-writes-trigger but the actual slowdown is controled by the flow-control.l0-file-threshold. So the simplest way to ensure ingest_maybe_stall integrates well with the flow control is to keep this two configuation in sync. Refer to config: override rocksdb config when flow control enabled #11840.

Similarly, flow-control.soft-pending-compaction-bytes-limit also overrides rocksdb.*cf.soft-pending-compaction-bytes-limit, and rocksdb.*cf.hard-pending-compaction-bytes-limit also overrides rocksdb.*cf.hard-pending-compaction-bytes-limit.

In recent tests, we encountered performance instability when flow control was enabled and either flow-contrl.l0-file-threshold or flow-control.soft-pending-compaction-bytes-limit was increased. Although the intention was to reduce the frequency of triggering flow control for stable performance, in reality it became more frequent, and the performance became less stable. As shown in the figure below:

Upon investgation, we found the root cause: compaction QPS dropped after increase this configurations. This is because RocksDB speeds up compaction when approaching level0-slowdown-writes-trigger and soft-pending-compaction-bytes-limit. Increase these values wakens the compaction spped up mechanism.

To address this issue, I think we should not overriding the level0-slowdown-writes-trigger and soft-pending-compaction-bytes-limit, and ingest_maybe_stall can reply on level0-stop-writes-trigger, which can still be override by the l0-file-threshold, preserving correctness.