Skip to content

compact() regresses L4 files to L2 when compacting mixed-level groups #27230

@davidby-influx

Description

@davidby-influx

The compaction planner change in d1d68d6 ("fix: handle nested low-level files in compaction") modified Plan() to include lower-level files that are nested among level 4 files, and the cold/forced compaction path has always collected all files regardless of level. Both paths can produce compaction groups that mix L1 files (seq=1) with L4 files (seq≥4). However, compact() computes the output filename using the max sequence scoped to the max generation only. When a newer L1 file has the highest generation number, its sequence of 1 is used, producing a level 2 output — regressing already-optimized L4 data to a lower level.

The most common trigger is cold compaction: a shard with L4 files receives new writes (creating L1 files), then goes idle for 4+ hours. If any large L4 generation is skipped by the cold planner's size check, the regressed output file becomes stranded — PlanLevel can't compact it (only one generation at that level) and hot Plan() can't reach it either. The shard depends entirely on repeated cold compaction cycles to recover, each bumping the level by 1, requiring up to 12+ hours and multiple full rewrites of already-compacted data to return to level 4.

Metadata

Metadata

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions