Skip to content

db/state: clip merge windows that straddle existing files#20909

Merged
AskAlexSharov merged 5 commits into
mainfrom
wmitsuda/merge-criteria
May 4, 2026
Merged

db/state: clip merge windows that straddle existing files#20909
AskAlexSharov merged 5 commits into
mainfrom
wmitsuda/merge-criteria

Conversation

@wmitsuda

Copy link
Copy Markdown
Member

possible solution to fix #20878

When the natural merge start (endTxNum minus the largest power-of-two
step span) falls strictly inside an existing visible file, bump it up
to that file's endTxNum so the candidate window cannot straddle.
Without this clip, non-power-of-2 step layouts — such as those produced
by a step-size rebase that turns one step into a non-power-of-2 number
of tx-nums — let the algorithm propose merge windows whose `from` lies
inside a pre-existing file. Downstream staticFilesInRange would then
silently drop that file and emit a merged segment whose name lies about
its coverage, leaving an overlap with the surviving straddler.

Unskip partial_overlap_must_not_be_selected (which now passes), and add
step_rebase_swallow covering the headline case where a single-step file
arriving at N=2_048_000 has natural span 16384 / natural start 2031616
that straddles [2016000, 2032000). After the clip the safe window
{2032000, 2048000} cleanly absorbs 15 trailing files.
@wmitsuda wmitsuda mentioned this pull request Apr 29, 2026
sudeepdino008 pushed a commit that referenced this pull request May 4, 2026
Cherry-pick of #20909 (by @wmitsuda), adapted for this branch.

Fixes #20878.

When the natural merge start (endTxNum minus the largest power-of-two
step span) falls strictly inside an existing visible file,
`clipMergeStartToFileBoundary` bumps it up to that file's endTxNum.
Without this clip, non-power-of-2 step layouts (e.g. after a step-size
rebase) let the algorithm propose windows whose `from` lies inside a
pre-existing file — `staticFilesInRange` then silently drops that file
and emits a merged segment that lies about its coverage.

Applied to `DomainRoTx`, `HistoryRoTx`, and `InvertedIndexRoTx`
`findMergeRange` (this branch has inline merge logic vs. the extracted
`findMergeRangeInFiles` helper on main).

Also: wrap walk errors with path context; guard `collectTorrentFiles`
against a missing torrent dir.
@AskAlexSharov AskAlexSharov marked this pull request as ready for review May 4, 2026 11:24
@AskAlexSharov AskAlexSharov enabled auto-merge May 4, 2026 11:24
@AskAlexSharov AskAlexSharov added this pull request to the merge queue May 4, 2026
Merged via the queue into main with commit 0237bd1 May 4, 2026
68 checks passed
@AskAlexSharov AskAlexSharov deleted the wmitsuda/merge-criteria branch May 4, 2026 12:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flexible merge ranges

2 participants