Skip to content

fix(#4319): MutablePage.move mis-tracked WAL range for backward shifts#4325

Merged
robfrank merged 3 commits into
mainfrom
arcadedb-bug-4
May 25, 2026
Merged

fix(#4319): MutablePage.move mis-tracked WAL range for backward shifts#4325
robfrank merged 3 commits into
mainfrom
arcadedb-bug-4

Conversation

@robfrank

Copy link
Copy Markdown
Collaborator

Closes #4319

Summary

MutablePage.move() called updateModifiedRange(startPosition, destPosition + length). For backward shifts (destPosition < startPosition — the LocalBucket.defragPage case), the lower bound was startPosition instead of destPosition, leaving the range [destPosition, startPosition) untracked in the WAL delta. A crash after defragPage would replay WAL without those bytes, silently corrupting the bucket page.

Fix: use Math.min(startPosition, destPosition) as the lower bound. The upper bound stays destPosition + length, which is always within page bounds (validated by Binary.move before updateModifiedRange is reached). The issue's suggested Math.max(start, dest) + length variant was not used because for a backward shift it can equal the physical page size and trigger updateModifiedRange's bounds check.

Test plan

  • New MutablePageMoveTest unit tests verify correct getModifiedRange() after backward shift, forward shift, same-position shift
  • RecordRecyclingTest (which exercises defragPage via record deletion and recycling) continues to pass
  • All 4 new tests pass, no regressions in the engine test suite

🤖 Generated with Claude Code

For backward shifts (destPosition < startPosition), updateModifiedRange
was called with startPosition as the lower bound instead of destPosition,
leaving [destPosition, startPosition) untracked in the WAL delta.
A crash after defragPage would replay WAL without those bytes,
corrupting the bucket page.

Fix: use Math.min(startPosition, destPosition) as the lower bound so
backward shifts start tracking at destPosition. Upper bound stays
destPosition + length which is always within page bounds.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@codacy-production

codacy-production Bot commented May 25, 2026

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Coverage 100.00% diff coverage · -7.36% coverage variation

Metric Results
Coverage variation -7.36% coverage variation
Diff coverage 100.00% diff coverage

View coverage diff in Codacy

Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (d1b1632) 127537 93311 73.16%
Head commit (aa361a3) 159216 (+31679) 104777 (+11466) 65.81% (-7.36%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#4325) 2 2 100.00%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request addresses issue #4319 by fixing a bug in MutablePage.move where backward shifts were incorrectly tracked in the Write-Ahead Log (WAL), potentially leading to page corruption upon recovery. The fix ensures the modified range lower bound is the minimum of the source and destination positions. Review feedback correctly identified an off-by-one error in the upper bound calculation of updateModifiedRange, which could trigger an IllegalArgumentException when data is moved to the end of a page. The reviewer also suggested ensuring the range is only updated for moves with a length greater than zero and adding a specific test case for boundary conditions at the end of the page.

Comment thread engine/src/main/java/com/arcadedb/engine/MutablePage.java Outdated
Comment thread docs/4319-mutablepage-move-wal-tracking.md Outdated
page.move(50, 50, 60);
final int[] range = page.getModifiedRange();
assertThat(range[0]).isEqualTo(50 + BasePage.PAGE_HEADER_SIZE);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

It would be beneficial to add a test case for moving data to the very end of the page to ensure that the inclusive boundary logic is correctly handled and doesn't trigger out-of-bounds exceptions.

  }

  @Test
  void moveToEndOfPage() {
    final MutablePage page = freshPage();
    // PAGE_SIZE=1024, HEADER=8. Max content length is 1016.
    // Move 10 bytes to the very end of the page (dest 1006 + length 10 = 1016 relative to header).
    page.move(0, 1006, 10);
    final int[] range = page.getModifiedRange();
    assertThat(range[1]).isEqualTo(1023);
  }

@claude

claude Bot commented May 25, 2026

Copy link
Copy Markdown

Code review for PR 4325 - MutablePage.move WAL range fix. The one-line fix (Math.min for lower bound) is correct and the root cause analysis is accurate. Four items before merging: (1) Remove docs/4319-mutablepage-move-wal-tracking.md - it is an analysis doc that belongs only in the PR description, not the source tree. (2) Pre-existing: move() passes destPosition+length as exclusive end to updateModifiedRange, but the convention elsewhere (checkBoundariesOnWrite line 235) is inclusive (start+length-1). Side effects: one extra byte over-tracked in WAL (harmless), and a potential bounds-check throw if bytes are moved to the very last position in the page. Not introduced by this PR but worth a follow-up fix changing the upper bound to destPosition+length-1. (3) Comment in forwardShiftModifiedRangeStartsAtSource is ambiguous - it says 'written bytes start at destPosition but range from source' which could mislead a reader into thinking the range should start at destPosition. Suggest: 'range starts at min(start,dest)=start for forward shifts; verifies the fix does not regress forward-shift behaviour.' (4) Upper bound assertion in backwardShiftModifiedRangeCoversWrittenBytes is loose - it only checks range[1] >= destPosition+length-1 with no upper cap, so it would pass even if modifiedRangeTo were MAX_VALUE. Adding range[1] <= destPosition+length would tighten it. What is good: backwardShiftModifiedRangeStartsAtDest directly catches the regression (old code gives range[0]=108, fix gives 58), Mockito is consistent with SQLMethodSizeTest, tests are in the right package for package-private access, no new dependencies.

robfrank and others added 2 commits May 25, 2026 14:33
…lePage.move

- Use Math.max(start, dest) + length - 1 (inclusive) for updateModifiedRange
  upper bound, consistent with all other call sites
- Add length > 0 guard to prevent inverted range when length is zero
- Add moveToEndOfPageDoesNotThrow test for boundary regression

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@claude

claude Bot commented May 25, 2026

Copy link
Copy Markdown

Code Review - PR #4325: fix(#4319): MutablePage.move mis-tracked WAL range for backward shifts

Overview

This PR fixes a data-corruption bug in MutablePage.move(): for backward shifts (destPosition < startPosition), the WAL delta was missing the byte range [destPosition, startPosition), which would be silently dropped on crash recovery. The fix is targeted, correct, and well-motivated.


Core Fix (MutablePage.java)

The change is correct. The new formula:

if (length > 0)
  updateModifiedRange(Math.min(startPosition, destPosition), Math.max(startPosition, destPosition) + length - 1);
  • Correctly uses Math.min to anchor the lower bound at the first written byte regardless of direction.
  • The length > 0 guard prevents an inverted range (start > end) being passed to updateModifiedRange when nothing is moved - good defensive code.

Bonus off-by-one fix: The original code used destPosition + length as the upper bound, which is exclusive-end semantics, while updateModifiedRange expects an inclusive last-byte index. The original was over-tracking by exactly 1 byte upward for forward shifts - benign but incorrect. The new + length - 1 is consistent with every other updateModifiedRange call site (e.g. checkBoundariesOnWrite uses start + length - 1). This is a correct incidental cleanup.

Conservative over-tracking for forward shifts: For dest > start, the tracked range is [startPosition, destPosition + length - 1], while only [destPosition, destPosition + length - 1] is actually written. This was also true in the original code and is harmless - the WAL includes the source region as well. Not a problem.


Test Coverage (MutablePageMoveTest.java)

Good targeted regression tests. A few observations:

  1. forwardShiftModifiedRangeStartsAtSource - misleading comment: The inline comment says "written bytes start at destPosition" but the assertion checks that range[0] == source + HEADER. This is correct behavior (conservative tracking from source), but the comment contradicts what the test actually validates. Consider revising to: "range tracks from source for forward shifts (conservative over-tracking)".

  2. Missing length == 0 test: The if (length > 0) guard means updateModifiedRange is skipped entirely for a zero-length move, leaving modifiedRangeFrom/modifiedRangeTo at their sentinel values. A test like:

    @Test
    void zeroLengthMoveDoesNotMarkModified() {
      final MutablePage page = freshPage();
      page.move(50, 100, 0);
      // modifiedRangeFrom stays at Integer.MAX_VALUE
      assertThat(page.getModifiedRange()[0]).isEqualTo(Integer.MAX_VALUE);
    }

    would document the intended no-op behavior.

  3. RecordRecyclingTest reference: The PR description mentions this test continues to pass, but it isn't in the diff. Confirming it was run is enough - no change needed.


Documentation File (docs/4319-mutablepage-move-wal-tracking.md)

Recommend removing this file. It's an ad-hoc analysis document tied to a specific issue number. This type of write-up belongs in the PR description (where it already lives, well-written) rather than checked into the source tree permanently. Files like this tend to become stale and clutter the docs/ directory with issue-scoped content that has no audience after the fix ships.

Additionally, there is a minor inaccuracy in the doc:

"The upper bound stays destPosition + length"

The actual upper bound in the fix is Math.max(startPosition, destPosition) + length - 1, not destPosition + length. This is contradicted by the diff shown two lines later in the same doc.


Summary

Category Assessment
Bug fix correctness Correct
Off-by-one cleanup Correct bonus fix
Test coverage Good, minor gaps noted
Style/conventions Consistent with codebase
Docs file Should be removed (transient analysis doc)

The core code change is solid. The main ask before merging is dropping the docs/4319-*.md file, and optionally adding a length == 0 test and fixing the misleading comment in forwardShiftModifiedRangeStartsAtSource.

@claude

claude Bot commented May 25, 2026

Copy link
Copy Markdown

Code Review

The bug fix itself is correct and important - a crash after defragPage would replay WAL without bytes [destPosition, startPosition), silently corrupting the bucket page. Good catch.

Fix Analysis

The one-liner change in MutablePage.java:223-224 is sound:

if (length > 0)
  updateModifiedRange(Math.min(startPosition, destPosition), Math.max(startPosition, destPosition) + length - 1);
  • length > 0 guard - correct; avoids passing an inverted range when length is zero (the underlying Binary.move already short-circuits for length == 0).
  • Lower bound Math.min(start, dest) - correct for backward shifts; the written region starts at destPosition.
  • Upper bound Math.max(start, dest) + length - 1 - fixes two bugs at once: the original used destPosition + length (exclusive, off-by-one for updateModifiedRange's inclusive semantics) and the wrong anchor for backward shifts.

The range is slightly conservative - for non-overlapping moves it includes the source region in addition to the written region. That is safe for WAL (recovery replays the wider slice, all bytes are consistent) and the simplicity of min/max is worth the minor extra WAL bytes.


Issues to Address

1. Remove docs/4319-mutablepage-move-wal-tracking.md (blocker)

This file should not be committed to the repository. It is an artifact of the AI-assisted development cycle, not maintenance documentation. Specific problems:

  • The "Review cycles" section records claude bot is not configured in this repository - this is process meta-information that has no value to future readers.
  • It embeds the PR URL, which becomes a dead external link in the source tree.
  • The root-level docs/ directory does not appear to be the project's documentation home; merging this creates an orphaned pattern others may follow.
  • The PR description and commit message already capture everything useful here.

Please drop this file from the PR.

2. Strengthen weak assertions in MutablePageMoveTest (minor)

backwardShiftModifiedRangeCoversWrittenBytes uses isLessThanOrEqualTo / isGreaterThanOrEqualTo where the exact values are deterministic:

// current - too loose, will not catch regressions that are still "in range"
assertThat(range[0]).isLessThanOrEqualTo(50 + BasePage.PAGE_HEADER_SIZE);
assertThat(range[1]).isGreaterThanOrEqualTo(50 + BasePage.PAGE_HEADER_SIZE + 60 - 1);

// preferred - exact contract, catches any drift
assertThat(range[0]).isEqualTo(50 + BasePage.PAGE_HEADER_SIZE);
assertThat(range[1]).isEqualTo(100 + BasePage.PAGE_HEADER_SIZE + 60 - 1); // Math.max(50,100)+60-1

The CLAUDE.md style guide prefers assertThat(x).isEqualTo(...) over inequality checks when the value is known.

3. Misleading test comment in forwardShiftModifiedRangeStartsAtSource (nit)

// dest(100) > start(50): forward shift — written bytes start at destPosition but range from source

The comment is correct but could confuse maintainers: the written bytes start at destPosition (100+H), not at startPosition. The tracked range conservatively starts at startPosition. Consider rewording to make the "conservative range" intent explicit:

// dest(100) > start(50): forward shift — actual writes go to [dest, dest+len-1],
// but updateModifiedRange conservatively starts at startPosition (the read region)

Summary

Bug fix correctness Correct
Off-by-one fix (inclusive upper bound) Correct, good catch
length == 0 guard Correct
Test coverage Good - backward, forward, same-position, boundary cases all covered
Blocking issue docs/4319-mutablepage-move-wal-tracking.md should not be committed
Minor issues Strengthen loose assertions; clarify one test comment

The fix is ready to merge once the documentation file is removed and the test assertions tightened.

@codecov

codecov Bot commented May 25, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 50.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 64.11%. Comparing base (d1b1632) to head (aa361a3).

Files with missing lines Patch % Lines
...src/main/java/com/arcadedb/engine/MutablePage.java 50.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff            @@
##               main    #4325   +/-   ##
=========================================
  Coverage     64.11%   64.11%           
+ Complexity      423      419    -4     
=========================================
  Files          1645     1645           
  Lines        127537   127538    +1     
  Branches      27348    27349    +1     
=========================================
+ Hits          81766    81775    +9     
- Misses        34257    34261    +4     
+ Partials      11514    11502   -12     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@robfrank robfrank merged commit 59349ea into main May 25, 2026
25 of 30 checks passed
@robfrank robfrank deleted the arcadedb-bug-4 branch May 25, 2026 13:31
@robfrank robfrank added this to the 26.6.1 milestone May 27, 2026
@robfrank robfrank self-assigned this May 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MutablePage.move mis-tracks modified range for backward shifts — WAL omits defrag bytes

1 participant