Skip to content

diff: HASH/KEY (incl. LINEAR) PARTITIONS n count changes via ADD PARTITION PARTITIONS / COALESCE PARTITION#52

Merged
winebarrel merged 3 commits into
mainfrom
partition-hash-key-count
May 3, 2026
Merged

diff: HASH/KEY (incl. LINEAR) PARTITIONS n count changes via ADD PARTITION PARTITIONS / COALESCE PARTITION#52
winebarrel merged 3 commits into
mainfrom
partition-hash-key-count

Conversation

@winebarrel

Copy link
Copy Markdown
Owner

Summary

Builds on PR #50's RANGE/LIST suffix-add + subset-drop work. HASH / KEY (incl. LINEAR HASH and LINEAR KEY, plus KEY ALGORITHM=N) now produce real DDL when the partition strategy is unchanged and only PARTITIONS n differs:

  • count growALTER TABLE … ADD PARTITION PARTITIONS n (n = desired - current). No data loss; new slots are empty until the next rebalance.
  • count shrinkALTER TABLE … COALESCE PARTITION n (n = current - desired). MySQL rebalances rows from the dropped slots into the survivors but the slots themselves go away — gated on --allow-drop=partition like RANGE/LIST DROP. Without the flag the COALESCE lands on the disallowed bucket as a -- skipped: line.

Equal counts continue to no-op (round 4); a header mismatch (Type / IsLinear / KeyAlgorithm / ColList / Expr) still falls through to the upstream "scheme/expression differs" error.

Behaviour matrix (delta from PR #50)

Scenario Result
HASH/KEY: scheme equal + count grow ALTER TABLE … ADD PARTITION PARTITIONS n
HASH/KEY: scheme equal + count shrink + --allow-drop=partition ALTER TABLE … COALESCE PARTITION n
HASH/KEY: scheme equal + count shrink, no --allow-drop=partition -- skipped: ALTER TABLE … COALESCE PARTITION n
HASH/KEY: scheme equal + count equal (incl. identifier-casing-only diff) no-op
HASH/KEY: scheme mismatch (e.g. KEY (a)KEY (b)) error: partition strategy / expression differs

Implementation

  • diff/partitions.go's HASH/KEY branch (which previously returned the unsupported-diff error) now emits the count-change statements. partitionHeaderEqual already covers LINEAR / KEY ALGORITHM via its IsLinear and KeyAlgorithm fields, so no new normalisation is needed.

Tests

  • 5 plan fixtures: HASH grow, HASH shrink with allow_drop, HASH shrink disallowed, KEY grow, LINEAR HASH grow.
  • 2 apply fixtures with verify_no_drift: true: HASH grow, HASH shrink. Pin that the generated DDL runs on MySQL and the next plan reports no drift.
  • 2 diff unit tests: KEY grow (ADD PARTITION PARTITIONS), KEY shrink with vs. without allow-drop=partition.
  • Replaced the old TestDiffPartitionsKEYCountChangeStillErrors unit test and testdata/plan/partition_hash_diff_errors.yml plan fixture (both pinned the now-obsolete unsupported-diff error).

Docs

  • CAVEATS.md "Partitioning" moves the HASH/KEY count change bullet from "Diffs that still error" to "Diffs that are generated", with the allow_drop gating noted.
  • AGENTS.md "In scope" lists the new shape; "Not yet implemented" entry no longer mentions HASH/KEY.
  • TODO.md entry rewritten — what's left is REORGANIZE PARTITION (mid-list value changes), scheme/expression changes, and adding/removing partitioning entirely.

Test plan

  • make lint
  • make test (8.0)

…/KEY count changes

Builds on PR #50's RANGE/LIST suffix add + subset drop. HASH /
KEY (incl. LINEAR HASH and LINEAR KEY, as well as KEY
ALGORITHM=N) now produce real DDL when the partition strategy
is unchanged and only `PARTITIONS n` differs:

- count grow → `ALTER TABLE … ADD PARTITION PARTITIONS n`
  (n = desired - current). No data loss; new slots are empty
  until the next rebalance.
- count shrink → `ALTER TABLE … COALESCE PARTITION n`
  (n = current - desired). MySQL rebalances rows from the
  dropped slots into the survivors, but the slots themselves
  go away — gated on `--allow-drop=partition` like RANGE/LIST
  DROP. Without the flag the COALESCE lands on the disallowed
  bucket as a `-- skipped:` line.

Equal counts already returned no-op (round 4); scheme mismatch
(Type / IsLinear / KeyAlgorithm / ColList / Expr) still falls
through to "scheme/expression differs" upstream.

Implementation: diff/partitions.go's HASH/KEY branch (which
previously returned the unsupported-diff error) now emits the
two count statements. partitionHeaderEqual already covers
LINEAR / KEY ALGORITHM via its IsLinear and KeyAlgorithm
fields, so no new normalisation is needed.

Tests:
- 5 plan fixtures: HASH grow, HASH shrink with allow_drop,
  HASH shrink disallowed, KEY grow, LINEAR HASH grow.
- 2 apply fixtures with verify_no_drift: HASH grow, HASH
  shrink. Pin that the generated DDL runs on MySQL and the
  next plan reports no drift.
- 2 diff unit tests: KEY grow (ADD PARTITION PARTITIONS),
  KEY shrink with vs. without allow-drop=partition.
- Replaced TestDiffPartitionsKEYCountChangeStillErrors (which
  pinned the old error) with the grow / shrink pair above.
- Removed testdata/plan/partition_hash_diff_errors.yml (same
  reason — count-change is no longer an error).

Docs:
- CAVEATS.md "Partitioning" moves the HASH/KEY count change
  from "Diffs that still error" to "Diffs that *are*
  generated".
- AGENTS.md "In scope" lists the count-change shape; the
  "Not yet implemented" entry no longer mentions HASH/KEY.
- TODO.md entry updated to reflect what's left (REORGANIZE,
  scheme change, add/remove partitioning).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings May 3, 2026 15:40
@codecov

codecov Bot commented May 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 72.74%. Comparing base (05e4227) to head (2a1eb42).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #52      +/-   ##
==========================================
+ Coverage   72.67%   72.74%   +0.06%     
==========================================
  Files          28       28              
  Lines        2734     2741       +7     
==========================================
+ Hits         1987     1994       +7     
  Misses        541      541              
  Partials      206      206              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Extends myschema’s partition diff generation to handle HASH/KEY (including LINEAR) partition count-only changes by emitting MySQL’s dedicated DDL, aligning HASH/KEY support with the already-supported RANGE/LIST partition diff shapes.

Changes:

  • Generate ALTER TABLE … ADD PARTITION PARTITIONS n for HASH/KEY count growth and ALTER TABLE … COALESCE PARTITION n for count shrink (shrink gated by --allow-drop=partition).
  • Update/replace unit tests and add new plan/apply fixtures to cover HASH/KEY/LINEAR HASH count changes and allow-drop behavior.
  • Refresh documentation (CAVEATS/AGENTS/TODO) to reflect the newly supported HASH/KEY count-change diff shape.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
diff/partitions.go Implements HASH/KEY count grow/shrink DDL generation with allow-drop gating for shrink.
diff/partitions_test.go Replaces the old “still errors” test with grow/shrink coverage (incl. allow-drop gating).
testdata/plan/partition_hash_count_grow.yml New plan fixture for HASH count growth.
testdata/plan/partition_hash_count_shrink_with_allow.yml New plan fixture for HASH count shrink when --allow-drop=partition is set.
testdata/plan/partition_hash_count_shrink_disallowed.yml New plan fixture verifying COALESCE is suppressed into disallowed_drops when drop policy disallows it.
testdata/plan/partition_key_count_grow.yml New plan fixture for KEY count growth.
testdata/plan/partition_linear_hash_count_grow.yml New plan fixture for LINEAR HASH count growth.
testdata/apply/partition_hash_count_grow.yml New apply fixture validating the emitted grow DDL executes and converges (no drift).
testdata/apply/partition_hash_count_shrink.yml New apply fixture validating the emitted shrink DDL executes and converges (no drift).
testdata/plan/partition_hash_diff_errors.yml Removes obsolete fixture that pinned the prior “HASH/KEY diffs not generated” behavior.
CAVEATS.md Documents HASH/KEY count-change DDL generation and allow-drop gating for shrink.
AGENTS.md Updates stated supported scope to include HASH/KEY count changes.
TODO.md Updates remaining partition-diff work items to reflect HASH/KEY count changes are now implemented.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread CAVEATS.md Outdated
Comment thread diff/partitions.go Outdated
Comment thread diff/partitions.go Outdated
Comment thread diff/partitions_test.go Outdated
Comment thread diff/partitions.go Outdated
Comment thread testdata/plan/partition_hash_count_grow.yml Outdated
…ge wording

Six wording nits, all my MySQL-knowledge mistake. Increasing
PARTITIONS changes the hash modulus, so MySQL has to
redistribute existing rows during the ALTER — new partitions
do NOT stay empty. COALESCE PARTITION likewise merges
partitions and redistributes rows into the survivors; it
does NOT discard rows. So:

- HASH/KEY grow: data-moving (rows redistributed across the
  new modulus), but no row loss.
- HASH/KEY shrink: data-moving (rows merged into the
  survivors), but no row loss.

The `--allow-drop=partition` gating on COALESCE stays — the
slot structure changes irreversibly and the operation is
operationally heavy / expensive — but the justification is
"destructive at the slot level" rather than "destructive at
the row level". Reword every comment / doc bullet that
implied row loss or empty new partitions:

- CAVEATS.md "Partitioning" HASH/KEY count change bullet
  rewritten to spell out the data-moving behaviour and the
  irreversibility-based gating reason.
- diff/partitions.go: docblock summary, the inline branch
  comment, and the gating-justification line all reworded.
- diff/partitions_test.go: KEY shrink test comment.
- testdata/plan/partition_hash_count_grow.yml and
  partition_hash_count_shrink_with_allow.yml fixture headers.

No implementation change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

I confirmed the COALESCE / ADD PARTITION PARTITIONS docs against
the MySQL 8.0 manual. The round-2 wording correctly noted both
operations are row-preserving and data-moving, but it described
the cost as if the table always uses regular HASH/KEY where the
modulus changes and most rows shift partition. That overstates
the cost for LINEAR HASH / LINEAR KEY users.

Per dev.mysql.com/doc/refman/8.0/en/partitioning-linear-hash.html
the linear-powers-of-two algorithm makes "adding, dropping,
merging, and splitting of partitions … much faster" because only
the partitions adjacent to the change need to be touched. That's
the headline reason for picking LINEAR over regular at table-
creation time, so the docs shouldn't read as if myschema
collapses both into the same heavyweight rewrite.

CAVEATS.md "Partitioning" HASH/KEY count change bullet, the
diff/partitions.go docblock and inline branch comment, and the
two HASH fixture headers all now spell out:

- both directions are row-preserving but data-moving;
- regular HASH/KEY rewrites most of the table (modulus shift);
- LINEAR HASH/KEY touches only the adjacent partitions and is
  "much faster" per the manual;
- the `--allow-drop=partition` gating on COALESCE is about the
  slot structure changing irreversibly, not about row loss.

No implementation change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@winebarrel winebarrel enabled auto-merge May 3, 2026 15:54
@winebarrel winebarrel merged commit 9004a12 into main May 3, 2026
3 checks passed
@winebarrel winebarrel deleted the partition-hash-key-count branch May 3, 2026 15:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants