Skip to content

diff: generate ADD/DROP PARTITION for RANGE/LIST suffix changes#50

Merged
winebarrel merged 19 commits into
mainfrom
partition-diff-range-list
May 3, 2026
Merged

diff: generate ADD/DROP PARTITION for RANGE/LIST suffix changes#50
winebarrel merged 19 commits into
mainfrom
partition-diff-range-list

Conversation

@winebarrel

@winebarrel winebarrel commented May 3, 2026

Copy link
Copy Markdown
Owner

Summary

Builds on PR #49's round-trip support. RANGE / LIST partitions now produce real ALTER TABLE … ADD PARTITION / DROP PARTITION for the two operational patterns that motivated declarative partition management:

  • Suffix add: roll the next month's / year's partition out → ALTER TABLE … ADD PARTITION (PARTITION p VALUES …).
  • Order-preserving subset drop: any combination of head / middle / tail removals (the typical retention workflow drops the oldest partition, which sits at the head of the list) → ALTER TABLE … DROP PARTITION p1, p2. Gated by --allow-drop=partition. Without that flag the DROP lands in the disallowed bucket as a -- skipped: line so the user sees what would have been removed.

Anything else still errors out (manage by hand) — value changes / interior inserts / reorders / merges / splits / retention-roll-forward (REORGANIZE territory; the diff can't tell those apart safely from names alone), HASH/KEY count operations, scheme/expression changes, adding/removing partitioning entirely, SUBPARTITION, and any unique key on a partitioned table that doesn't cover all the partition columns. Each error string names the specific shape so users know which manual ALTER to run.

Behaviour matrix

Scenario Result
both sides nil / byte-equal no-op
RANGE/LIST: catalog ⊂ desired (suffix add) ALTER TABLE … ADD PARTITION (...)
RANGE/LIST: desired matches an order-preserving subset of catalog (head / middle / tail drop, or any combination) + --allow-drop=partition ALTER TABLE … DROP PARTITION ...
Same DROP scenario, no --allow-drop=partition -- skipped: ALTER TABLE … DROP PARTITION ...
RANGE/LIST: drops AND adds both non-empty (any shape — value change, interior insert, reorder, merge / split with new names, retention roll-forward) error: REORGANIZE PARTITION generation is not yet implemented
HASH/KEY: any diff error: HASH / KEY partition diffs are not yet generated
Strategy / expression change (e.g. RANGE → HASH) error: partition strategy / expression differs
desired adds partitioning to an unpartitioned catalog table error: first-time ALTER TABLE … PARTITION BY … is not yet generated
desired drops partitioning from a partitioned catalog table error: ALTER TABLE … REMOVE PARTITIONING is not yet generated
Column rename of a partition-key column (a valid CREATE TABLE necessarily updates the desired-side PARTITION BY to use the new name) error: partition strategy / expression differs (renamed expression looks like a scheme change to the diff)
Desired-side PRIMARY KEY / UNIQUE INDEX missing partition columns (existing or brand-new partitioned table; covers the "drop partition column" case too because the desired CREATE TABLE wouldn't parse on real MySQL otherwise) error: … is missing partition column(s) [...]
SUBPARTITION on either side parser / catalog reject earlier (PR #49)

Implementation

  • parser/partition.go::ParsePartitionClause re-parses a normalised partition clause back into a *sqlparser.PartitionOption. FormatPartitionDefinition renders one definition; NormalizePartitionOption and FormatPartitionDefinition both nil out def.Options.Engine at the AST level so per-partition ENGINE = ... doesn't leak into the canonical form (and string literals containing the word engine aren't mangled by a regex).
  • diff/partitions.go (new): byte-compare fast path; on mismatch re-parse both sides, check the header (Type / IsLinear / KeyAlgorithm / ColList / Expr), short-circuit HASH/KEY (with a Partitions-count no-op path so identifier-casing-only diffs don't false-positive). Definitions go through partitionByNameOrderPreserving: walk current once, match each entry against the next-unconsumed desired entry, drop on mismatch, treat anything left in desired as a suffix add. Both empty → no-op; pure ADD or pure DROP → emitted; both non-empty → REORGANIZE error (unconditionally — disjoint-name checks aren't enough to tell merge/split/replace apart from a "discard rows" retention pair, and a plain DROP+ADD on the wrong shape would silently lose data).
  • diff/rename.go::partitionRequiredColumns returns the columns the partition expression references. diff/tables.go::uniqueKeyPartitionCoverGap then checks whether desired's PRIMARY KEY / UNIQUE INDEX covers it. The cover guard runs on both the new-tables loop in DiffTables and inside diffTable after diffPartitions (so more-actionable partition errors take precedence). It catches every valid-DDL "drop a partition-key column" shape too: if the column is missing from desired, the desired PRIMARY KEY / UNIQUE INDEX necessarily can't reference it either, so the cover gap fires first.
  • options.go::DropPolicy.AllowDrop enum gains partition.

Tests

Plan and apply fixtures cover suffix add, multi add, RANGE COLUMNS, plain LIST, LIST COLUMNS, head drop, middle drop, multi drop, drop disallowed, alter-hint splice, MAXVALUE catch-all rejection, partitioning add / remove / scheme errors, the partition-column rename "implicit scheme change" path (RANGE expression + KEY + LIST COLUMNS variants), brand-new partitioned table cover gap, the cover-gap rejection on diff path, and the unconditional both-ADD-and-DROP rejection (disjoint-names case included). Diff unit tests pin KEY no-op across identifier casing, count-change still erroring, ADD / DROP partition reserved-word-name quoting, and partition-header equality preserving string-literal case. Parser unit test pins LIST literal containing the substring engine ('foo engine innodb') round-tripping whole. Options test pins kong CLI parsing --allow-drop=partition and rejecting unknown tokens.

Docs

  • CAVEATS.md "Partitioning" rewritten: title now reflects "suffix add + subset drop" scope, "what's generated" up front, "what still errors" listed by category with the exact error string for each (including the explicit note that the cover guard fires regardless of --allow-drop and absorbs the partition-key drop case for any valid CREATE TABLE), manual workaround for the unsupported cases.
  • AGENTS.md "In scope" lists RANGE / LIST suffix add + order-preserving subset DROP and the expanded --allow-drop enum; "Not yet implemented" entry scoped down to the remaining cases.
  • TODO.md entry rewritten to enumerate what's left.

Test plan

  • make lint
  • make test (8.0)

Builds on PR #49's round-trip support. RANGE and LIST partitions
now produce real ALTER TABLE statements when the desired side is
a strict suffix-extension or suffix-trim of the catalog —
covering the two operational patterns that motivated declarative
partition management in the first place:

  - Roll the next month's / year's partition out:
    `ALTER TABLE … ADD PARTITION (PARTITION p VALUES …)`.
  - Drop the oldest partition: `ALTER TABLE … DROP PARTITION p1`,
    gated by `--allow-drop=partition` (otherwise the DROP lands
    in the disallowed bucket as `-- skipped: …`).

Anything else still errors out (manage by hand) — mid-list
value changes, HASH/KEY count operations, strategy / expression
changes, adding or removing partitioning entirely, SUBPARTITION.
The error messages name the specific shape so users know which
manual ALTER to run.

Implementation:

- `parser/partition.go`: `ParsePartitionClause` re-parses a
  normalised partition clause (the form stored on
  `model.Table.Partition`) back into a
  `*sqlparser.PartitionOption` so the diff layer can inspect
  individual definitions; `FormatPartitionDefinition` renders a
  single `*sqlparser.PartitionDefinition` for ADD PARTITION
  output, with the same per-partition `engine` strip
  NormalizePartitionOption performs at clause level.
- `diff/partitions.go` (new): `diffPartitions` returns
  (stmts, disallowed, err). Both sides are first byte-compared
  (cheap fast path for unchanged partitioned tables), then re-
  parsed only when they differ. The header — Type, IsLinear,
  KeyAlgorithm, ColList, Expr — must match for any diff to be
  generated; HASH/KEY tables short-circuit to error. RANGE/LIST
  definitions are then compared positionally: the longest
  common prefix determines the boundary, anything past it is
  either a suffix add (desired only) or a suffix drop (current
  only); both tails non-empty → mid-list mismatch error.
- `diff/tables.go::diffTable` now calls diffPartitions in place
  of the previous `partition definition differs` hard error.
- `options.go::DropPolicy.AllowDrop` enum gains `partition`.

Tests:
- 7 plan fixtures (add_range, add_list, drop_with_allow,
  drop_disallowed, mid_mismatch_errors, hash_diff_errors,
  scheme_change_errors) plus 2 apply fixtures with
  verify_no_drift: true (add_range, drop_range) — the apply
  pair pins that the generated ALTERs actually run on MySQL
  and a follow-up plan reports no drift.

Docs:
- CAVEATS.md "Partitioning" section rewritten: "what's
  generated" up front, "what still errors" listed by category
  with the exact error string for each, manual workaround for
  the unsupported cases.
- AGENTS.md "In scope" lists RANGE/LIST suffix add/drop and the
  expanded `--allow-drop` enum; "Not yet implemented" entry
  scoped down to the cases that still error.
- TODO.md entry rewritten to enumerate what's left
  (mid-list / HASH-KEY count / scheme change / first-time
  partitioning).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings May 3, 2026 11:26
@codecov

codecov Bot commented May 3, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 38.37209% with 106 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.67%. Comparing base (a19fe7b) to head (f28ee66).
⚠️ Report is 23 commits behind head on main.

Files with missing lines Patch % Lines
diff/partitions.go 63.33% 16 Missing and 17 partials ⚠️
diff/tables.go 8.33% 30 Missing and 3 partials ⚠️
diff/rename.go 0.00% 20 Missing ⚠️
parser/partition.go 23.07% 20 Missing ⚠️

❌ Your patch status has failed because the patch coverage (38.37%) is below the target coverage (50.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #50      +/-   ##
==========================================
- Coverage   73.71%   72.67%   -1.04%     
==========================================
  Files          29       28       -1     
  Lines        2610     2734     +124     
==========================================
+ Hits         1924     1987      +63     
- Misses        499      541      +42     
- Partials      187      206      +19     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the partitioning work from the earlier round-trip/drift-detect support by generating real partition DDL for the safe subset of partition changes: suffix adds and suffix drops on RANGE/LIST partitioned tables. It fits into the schema-diff engine by replacing the old “any partition mismatch is an error” behavior with targeted ALTER TABLE ... ADD/DROP PARTITION generation plus updated drop-policy handling and docs/tests.

Changes:

  • Add partition-diff generation for RANGE/LIST suffix add/drop, with unsupported partition changes still failing fast with specific errors.
  • Extend --allow-drop to support partition, routing disallowed partition drops into the skipped/disallowed bucket.
  • Add plan/apply fixtures and update docs to describe the new supported partitioning scope.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
testdata/plan/partition_scheme_change_errors.yml Adds fixture for scheme/expression-change error behavior.
testdata/plan/partition_mid_mismatch_errors.yml Adds fixture for mid-list partition mismatch error behavior.
testdata/plan/partition_hash_diff_errors.yml Adds fixture for unsupported HASH partition diff errors.
testdata/plan/partition_drop_with_allow.yml Adds plan fixture for allowed partition drops.
testdata/plan/partition_drop_disallowed.yml Adds plan fixture for suppressed partition drops.
testdata/plan/partition_add_range.yml Updates RANGE suffix-add plan fixture to expect generated ALTER.
testdata/plan/partition_add_list.yml Adds LIST suffix-add plan fixture.
testdata/apply/partition_drop_range.yml Adds end-to-end apply coverage for RANGE partition drop.
testdata/apply/partition_add_range.yml Adds end-to-end apply coverage for RANGE partition add.
parser/partition.go Adds helpers to reparse partition clauses and format individual definitions.
options.go Extends drop-policy enum/help text with partition.
diff/tables.go Wires partition diff generation into table diffing.
diff/partitions.go Implements partition diff analysis and ADD/DROP generation.
TODO.md Narrows remaining partition-diff TODOs to unsupported cases.
CAVEATS.md Rewrites partitioning caveats for the new supported scope.
AGENTS.md Updates contributor-facing scope/docs for partition diff generation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread diff/partitions.go Outdated
Comment thread diff/partitions.go Outdated
Comment thread CAVEATS.md Outdated
Three nits — two real bugs, one doc typo:

- formatExprNormalised lower-cased the entire serialised
  expression including string literals, so a real expression
  change like `WHEN region='US'` ↔ `WHEN region='us'` would
  silently compare equal and skip the strategy/expression
  error. Both AST trees were already identifier-normalised
  by parser.NormalizePartitionOption (lower-cased ColName /
  FuncExpr) before re-parsing, so the second normalisation
  pass was both redundant and wrong. Drop strings.ToLower and
  rename the helper formatExpr; pin with a regression test
  using a CASE-WHEN string literal.
- DROP PARTITION emitted partition names with d.Name.String()
  directly, with no quoting. A partition named with a MySQL
  reserved word (or any non-safe identifier) would produce
  invalid SQL even though the parser accepted it and the ADD
  PARTITION path round-tripped it correctly. Quote through
  model.Ident; pin with a regression test using a partition
  literally named `select`.
- CAVEATS workaround text said `ALTER TABLE … {REORGANIZE|
  COALESCE|REMOVE} PARTITION`, but MySQL's syntax is REMOVE
  PARTITIONING (not PARTITION). Rewrite the workaround
  paragraph to spell each ALTER out separately so the
  documented fallback commands are syntactically correct for
  the cases this section is trying to help with.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 5 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread AGENTS.md Outdated
Comment thread diff/partitions.go Outdated
Comment thread CAVEATS.md Outdated
Comment thread AGENTS.md Outdated
Comment thread TODO.md Outdated
Five doc / coverage nits — no implementation change:

- CAVEATS.md "Partitioning" section title and "Diffs that are
  generated" intro now spell out that the supported set
  includes the `RANGE COLUMNS` and `LIST COLUMNS` variants;
  diffPartitions has supported them all along (header equality
  flows through ColList) but the doc only listed plain
  RANGE / LIST.
- Same fix on AGENTS.md "In scope" — Diff bullet now
  advertises RANGE / LIST + the COLUMNS variants instead of
  just RANGE/LIST.
- CAVEATS.md and TODO.md "adding/removing partitioning"
  bullets now mention both directions: adding partitioning to
  an unpartitioned catalog table *and* removing it from an
  already-partitioned one. The diff still rejects both; only
  the docs were one-sided.
- AGENTS.md "Not yet implemented" partition entry mirrors the
  same both-directions fix.
- Add a LIST-flavoured apply fixture
  (`testdata/apply/partition_add_list.yml`) so the LIST ADD
  path is exercised end-to-end against MySQL — previously only
  RANGE had an apply fixture and LIST coverage stopped at the
  planned-SQL string assertion.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread diff/partitions_test.go
Comment thread diff/partitions.go
KEY (and HASH) partitions had no semantic no-op path beyond the
raw-string fast path. partitionHeaderEqual lower-cases ColList,
so two semantically-equal KEY clauses that differ only in
identifier casing (`partition by key (UserID)` vs
`partition by key (userid)`) survived as "headers equal" and
then the unconditional HASH/KEY error path returned the
unsupported-diff error instead of no changes.

Add an equal-`Partitions`-count check before the error: same
header + same count = no-op, different count still errors so
the user knows to run COALESCE / ADD PARTITION PARTITIONS by
hand.

Two new diff tests pin the behaviour: no-op across identifier
casing, and the count-change path still raising the
unsupported-diff error.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread diff/tables.go
Comment thread diff/partitions.go Outdated
Comment thread diff/partitions.go Outdated
Three plan fixtures pin format paths the existing partition
fixtures left uncovered:

- partition_add_range_multi: ADD PARTITION with two new
  trailing partitions exercises the comma+newline join in the
  builder loop (previous fixtures only added one partition, so
  the loop never ran past i==0).
- partition_drop_with_allow_multi: DROP PARTITION trims two
  trailing partitions so the comma-joined name list is
  exercised end-to-end.
- partition_add_range_with_alter_hints: ADD PARTITION carried
  through the global appendAlterHints rewrite. The hint helper
  splices `, ALGORITHM=…, LOCK=…` just before the trailing `;`
  at the string level after all per-table buckets are merged;
  without an explicit fixture, a future format tweak to either
  the ADD PARTITION builder or appendAlterHints could quietly
  break the splice point and ship invalid SQL to MySQL.

No implementation change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread diff/partitions.go
Comment thread diff/partitions.go Outdated
Comment thread CAVEATS.md Outdated
Three coverage / doc nits — no implementation change:

- Add plan fixtures for plain `LIST (col)` and `RANGE COLUMNS(...)`.
  diffPartitions has supported them all along (same code path as
  the cases already covered) but the existing fixtures only hit
  RANGE and `LIST COLUMNS`, leaving two of the advertised shapes
  exercised only by the round-trip path.
- Add an apply fixture for the multi-partition ADD path. Plan
  fixtures pin the format string but only an apply against MySQL
  proves the comma+newline-joined definition list is accepted by
  the server. verify_no_drift: true confirms one-shot
  convergence.
- CAVEATS "Suffix add" example now flags the MAXVALUE / DEFAULT
  catch-all caveat: when the live table ends in a catch-all
  partition, adding the next real partition is a mid-list
  change (the new partition would land before the catch-all)
  and the diff falls through to the REORGANIZE error rather
  than emitting ADD PARTITION. Documented workaround: drop the
  catch-all first or run REORGANIZE PARTITION by hand.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 24 out of 24 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread diff/tables.go Outdated
Comment thread diff/partitions.go Outdated
Comment thread options.go
… coverage gaps

Two real bugs from the Copilot pass plus the coverage holes the
post-fix audit surfaced. Detail:

Bug fixes:

- Column rename + partitioned table no longer surfaces a spurious
  "partition strategy / expression differs" error. The column-
  rename pass on `current` now also rewrites `current.Partition`
  via a new rewritePartitionColumnRefs helper that re-parses the
  clause, walks the AST replacing ColName.Name and ColList
  entries from the rename map (case-insensitive lookup so it
  matches partitionHeaderEqual's ColList normalisation), then
  re-formats through NormalizePartitionOption. If re-parse fails
  the field is left untouched.
- diffPartitions now reports the right "manage by hand" remediation
  for each direction of the nil/non-nil mismatch:
    - desired adds partitioning to an unpartitioned catalog table
      → "first-time ALTER TABLE … PARTITION BY … is not yet
      generated"
    - desired drops partitioning from a partitioned catalog table
      → "ALTER TABLE … REMOVE PARTITIONING is not yet generated"
  Previously both directions returned the same message, so one
  branch always sent users at the inverse operation.

Coverage gaps (caught while auditing the partition path before
Copilot saw it):

- options_test.go: kong CLI parse coverage for `--allow-drop`.
  Asserts every supported token (including the new `partition`)
  parses cleanly through DropPolicy and that an unknown token is
  rejected — the partition fixtures populate AllowDrop directly so
  a typo in the kong enum tag would otherwise slip past CI.
- testdata/plan/partition_no_diff_with_column_rename.yml: pin the
  bug fix above end-to-end (RENAME COLUMN + matching partition key
  rewritten, no scheme-error).
- testdata/plan/partition_add_partitioning_errors.yml: pin the
  "desired adds partitioning" error message direction.
- testdata/plan/partition_remove_partitioning_errors.yml: pin the
  inverse "desired removes partitioning" error message direction.
- testdata/apply/partition_drop_range_multi.yml: end-to-end pin
  for the multi-DROP path (plan-only fixture pinned the format,
  this proves MySQL accepts the comma-joined name list).
- diff/partitions_test.go: ADD PARTITION reserved-word name pin
  (`PARTITION `select` VALUES …`). DROP was already pinned in
  round 2; ADD relied on vitess's formatter, now explicit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@winebarrel winebarrel requested a review from Copilot May 3, 2026 12:43

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated 5 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread testdata/plan/partition_add_range_columns.yml
Comment thread testdata/plan/partition_add_list_plain.yml
Comment thread CAVEATS.md Outdated
Comment thread diff/partitions.go Outdated
Comment thread testdata/plan/partition_no_diff_with_column_rename.yml Outdated
Five test/docs nits, plus a workaround for a MySQL-side
limitation surfaced while writing one of them.

Apply-side coverage extensions:

- partition_add_range_columns apply fixture pins MySQL acceptance
  of multi-column tuple `VALUES LESS THAN (10, 100)` in ADD
  PARTITION. Plan-only fixture pinned the format string; this
  proves the server takes the result.
- partition_add_list_plain apply fixture does the same for the
  non-COLUMNS LIST shape (`PARTITION BY LIST (col)` with
  `VALUES IN (...)`).
- partition_no_diff_with_column_rename apply fixture exercises
  the column-rename pass on a partitioned table end-to-end. Note
  the column being renamed here is *not* the partition key —
  MySQL itself rejects renaming a column the partition expression
  depends on (Error 3855: "Column ... has a partitioning function
  dependency and cannot be dropped or renamed"), so the
  partition-key rewrite path stays plan-side only via the existing
  partition_no_diff_with_column_rename plan fixture. The apply
  fixture instead pins that rewritePartitionColumnRefs is a no-op
  (the renamed column isn't in the partition clause) and the rest
  of the table-diff pipeline emits a clean RENAME COLUMN that
  apply runs and verify_no_drift confirms.

Docs / message clarity:

- The "scheme / expression differs" remediation in diff/partitions
  used to suggest `REMOVE PARTITIONING + PARTITION BY` as if it
  were one statement. Spell it out as two separate ALTERs so the
  user sees exactly what to run by hand.
- CAVEATS "Adding or removing partitioning entirely" bullet
  documented the old single-message error string. The
  implementation now returns two direction-specific messages
  (round 7); update the doc to list both so users can grep for
  the message they actually see.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@winebarrel winebarrel requested a review from Copilot May 3, 2026 12:54

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 45 out of 45 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread testdata/apply/partition_no_diff_with_non_partition_column_rename.yml Outdated
Comment thread testdata/plan/partition_key_drop_conflict_errors.yml Outdated
Comment thread diff/tables.go Outdated
Comment thread CAVEATS.md Outdated
Two real bugs + two doc/comment refs:

- Restore the partition-key rename guard, but only for the
  reachable case: when the desired-side `PARTITION BY` clause
  is byte-equal to the catalog's. Round 13 dropped the rename
  guard entirely on the assumption that any partition-key
  rename also changes the partition expression — but that
  only holds when the user updates the desired clause to use
  the new name. Forgetting to update the clause leaves
  diffPartitions a no-op while the column-rename pass still
  emits `RENAME COLUMN`, which MySQL then rejects with Error
  3855. The guard now catches that edge case at plan time;
  the existing implicit-scheme-change fixtures still cover the
  other (clause-updated) shape.
- New plan fixture `partition_key_rename_clause_unchanged_errors`
  pins the rename-guard reachable case (catalog and desired
  partition clauses both still reference `old_dt` while the
  column itself is renamed `old_dt → dt`).

Doc cleanup:

- Apply fixture comment for the partitioned non-partition-column
  rename now references `partitionColumnConflicts` (the actual
  helper) and points at both the implicit-scheme-change and
  rename-clause-unchanged plan fixtures.
- Plan fixture `partition_key_drop_conflict_errors` header now
  cross-references the rename-side counterpart by its current
  name (round 13 renamed it).
- PR description's behaviour matrix split the rename row into
  two cases (clause updated vs unchanged) so the documented
  errors match what diffTable actually emits.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 46 out of 46 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread diff/tables.go Outdated
Comment thread diff/partitions.go Outdated
Two real bugs:

- Add a unique-key partition-cover guard. MySQL requires every
  unique key (including PRIMARY KEY) on a partitioned table to
  include all columns in the partitioning function — a desired
  schema that omits a partition column from PRIMARY KEY (or
  any UNIQUE INDEX) used to slip past myschema and emit ADD
  PRIMARY KEY / ADD UNIQUE INDEX statements MySQL then
  rejected. Add `partitionRequiredColumns` (re-uses the
  partition-clause walker) and `uniqueKeyPartitionCoverGap`
  (PRIMARY KEY is modelled as `Index{Primary: true}` so one
  walk over `desired.Indexes` covers PK + UNIQUE), call them
  from diffTable after diffPartitions so more-specific
  partition errors take precedence, only when desired stays
  partitioned.
- Loosen the both-ADD-and-DROP error in diffPartitions to the
  cases that actually need REORGANIZE: when the drop and add
  name sets *intersect* (i.e. the same partition name appears
  on both sides — a value change, an interior insert in front
  of a catch-all, etc.) the user has to preserve data with
  REORGANIZE. When the sets are disjoint (the canonical
  retention-roll-forward `[p2020, p2021] -> [p2021, p2022]`
  shape) it's fine to emit DROP and ADD as two ordinary
  statements; MySQL runs them in order. Pinned with new plan
  + apply fixtures (`partition_drop_and_add_retention`).

Fixtures:
- New `partition_unique_key_misses_partition_col_errors` pins
  the cover guard above (renamed from `partition_key_drop_disallowed`,
  whose old shape was made unreachable by the new guard).
- New `partition_drop_and_add_retention` (plan + apply) pins
  the disjoint-name case generating a clean DROP + ADD pair.

Docs:
- CAVEATS "Partitioning" entry adds a "Retention roll-forward"
  bullet under generated diffs and a new
  "PRIMARY KEY / UNIQUE INDEX missing partition column(s)"
  entry under unsupported diffs. The both-ADD-and-DROP
  unsupported entry is reworded to "same partition appears in
  both ADD and DROP" so it matches the new logic.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 48 out of 48 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread diff/tables.go
Comment thread diff/tables.go Outdated
Two real bugs:

- Run the unique-key partition-cover guard on the create-table
  path too. The previous implementation only fired inside
  diffTable, so a brand-new partitioned table whose PRIMARY /
  UNIQUE key omits a partition column would still emit a
  CREATE TABLE that MySQL rejects at apply time. Add the same
  partitionRequiredColumns + uniqueKeyPartitionCoverGap check
  to the new-tables loop in DiffTables. Pinned with
  `partition_create_table_unique_key_misses_partition_col_errors`.
- Loosen the rename-conflict gate from byte-equality of the
  partition clause to "same partition scheme/expression". The
  previous gate was over-restrictive: a partition-key column
  rename combined with a supported RANGE/LIST suffix
  ADD/DROP produces matching headers but byte-different
  clauses, so diffPartitions returned ADD/DROP statements
  while the rename guard skipped — and the column-rename pass
  emitted a RENAME COLUMN MySQL would reject with Error 3855.
  New helper `partitionExpressionsEqual` re-parses both sides
  and runs `partitionHeaderEqual`; the rename and drop guards
  in diffTable now key off that. Pinned with
  `partition_key_rename_with_suffix_add_errors`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 50 out of 50 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread diff/tables.go
The unique-key cover guard added in round 15 fires regardless of
`--allow-drop=column`, by design — the desired-side schema
itself is invalid for a partitioned table (PRIMARY KEY no longer
covers the partition expression), so a half-applied state where
PK shrinks while MySQL still has the partition column would only
fail on the next ALTER. But the PR description and CAVEATS still
documented the old behaviour where the partition-key column drop
without `--allow-drop=column` "lands on the disallowed bucket;
plan does NOT abort". That promise contradicts what diffTable
now does.

This commit reconciles the docs to match the implementation:

- CAVEATS "PRIMARY KEY / UNIQUE INDEX missing partition
  column(s)" entry now explicitly notes that the guard runs on
  both the diff path and the create-table path, and that
  `--allow-drop` does not bypass it. Spells out why (a
  partial apply with PK shrunk but column still present is
  exactly the state MySQL would reject on the next ALTER).
- PR description's behaviour matrix splits the partition-key
  drop case so users can see what to expect with vs. without
  `--allow-drop=column`, and adds a new row for the
  desired-side PRIMARY KEY / UNIQUE INDEX cover gap (covering
  brand-new partitioned tables too).

No implementation change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 50 out of 50 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread diff/partitions.go Outdated
Revert round 15's "disjoint-name DROP+ADD pair" relaxation. It
was unsafe: name disjointness alone can't tell a retention
roll-forward (`[p2020, p2021] -> [p2021, p2022]` — really
discarding the old data) apart from a merge / split / replace
shape (`[p0<10, p1<20, p2<30] -> [p0<10, q1<40]` — q1
semantically inherits p1's and p2's rows). The previous logic
emitted DROP + ADD as two ordinary statements for the second
shape and would have silently lost data.

The conservative fix is to error on every both-ADD-and-DROP
diff, matching how round 14 had it before the relaxation.
Users who really do want to discard rows can run
`DROP PARTITION p_old; ADD PARTITION p_new …;` themselves; the
typical retention case is then a one-statement DROP that the
diff already handles via the order-preserving subset path.

Fixtures:
- Drop the apply fixture (the shape no longer applies).
- Repurpose the plan fixture as
  `partition_drop_and_add_disjoint_errors`: same disjoint-name
  shape, asserts the REORGANIZE error fires now.

Docs:
- CAVEATS "Partitioning" entry reworded — the "Retention
  roll-forward" generated bullet is gone, the unsupported
  bullet now spells out why disjoint-name detection isn't
  enough on its own (the merge / split shapes that look
  harmless but aren't) and points users at the manual
  REORGANIZE / DROP+ADD workaround.
- PR description's behaviour matrix collapses the two
  both-ADD-and-DROP rows back into one and updates the summary
  text to match.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 49 out of 49 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread CAVEATS.md Outdated
Comment thread testdata/plan/partition_key_rename_clause_unchanged_errors.yml Outdated
Comment thread testdata/plan/partition_key_rename_with_suffix_add_errors.yml Outdated
Comment thread testdata/plan/partition_key_drop_conflict_errors.yml Outdated
Drop the partition-column rename / drop *conflict guards* and
their fixtures. Copilot's analysis was correct: every fixture
that exercised those guards relied on a desired CREATE TABLE
that's actually invalid MySQL DDL (rename old_dt → dt while
PARTITION BY still references old_dt; or drop dt while
PARTITION BY still references dt). The guards were therefore
only reachable when myschema's parser silently accepted an
invalid cross-reference, which isn't a shape we should be
testing for.

For valid desired CREATE TABLEs, the guards are unreachable:

- Renaming a partition-key column is only valid when the
  desired-side `PARTITION BY` clause is updated to use the new
  name too. That makes the partition expression different from
  the catalog's, and `partition strategy / expression differs`
  fires upstream from `diffPartitions`.
- Dropping a partition-key column is only valid when the column
  is also removed from the desired PRIMARY KEY / UNIQUE INDEX
  (otherwise the CREATE TABLE wouldn't parse on real MySQL).
  The unique-key cover guard added in round 15 then fires with
  `… is missing partition column(s) [...]` because a desired
  PRIMARY KEY without the partition column can't cover the
  partition expression.

So no functional shape regresses; the guards become dead code
for any valid-DDL input. Remove them along with the unreachable
fixtures.

Removed code:
- diff/rename.go::partitionColumnConflicts
- diff/partitions.go::partitionExpressionsEqual
- The rename/drop conflict block in diff/tables.go::diffTable
  (between diffPartitions and the cover guard)

Removed fixtures:
- testdata/plan/partition_key_rename_clause_unchanged_errors
- testdata/plan/partition_key_rename_with_suffix_add_errors
- testdata/plan/partition_key_drop_conflict_errors

Doc / comment updates:
- CAVEATS "Partitioning" entry collapses the conflict bullet
  into the cover-guard bullet; calls out that the cover guard
  absorbs the drop-partition-column case for any valid CREATE
  TABLE, and that the rename case is caught upstream as
  "scheme/expression differs".
- Apply fixture comment for partition_no_diff_with_non_partition_column_rename
  no longer references the removed conflict guard or its
  fixtures.
- PR description's behaviour matrix collapses the rename /
  drop conflict rows into one rename row + the cover-gap row,
  and the implementation section reflects the simpler design.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 46 out of 46 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@winebarrel winebarrel merged commit 05e4227 into main May 3, 2026
8 of 9 checks passed
@winebarrel winebarrel deleted the partition-diff-range-list branch May 3, 2026 15:32
winebarrel added a commit that referenced this pull request May 3, 2026
…/KEY count changes

Builds on PR #50's RANGE/LIST suffix add + subset drop. HASH /
KEY (incl. LINEAR HASH and LINEAR KEY, as well as KEY
ALGORITHM=N) now produce real DDL when the partition strategy
is unchanged and only `PARTITIONS n` differs:

- count grow → `ALTER TABLE … ADD PARTITION PARTITIONS n`
  (n = desired - current). No data loss; new slots are empty
  until the next rebalance.
- count shrink → `ALTER TABLE … COALESCE PARTITION n`
  (n = current - desired). MySQL rebalances rows from the
  dropped slots into the survivors, but the slots themselves
  go away — gated on `--allow-drop=partition` like RANGE/LIST
  DROP. Without the flag the COALESCE lands on the disallowed
  bucket as a `-- skipped:` line.

Equal counts already returned no-op (round 4); scheme mismatch
(Type / IsLinear / KeyAlgorithm / ColList / Expr) still falls
through to "scheme/expression differs" upstream.

Implementation: diff/partitions.go's HASH/KEY branch (which
previously returned the unsupported-diff error) now emits the
two count statements. partitionHeaderEqual already covers
LINEAR / KEY ALGORITHM via its IsLinear and KeyAlgorithm
fields, so no new normalisation is needed.

Tests:
- 5 plan fixtures: HASH grow, HASH shrink with allow_drop,
  HASH shrink disallowed, KEY grow, LINEAR HASH grow.
- 2 apply fixtures with verify_no_drift: HASH grow, HASH
  shrink. Pin that the generated DDL runs on MySQL and the
  next plan reports no drift.
- 2 diff unit tests: KEY grow (ADD PARTITION PARTITIONS),
  KEY shrink with vs. without allow-drop=partition.
- Replaced TestDiffPartitionsKEYCountChangeStillErrors (which
  pinned the old error) with the grow / shrink pair above.
- Removed testdata/plan/partition_hash_diff_errors.yml (same
  reason — count-change is no longer an error).

Docs:
- CAVEATS.md "Partitioning" moves the HASH/KEY count change
  from "Diffs that still error" to "Diffs that *are*
  generated".
- AGENTS.md "In scope" lists the count-change shape; the
  "Not yet implemented" entry no longer mentions HASH/KEY.
- TODO.md entry updated to reflect what's left (REORGANIZE,
  scheme change, add/remove partitioning).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants