options, diff: add --bulk-alter to combine same-table ALTER statements#77
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #77 +/- ##
==========================================
+ Coverage 86.33% 86.62% +0.28%
==========================================
Files 29 30 +1
Lines 3169 3237 +68
==========================================
+ Hits 2736 2804 +68
Misses 267 267
Partials 166 166 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
Adds an opt-in --bulk-alter path to the diff/apply pipeline so consecutive same-table ALTER TABLE statements can be folded into a single multi-spec ALTER without reordering buckets that preserve FK safety.
Changes:
- Added
AlterOption.BulkAlterand wired it intodiffAll()so only the main table-change bucket is eligible for same-table ALTER folding. - Introduced
combineSameTableAlters/splitCombinableAlterplus unit tests covering combinable and separator cases. - Added apply fixtures and documentation describing the new behavior and FK/partition/
CREATE INDEXexclusions.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
bulk_alter.go |
Implements the ALTER-combining helper and parser. |
diff_all.go |
Applies bulk-alter to tableDiff.Stmts before bucket merge. |
options.go |
Adds the new BulkAlter option/env wiring. |
bulk_alter_test.go |
Adds unit coverage for combine/pass-through behavior. |
apply_test.go |
Extends apply YAML fixtures to pass bulk_alter. |
testdata/apply/bulk_alter_combines_same_table_columns.yml |
End-to-end fixture for combining consecutive column alters. |
testdata/apply/bulk_alter_keeps_fk_separate.yml |
End-to-end fixture for keeping FK adds separate under bulk-alter. |
CAVEATS.md |
Documents FK-ordering rationale and excluded statement shapes. |
AGENTS.md |
Documents the new option for repository guidance. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- bulk_alter.go: drop the misleading `DumpOptions` reference from the combineSameTableAlters comment. dump does not embed AlterOption — bulk-alter is a diff-time concern only. Mention plan / apply explicitly so future maintainers trace it correctly. - AGENTS.md / CAVEATS.md: correct the combinable list. Secondary-index ADDs are emitted as standalone `CREATE INDEX … ON t (…);` (see diff/tables.go:728), not as `ALTER TABLE … ADD KEY` — so they never fold under --bulk-alter. Index *DROPs* still combine (they're emitted as `ALTER TABLE … DROP INDEX`); call out the asymmetry. Drop the inaccurate "ADD / DROP INDEX (when emitted as ALTER TABLE … ADD KEY)" wording from both docs. - plan_test.go: extend planTestCase with a `bulk_alter` field and thread it into PlanOptions.AlterOption so plan-side fixtures can pin the rendered SQL directly. Add testdata/plan/bulk_alter_combine_with_index_add_separate.yml that exercises three column adds + one CREATE INDEX with --bulk-alter and --alter-algorithm/--alter-lock combined: catches a render or splice regression that the apply post-drift check cannot see. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Address Copilot review #2 on PR #77, which flagged that the existing bulk_alter fixtures don't exercise: - `-- myschema:renamed-from` flows folded into a multi-spec ALTER. `ALTER TABLE t RENAME COLUMN a TO b, ADD COLUMN c …;` puts MySQL in charge of the within-statement evaluation order; the diff layer's column-ref rewrites are necessary but not sufficient proof that the combined shape converges. - Same-name `DROP CHECK + ADD CONSTRAINT` replacement rewritten as `DROP CHECK x, ADD CONSTRAINT x CHECK (…);`. Wrong relative order inside one ALTER would either duplicate-name error (ADD first) or leave no constraint at all (DROP first with an evaluation gap), so end-to-end coverage was missing. Both new fixtures pin `verify_no_drift: true` so the round-trip is asserted on top of the rendered SQL. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 13 out of 13 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Address Copilot review #3 on PR #77: - Comment in testdata/apply/change_primary_key.yml claimed MySQL "can't combine [DROP PK + ADD PK] in a single statement when the column set changes." That's wrong on myschema's MySQL 8.0+ baseline: the parser explicitly accepts `DROP PRIMARY KEY` followed by `ADD PRIMARY KEY` in one alter_specifications list as an atomic replacement. New fixture testdata/apply/bulk_alter_pk_replace.yml pins that the combined form applies cleanly and round-trips empty (verify_no_drift: true) — so the AGENTS.md / CAVEATS.md combinable list correctly includes PK constraints under --bulk-alter. change_primary_key.yml's comment now describes the two-statement default-mode shape it actually pins, and points at the bulk_alter variant. - Plan-side fixture comment in testdata/plan/bulk_alter_combine_with_index_add_separate.yml said "Three column adds" but the YAML adds four (a, b, c, email). Off-by-one in the description; corrected. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 17 out of 17 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Opt-in flag (default off; env MYSCHEMA_BULK_ALTER) that folds
*consecutive* same-table single-spec ALTER TABLE statements in the
table-change bucket into one multi-spec ALTER. The combiner is
order-preserving and runs only on tableDiff.Stmts — not on the
RenameStmts / FKDropStmts / FKAddStmts buckets — so the diff
pipeline's cross-table FK ordering invariants stay intact.
Excluded (each acts as a run separator):
- FK adds / drops live in dedicated FK buckets so cross-table
ordering survives. CAVEATS.md "Bulk-alter does not combine FK
operations" explains the rationale.
- Partition ops (REORGANIZE / ADD / DROP / COALESCE / TRUNCATE /
EXCHANGE PARTITION). MySQL's grammar rejects the trailing-comma
multi-spec form for these clauses; detected via the existing
partitionOpInsertPos helper.
- Standalone CREATE INDEX (different statement shape).
- CREATE / DROP / RENAME TABLE.
Tests: 12 unit cases in bulk_alter_test.go (combineSameTableAlters
and splitCombinableAlter both 100%) covering the headline behaviour,
edge cases (nil/single/no-semicolon/back-ticked), and run-separator
shapes (different-table, non-contiguous, partition, CREATE INDEX,
malformed, DROP TABLE). Two YAML fixtures pin the end-to-end
behaviour: same-table column folds combine, FK ops + standalone
CREATE INDEX stay separate.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- bulk_alter.go: drop the misleading `DumpOptions` reference from the combineSameTableAlters comment. dump does not embed AlterOption — bulk-alter is a diff-time concern only. Mention plan / apply explicitly so future maintainers trace it correctly. - AGENTS.md / CAVEATS.md: correct the combinable list. Secondary-index ADDs are emitted as standalone `CREATE INDEX … ON t (…);` (see diff/tables.go:728), not as `ALTER TABLE … ADD KEY` — so they never fold under --bulk-alter. Index *DROPs* still combine (they're emitted as `ALTER TABLE … DROP INDEX`); call out the asymmetry. Drop the inaccurate "ADD / DROP INDEX (when emitted as ALTER TABLE … ADD KEY)" wording from both docs. - plan_test.go: extend planTestCase with a `bulk_alter` field and thread it into PlanOptions.AlterOption so plan-side fixtures can pin the rendered SQL directly. Add testdata/plan/bulk_alter_combine_with_index_add_separate.yml that exercises three column adds + one CREATE INDEX with --bulk-alter and --alter-algorithm/--alter-lock combined: catches a render or splice regression that the apply post-drift check cannot see. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Address Copilot review #2 on PR #77, which flagged that the existing bulk_alter fixtures don't exercise: - `-- myschema:renamed-from` flows folded into a multi-spec ALTER. `ALTER TABLE t RENAME COLUMN a TO b, ADD COLUMN c …;` puts MySQL in charge of the within-statement evaluation order; the diff layer's column-ref rewrites are necessary but not sufficient proof that the combined shape converges. - Same-name `DROP CHECK + ADD CONSTRAINT` replacement rewritten as `DROP CHECK x, ADD CONSTRAINT x CHECK (…);`. Wrong relative order inside one ALTER would either duplicate-name error (ADD first) or leave no constraint at all (DROP first with an evaluation gap), so end-to-end coverage was missing. Both new fixtures pin `verify_no_drift: true` so the round-trip is asserted on top of the rendered SQL. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Address Copilot review #3 on PR #77: - Comment in testdata/apply/change_primary_key.yml claimed MySQL "can't combine [DROP PK + ADD PK] in a single statement when the column set changes." That's wrong on myschema's MySQL 8.0+ baseline: the parser explicitly accepts `DROP PRIMARY KEY` followed by `ADD PRIMARY KEY` in one alter_specifications list as an atomic replacement. New fixture testdata/apply/bulk_alter_pk_replace.yml pins that the combined form applies cleanly and round-trips empty (verify_no_drift: true) — so the AGENTS.md / CAVEATS.md combinable list correctly includes PK constraints under --bulk-alter. change_primary_key.yml's comment now describes the two-statement default-mode shape it actually pins, and points at the bulk_alter variant. - Plan-side fixture comment in testdata/plan/bulk_alter_combine_with_index_add_separate.yml said "Three column adds" but the YAML adds four (a, b, c, email). Off-by-one in the description; corrected. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Probe fixture revealed a real bug: MySQL rejects ALTER TABLE users RENAME COLUMN old_name TO name, MODIFY COLUMN name varchar(128) NOT NULL; with `Error 1054 (42S22): Unknown column 'name' in 'users'`. Within one ALTER, MySQL resolves spec-target identifiers (`MODIFY COLUMN <name>`, `DROP INDEX <name>`, …) against the *original* table state, so any spec referring to the renamed object by its new name fails. The diff layer emits column rename + a follow-on MODIFY (or index rename + follow-on DROP) into Stmts as two separate ALTERs, which --bulk-alter was happily folding into the broken combined form. Fix: splitCombinableAlter rejects any spec starting with `RENAME COLUMN` or `RENAME INDEX`, so renames keep their own ALTER even under --bulk-alter. Whole-table `RENAME TO` lives in the RenameStmts bucket upstream and never reaches the combiner, so no guard needed for that shape. Test changes: - New unit tests pin the RENAME COLUMN and RENAME INDEX run-separator behaviour (both 100% covered). - testdata/apply/bulk_alter_rename_then_modify_column.yml is the end-to-end regression — would fail with 1054 without the guard. - testdata/apply/bulk_alter_renamed_column_with_followup.yml updated to expect the post-fix two-ALTER shape (was previously asserting the unsafe combined form, which only worked because the follow-on ADD COLUMN happened to use AFTER instead of MODIFY). - testdata/apply/bulk_alter_charset_with_column_change.yml: new fixture for Copilot review #4, pinning DEFAULT CHARSET combined with ADD COLUMN. The CHARSET / column two-apply convergence (CAVEATS.md) is unchanged by --bulk-alter — verify_no_drift: false matches change_table_charset.yml's baseline. CAVEATS.md / AGENTS.md updated with the rename trap and rationale. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2ac6ad5 to
c558479
Compare
Apply the three review items that pay for themselves; skip the help-text expansion (per prior user feedback that --bulk-alter help is already long enough — the same applies to the rename exclusions). - CAVEATS.md gains a "--bulk-alter interacts with --alter-algorithm / --alter-lock" subsection. Combining specs into one ALTER means MySQL applies a single ALGORITHM= / LOCK= choice to the whole statement, not per-spec. A run mixing INSTANT-eligible specs with one that needs INPLACE / COPY (DEFAULT CHARSET, generated columns, some MODIFY COLUMNs) becomes a COPY-or-fail under --alter-algorithm=INSTANT — the documentation now warns operators to plan first and verify each spec before opting in. - testdata/apply/bulk_alter_convert_charset_with_column_add.yml: end-to-end fixture for `-- myschema:convert-charset` folded with a same-table ADD COLUMN under bulk-alter. CONVERT TO CHARACTER SET has different syntax / rebuild semantics than DEFAULT CHARSET (data rewrite vs. table-default flip), so the existing bulk_alter_charset_with_column_change.yml didn't cover it. - options_test.go: TestAlterOptionKongBulkAlter pins the kong tag wiring for --bulk-alter and MYSCHEMA_BULK_ALTER. The YAML fixtures set BulkAlter directly via Go, so a typo in the kong field-name → flag-name mapping or env tag content would otherwise slip past CI. Mirrors the existing pre-sql / allow-drop kong-parse pins. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 19 out of 19 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- options.go: --bulk-alter help string now lists `RENAME COLUMN / RENAME INDEX` alongside FK / partition / standalone CREATE INDEX. CLI users hitting --help get the same exclusion picture as CAVEATS.md / AGENTS.md. - AGENTS.md: drop the false "and the partition shapes that aren't on the excluded list below" tail from the Combinable bullet. The excluded list covers every partition op the diff currently emits (ADD / DROP / COALESCE / REORGANIZE / TRUNCATE / EXCHANGE PARTITION), so no partition shape is combinable today. Mention CONVERT TO CHARACTER SET in the combinable list since bulk_alter_convert_charset_with_column_add.yml pins it. - bulk_alter.go: rewrite the partitionOpInsertPos comment. Previous "a partition op anywhere after the table name disqualifies the whole stmt" overstated the helper — partitionOpInsertPos only matches a partition keyword *immediately* after the table name (the same anchor appendAlterHints uses for splicing). The new comment names that anchor and the why (one alter_specification per stmt in the diff layer's output puts the partition op exactly there). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 22 out of 22 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Address Copilot review on PR #77 raising two potential evaluation- order traps. Probe fixtures confirm MySQL 8.0+ accepts both combined forms; promote both to durable regression pins so a future MySQL behaviour change (or a regression in the combiner) surfaces in CI rather than in production: - bulk_alter_first_after_chain.yml: `ADD COLUMN head_a INT FIRST, ADD COLUMN head_b INT AFTER head_a;` — the AFTER clause targets the column the preceding spec just inserted at FIRST. MySQL resolves it against the post-FIRST column list. - bulk_alter_add_column_with_check.yml: `ADD COLUMN price INT, ADD CONSTRAINT chk CHECK (price >= 0);` — the constraint references the just-added column. Same evaluation-order class as the RENAME COLUMN + MODIFY trap (CAVEATS.md), but on this path MySQL 8.0+ resolves the CHECK against the post-spec state, so the combined form applies cleanly. Both round-trip empty (verify_no_drift: true). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 24 out of 24 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Address Copilot review on PR #77. The previous "Drop one of the flags if the combined ALGORITHM/LOCK choice doesn't fit" was misleading because `appendAlterHints` always appends the user-supplied ALGORITHM= clause to every generated ALTER TABLE, regardless of `--bulk-alter`. So dropping `--bulk-alter` alone splits the run into per-spec ALTERs but each one still carries `ALGORITHM=INSTANT` — a non-INSTANT-eligible spec still fails on its own statement. Rewrite the section to call this out and split the recovery guidance: - Loosening the algorithm pin (or dropping `--alter-algorithm` entirely) is the only fix that changes what algorithm MySQL applies. - Dropping `--bulk-alter` recovers the *failure shape* (each spec succeeds or fails on its own ALTER instead of one combined abort), but doesn't help any spec succeed on its own. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 24 out of 24 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Address Copilot review on PR #77: - CAVEATS.md: the previous bullet asserted "two ADD COLUMN adds ... is fine — both qualify" for `--alter-algorithm=INSTANT`. INSTANT eligibility depends on exact 8.0.x version, column position, and presence of generated columns ahead of the new one — myschema's baseline is "8.0+" without pinning a minor, so the absolute claim was misleading. Reword to "when every spec individually qualifies for the pinned algorithm, the combined ALTER inherits that qualification" with an explicit per-spec verification disclaimer. - testdata/apply/bulk_alter_table_comment_with_column_add.yml: end-to-end fixture for `COMMENT=` (table_option) folded with a same-table ADD COLUMN. Different grammar shape from per-column alter_specifications; combinable list documents it but no apply pin existed. The diff orders the COMMENT= spec before column adds (matching diff_all.go's bucket assembly), so the asserted shape is `ALTER TABLE t COMMENT='new', ADD COLUMN ...`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 25 out of 25 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Address Copilot review: - options.go: AlterOption struct doc previously read "carries the MySQL online-DDL hints that get appended to every generated ALTER TABLE / CREATE INDEX statement" — accurate when the type only held AlterAlgorithm / AlterLock, but now the same struct also carries BulkAlter. Restructure the doc as a per-field bullet list so Go docs and editor hovers describe both responsibilities accurately. - testdata/plan/bulk_alter_combine_with_index_add_separate.yml: drop the hard-coded `diff/tables.go:728` line reference and point at the CAVEATS.md "Secondary-index ADDs" subsection instead, so the rationale survives unrelated edits to diff/tables.go. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 25 out of 25 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Address Copilot review on PR #77 with two more end-to-end pins: - bulk_alter_charset_with_modify_column.yml: counterpart to bulk_alter_charset_with_column_change.yml. The CHARSET + ADD COLUMN shape was covered, but DEFAULT CHARSET + MODIFY COLUMN is the more fragile path because string columns inherit the table charset only when their per-column metadata isn't already pinned. CAVEATS.md flags the two-apply convergence rule there; bulk-alter doesn't change that, so verify_no_drift: false matches change_table_charset.yml's baseline. The fixture pins that the combined first-apply ALTER applies syntactically and that the bucket order (tableCharsetCollationSQL before diffColumns) survives a refactor. - bulk_alter_drop_column_with_drop_index.yml: bulk-alter counterpart to drop_one_of_two_index_columns.yml. When only some of an index's columns are dropped, MySQL keeps the index (folded down to surviving columns), so the diff still emits DROP INDEX + CREATE INDEX rebuild. Under bulk-alter the DROP COLUMN + DROP INDEX fold into one ALTER while the CREATE INDEX stays standalone (separate-bucket rule). Pins that `ALTER TABLE t DROP COLUMN a, DROP INDEX i_combined;` is accepted when the index's other column survives, and that the rebuild then lands on the post-ALTER state. (The replace_index_with_column_drop.yml shape — drop *all* index columns — uses dependent-index suppression and doesn't emit DROP INDEX at all, so bulk-alter has nothing to fold there.) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 27 out of 27 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 27 out of 27 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 27 out of 27 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Summary
--bulk-alterflag (envMYSCHEMA_BULK_ALTER, default off) folds consecutive same-table single-specALTER TABLEstatements into one multi-spec ALTER. Order-preserving — never reorders.tableDiff.Stmts. The dedicatedRenameStmts/FKDropStmts/FKAddStmtsbuckets are untouched, so cross-table FK ordering invariants stay intact (FK adds still run last, FK drops still run first).REORGANIZE/ADD/DROP/COALESCE/TRUNCATE/EXCHANGE PARTITION— MySQL grammar rejects the trailing-comma multi-spec form; detected via existingpartitionOpInsertPos), standaloneCREATE INDEX, andCREATE/DROP/RENAME TABLE.See
CAVEATS.md"Bulk-alter does not combine FK operations" for the FK exclusion rationale.Test plan
bulk_alter_test.gocover headline behaviour, edge cases (nil / single / no-semicolon / back-ticked), and run-separator shapes (different-table, non-contiguous, partition, CREATE INDEX, malformed, DROP TABLE).combineSameTableAltersandsplitCombinableAlterboth 100% covered.bulk_alter_combines_same_table_columns.yml: same-table ADD COLUMN ×2 fold into one multi-spec ALTER, thenverify_no_driftround-trip.bulk_alter_keeps_fk_separate.yml: even when bulk-alter is on and the FK targets the same table that just got new columns, the FK add lands as its own ALTER. StandaloneCREATE INDEXalso breaks the run as documented.make lintclean.make testall packages green (mysql 8.0).make test-scenariogreen.🤖 Generated with Claude Code