Skip to content

catalog: normalise empty-string DEFAULT for string-shaped columns#33

Merged
winebarrel merged 4 commits into
mainfrom
fix-empty-default-drift
May 3, 2026
Merged

catalog: normalise empty-string DEFAULT for string-shaped columns#33
winebarrel merged 4 commits into
mainfrom
fix-empty-default-drift

Conversation

@winebarrel

@winebarrel winebarrel commented May 2, 2026

Copy link
Copy Markdown
Owner

Summary

Closes the empty-string DEFAULT '' round-trip drift documented in PR #32.

A column declared with NOT NULL DEFAULT '' apply-looped: every post-apply plan re-emitted MODIFY COLUMN … DEFAULT '' because the catalog returned the bare empty string from information_schema.COLUMNS.COLUMN_DEFAULT while the parser side stored '' (quoted empty string).

Fix

catalog.normalizeColumnDefault short-circuited on def == "" before its vitess-based bareword check. Removing the short-circuit alone doesn't help — vitess can't parse SELECT with an empty expression. The fix is type-aware: when the column type accepts DEFAULT '' as the bare empty string (VARCHAR / CHAR / VARBINARY / ENUM / SET), normalise to ''. The function finally consumes its long-dormant typeName argument.

Fixed-width BINARY(N) is intentionally excluded from this normalisation: MySQL pads its empty default to N zero bytes and surfaces it through information_schema as a hex literal (0x for the degenerate case, 0x000000… for non-zero N), so it needs its own catalog-side normalisation. That remains the only outstanding follow-up in TODO.md. VARBINARY (variable-length) does surface its empty default as the bare empty string and is fixed in this PR.

Also in this PR

Drops misspell from .golangci.yml. The linter was rewriting consecutive ASCII single-quotes ('') into a typographic right-double-quote in the very docstrings this PR adds, which then surfaced as Copilot review nits. (Independently, gofmt in Go 1.26 does the same typographic collapse, so the empty-default explanation is phrased as prose — "the quoted empty literal" / "a pair of single-quote characters" — to survive a format pass.)

Tests

  • catalog/catalog_test.go::TestColumnDefaultEmptyStringNormalisation — VARCHAR / CHAR / VARBINARY / ENUM / SET DEFAULT '' all normalise to ''.
  • catalog/catalog_test.go::TestColumnDefaultBinaryEmptyStringIsHex — pins the documented BINARY exclusion (catches a future refactor that accidentally normalises the hex literal before the proper BINARY-side fix lands).
  • testdata/apply/empty_string_default_no_drift.yml — end-to-end YAML with verify_no_drift: true, includes a VARBINARY column.
  • test/scenario/testdata/evolution/display_name columns get their NOT NULL DEFAULT '' back (PR test/scenario: cover the major happy-path workflows #31 worked around this bug by dropping the explicit DEFAULT). Evolution scenario now exercises the empty-default round-trip.

Verified on mysql 8.0 + 9.4: full Go suite, YAML harness, scenario suite all green.

Test plan

  • make test (8.0)
  • make test MYSQL_PORT=3307 (9.4)
  • make test-scenario (8.0 + 9.4)
  • make lint

🤖 Generated with Claude Code

A column declared with `NOT NULL DEFAULT ''` apply-looped: every
post-apply plan re-emitted `MODIFY COLUMN … DEFAULT ''` because the
catalog returned the bare empty string from
`information_schema.COLUMNS.COLUMN_DEFAULT` while the parser side
stored `''` (quoted empty string). They never compared equal.

`catalog.normalizeColumnDefault` short-circuited on `def == ""`
before its vitess-based bareword check, which was the root cause —
removing the short-circuit alone doesn't help because vitess can't
parse `SELECT` with an empty expression. The fix is type-aware:
when the column type accepts `DEFAULT ''` as the bare empty string
(VARCHAR / CHAR / ENUM / SET), normalise to `''`. The function
finally consumes its long-dormant `typeName` argument.

BINARY / VARBINARY have a different round-trip drift (MySQL stores
their `''` default as a hex literal `0x` / `0x000000…`). That needs
its own catalog-side normalisation; filed in TODO.md as a follow-up.

Tests
  - catalog/catalog_test.go: TestColumnDefaultEmptyStringNormalisation
    sets up a table with VARCHAR / CHAR / ENUM / SET DEFAULT ''
    columns and asserts each normalises to `''`.
  - testdata/apply/empty_string_default_no_drift.yml: end-to-end
    YAML fixture with `verify_no_drift: true` so the round-trip is
    pinned through the full plan/apply pipeline.
  - test/scenario/testdata/evolution/0[2-5]_*.sql: restore the
    `NOT NULL DEFAULT ''` clauses on display_name (PR #31 worked
    around the bug by dropping the explicit DEFAULT). The evolution
    scenario now exercises the empty-default round-trip too.
  - test/scenario/evolution.test.sh: header comment updated.

TODO.md
  - Remove the empty-DEFAULT entry (fixed here).
  - Add a follow-up entry for the BINARY / VARBINARY case so the
    workaround is documented.

Verified on both compose services (mysql 8.0, mysql 9.4): full Go
suite, YAML harness, scenario suite all pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings May 2, 2026 16:28
@codecov

codecov Bot commented May 2, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 66.66667% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.38%. Comparing base (9393ca8) to head (e94da53).
⚠️ Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
catalog/tables.go 66.66% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main      #33   +/-   ##
=======================================
  Coverage   73.37%   73.38%           
=======================================
  Files          27       27           
  Lines        2141     2149    +8     
=======================================
+ Hits         1571     1577    +6     
- Misses        410      412    +2     
  Partials      160      160           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes the catalog/parser mismatch for empty-string column defaults so DEFAULT '' on string-like columns no longer causes perpetual post-apply drift. It fits into myschema’s catalog/diff pipeline by making catalog-loaded defaults align with the parser’s normalized representation, then extending regression coverage across unit, YAML, and scenario tests.

Changes:

  • Make catalog.normalizeColumnDefault type-aware for empty-string defaults on supported string-shaped column types.
  • Add catalog and apply-level regressions for empty-string default round-trips with verify_no_drift.
  • Re-enable NOT NULL DEFAULT '' in the evolution scenario and update TODOs/comments for the remaining binary-type follow-up.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
catalog/tables.go Adds the empty-string default normalization branch and helper for supported column types.
catalog/catalog_test.go Adds an integration test covering catalog-loaded empty-string defaults for varchar/char/enum/set.
testdata/apply/empty_string_default_no_drift.yml Adds an end-to-end apply fixture asserting no drift after applying empty-string defaults.
test/scenario/evolution.test.sh Updates scenario documentation to reflect that evolution now covers DEFAULT ''.
test/scenario/testdata/evolution/02_add_column.sql Reintroduces NOT NULL DEFAULT '' in the add-column stage fixture.
test/scenario/testdata/evolution/03_add_index.sql Keeps the empty-string default present while exercising later index changes.
test/scenario/testdata/evolution/04_add_fk.sql Keeps the empty-string default present while adding the FK stage.
test/scenario/testdata/evolution/05_modify_column.sql Keeps the empty-string default present while widening the column type.
TODO.md Replaces the resolved empty-string drift item with the remaining BINARY/VARBINARY follow-up.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread catalog/tables.go Outdated
Comment thread catalog/catalog_test.go Outdated
Comment thread catalog/tables.go Outdated
Three inline comments, plus a refinement uncovered while writing the
new test:

catalog/tables.go + catalog/catalog_test.go (typography fix)
  - Replace stray U+201D ("right double quotation mark") with the
    ASCII `''` literal in the doc comments. Editor / clipboard
    contamination from the previous round; the SQL examples now
    read correctly. (Copilot #1, #2.)

catalog/tables.go (BINARY/VARBINARY split)
  - The original `columnTypeAllowsEmptyStringDefault` excluded
    both BINARY and VARBINARY together. Verifying with information_
    schema showed only fixed-width BINARY surfaces its empty
    default as a hex literal (`0x` for the degenerate case;
    `0x000000…` for non-zero N). VARBINARY (variable-length)
    surfaces the bare empty string just like VARCHAR, so it
    round-trips cleanly via the empty-string path. Narrow the
    exclusion to BINARY only and add VARBINARY to the supported
    list. The function now checks `varbinary` before `binary` so
    HasPrefix doesn't let one match the other.

catalog/catalog_test.go (test for the BINARY exclusion)
  - TestColumnDefaultBinaryEmptyStringIsHex pins the documented
    BINARY behaviour and catches a future refactor that
    accidentally normalises the hex literal to ''. The companion
    TestColumnDefaultEmptyStringNormalisation gains a VARBINARY
    column to exercise the type's now-supported path. (Copilot #3.)

`make test` (8.0) — pass; `make test-scenario` (8.0) — pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread catalog/tables.go Outdated
Comment thread TODO.md Outdated
Comment thread testdata/apply/empty_string_default_no_drift.yml
Comment thread catalog/catalog_test.go Outdated
Four Copilot inline comments on PR #33:

catalog/tables.go (doc) + TODO.md
  - Doc and TODO still listed VARBINARY alongside BINARY as "needs
    its own normalisation". The previous round actually moved
    VARBINARY into the supported set (it surfaces its DEFAULT '' as
    the bare empty string, so the existing path round-trips it
    cleanly). Update the function-doc and TODO to spell out that
    only fixed-width BINARY remains broken. (Copilot #1, #2.)

testdata/apply/empty_string_default_no_drift.yml
  - Add a VARBINARY(8) column to the no-drift fixture so the new
    branch is exercised through plan + apply + drift-check, not
    just the catalog unit test. (Copilot #3.)

catalog/tables.go + catalog/catalog_test.go (typography)
  - Bulk replace stray U+201D ("right double quotation mark") with
    ASCII `''` across both files. The previous round's Edit tool
    ate the quote characters; verified the bytes with a follow-up
    grep. (Copilot #4.)

`make test` (8.0) — pass; `make test-scenario` (8.0) — pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread catalog/tables.go Outdated
Comment thread catalog/tables.go Outdated
Comment thread catalog/catalog_test.go Outdated
Comment thread catalog/catalog_test.go Outdated
`misspell` was rewriting consecutive ASCII single-quotes (`''`) into a
typographic right-double-quote in comments, which then became a Copilot
review nit. Drop the linter — the project is small enough that an
in-band misspelling check isn't paying for itself.

Independently, `gofmt` in Go 1.26 also collapses `''` to `”` in
comments, so even with misspell gone the literal pair can't survive a
format pass. Rewrite the empty-default explanation in tables.go and
catalog_test.go to refer to "the quoted empty literal" / "a pair of
single-quote characters" in prose instead, which both readers and
gofmt are fine with.

Also fold in the function-body restructure for
`columnTypeAllowsEmptyStringDefault`: BINARY is the sole excluded
type, so check it first and let the supported list (VARCHAR / CHAR /
VARBINARY / ENUM / SET) be a flat OR — the previous "VARBINARY before
BINARY" ordering hinted at a HasPrefix collision that doesn't actually
exist (HasPrefix("varbinary", "binary") is false).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@winebarrel winebarrel requested a review from Copilot May 3, 2026 01:23

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .golangci.yml
Comment thread catalog/tables.go

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@winebarrel winebarrel merged commit 3ab0981 into main May 3, 2026
12 of 13 checks passed
@winebarrel winebarrel deleted the fix-empty-default-drift branch May 3, 2026 01:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants