Skip to content

options, apply, plan: add --pre-sql / --pre-sql-file#75

Merged
winebarrel merged 5 commits into
mainfrom
add-pre-sql-option
May 5, 2026
Merged

options, apply, plan: add --pre-sql / --pre-sql-file#75
winebarrel merged 5 commits into
mainfrom
add-pre-sql-option

Conversation

@winebarrel

Copy link
Copy Markdown
Owner

Summary

Closes the TODO entry "`--pre-sql` / `--pre-sql-file` (and `MYSCHEMA_PRE_SQL` env vars)". Lets the operator run session-level SQL on the connection right after `connect()` and before any diff work — typical use:

  • `SET FOREIGN_KEY_CHECKS=0` for legacy migrations
  • `SET sql_mode='TRADITIONAL'` to control how DDL gets interpreted
  • `SET explicit_defaults_for_timestamp=ON` to flip TIMESTAMP behaviour

Design

  • Apply / plan only. `dump` is read-only and pre-SQL's typical use is DDL session setup; running it on dump would be confusing rather than useful.
  • `--pre-sql` and `--pre-sql-file` are mutually exclusive — keeps the precedence rule trivial. Both have matching env vars (`MYSCHEMA_PRE_SQL` / `MYSCHEMA_PRE_SQL_FILE`).
  • Multi-statement payload split via vitess' `SplitStatementToPieces` (same splitter `parser/parser.go` uses for desired SQL), so a chain like `SET a=1; SET b=2;` runs as two Exec calls in order. Splitting via vitess (rather than naive `strings.Split` on `;`) keeps `;` inside string literals from breaking the chain. `MultiStatements` stays disabled at the driver level — `client.go`'s "single-statement-per-Exec" safety posture is unchanged.
  • pre-SQL failure aborts the whole apply / plan before touching anything else. The error message includes the exact statement that failed (helpful when chaining several SETs).

Implementation

  • `options.go`: new `PreSQLOption` struct.
  • `apply.go` / `plan.go`: embed `PreSQLOption`, call `runPreSQL` right after `connect()`.
  • `pre_sql.go` (new): `runPreSQL` + `loadPreSQL` helpers.

Test plan (10 sub-tests)

Scenario Test Status
inline payload runs (apply) `TestApply_PreSQLString`
inline payload runs (plan) `TestPlan_PreSQLString`
file payload runs `TestApply_PreSQLFile`
both flags set `TestApply/Plan_PreSQLBothSetError`
bad SQL aborts apply `TestApply_PreSQLFailureSurfaces`
SET actually applied to session `TestApply_PreSQLAppliesToSession`
multi-statement split `TestApply_PreSQLMultiStatement`
neither set = no-op `TestApply_PreSQLNothingSet`
missing file path `TestApply_PreSQLFileMissing`
whitespace-only file `TestApply_PreSQLEmptyFile`

Coverage

  • `runPreSQL` 88.9% (remaining gap: parser-init / split-error defensive paths needing fault injection)
  • `loadPreSQL` 100%
  • Root `myschema` package 93.7% (unchanged)
  • Total project coverage 90.0% (unchanged)

Doc updates

  • `AGENTS.md` "In scope (v1)" gains a paragraph for `--pre-sql` / `--pre-sql-file` matching the existing `--alter-algorithm` / `--alter-lock` entry style.
  • `TODO.md`: the `--pre-sql` line is removed.

🤖 Generated with Claude Code

Closes the TODO entry "--pre-sql / --pre-sql-file (and
MYSCHEMA_PRE_SQL env vars)". Lets the operator run session-level
SQL on the connection right after connect() and before any diff
work — typical use is `SET FOREIGN_KEY_CHECKS=0`,
`SET sql_mode='TRADITIONAL'`, or
`SET explicit_defaults_for_timestamp=ON` to control how the server
interprets the upcoming DDL or catalog reads.

Design choices:

- Apply / plan only. dump is read-only and pre-SQL's typical use
  is DDL session setup; running it on dump would be confusing
  rather than useful.
- `--pre-sql` and `--pre-sql-file` are mutually exclusive — keeps
  the precedence rule trivial. Both have matching env vars
  (MYSCHEMA_PRE_SQL / MYSCHEMA_PRE_SQL_FILE).
- Multi-statement payload split via vitess'
  SplitStatementToPieces (same splitter parser/parser.go uses for
  desired SQL), so a chain like `SET a=1; SET b=2;` runs as two
  Exec calls in order. Splitting via vitess (rather than naive
  strings.Split on `;`) keeps `;` inside string literals from
  breaking the chain. MultiStatements stays disabled at the
  driver level — client.go's "single-statement-per-Exec" safety
  posture is unchanged.
- pre-SQL failure aborts the whole apply / plan before touching
  anything else. The error message includes the exact statement
  that failed (helpful when chaining several SETs).

Implementation: new PreSQLOption struct in options.go, embedded
in ApplyOptions / PlanOptions. New pre_sql.go holds runPreSQL +
loadPreSQL. apply.go / plan.go call runPreSQL right after
connect().

Test plan (10 sub-tests):

- TestApply_PreSQLString — inline payload runs.
- TestApply_PreSQLFile — file payload runs.
- TestApply/Plan_PreSQLBothSetError — mutually-exclusive contract.
- TestApply_PreSQLFailureSurfaces — bad SQL aborts apply with
  a "pre-sql" error.
- TestApply_PreSQLAppliesToSession — proves the SET runs on the
  connection apply uses (not a side channel).
- TestApply_PreSQLMultiStatement — three SETs in one payload
  exercise the vitess split path.
- TestApply_PreSQLNothingSet — neither flag = no-op.
- TestApply_PreSQLFileMissing — bad path errors with the path
  in the message.
- TestApply_PreSQLEmptyFile — whitespace-only file is a no-op.
- TestPlan_PreSQLString — same posture for plan.

Coverage: runPreSQL 88.9%, loadPreSQL 100%. Total project
coverage stays at 90.0%. Root myschema package 93.7%
(unchanged). The 11.1% gap on runPreSQL is the
parser-init / split-error defensive paths that need fault
injection to reach.

Doc updates:
- AGENTS.md "In scope (v1)" gains a paragraph for --pre-sql /
  --pre-sql-file matching the existing --alter-algorithm /
  --alter-lock entry style.
- TODO.md: --pre-sql line removed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings May 5, 2026 01:48
@codecov

codecov Bot commented May 5, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.36585% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.21%. Comparing base (e753d9e) to head (004630d).

Files with missing lines Patch % Lines
pre_sql.go 80.64% 3 Missing and 3 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #75      +/-   ##
==========================================
- Coverage   86.22%   86.21%   -0.02%     
==========================================
  Files          28       29       +1     
  Lines        3099     3140      +41     
==========================================
+ Hits         2672     2707      +35     
- Misses        264      267       +3     
- Partials      163      166       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds support for running operator-supplied “pre-SQL” on the session connection immediately after connect() and before plan/apply performs any diff work, enabling session-level configuration (e.g. FOREIGN_KEY_CHECKS, sql_mode) without enabling driver multi-statements.

Changes:

  • Introduce PreSQLOption (--pre-sql / --pre-sql-file + env vars) and wire it into plan and apply.
  • Add runPreSQL/loadPreSQL helper (new pre_sql.go) that splits multi-statement payloads via Vitess and executes each statement sequentially.
  • Update docs/tests and remove the completed TODO entry.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
TODO.md Removes the completed TODO item for pre-SQL flags.
pre_sql.go Implements loading/validating/executing pre-SQL (inline or file) and multi-statement splitting.
options.go Adds PreSQLOption to CLI option structs (env vars + help text).
apply.go Runs pre-SQL after acquiring the apply connection and before diff/apply work.
plan.go Runs pre-SQL after acquiring the plan connection and before diff work.
apply_test.go Adds apply-side tests for pre-SQL flags and error paths.
plan_test.go Adds plan-side tests for pre-SQL presence and mutual exclusion.
AGENTS.md Documents the new flags/env vars and intended behavior/scope.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread pre_sql.go Outdated
Comment thread pre_sql.go Outdated
Comment thread apply.go
Comment thread plan.go
Comment thread plan_test.go
Comment thread apply_test.go Outdated
Comment thread apply_test.go Outdated
Comment thread apply_test.go Outdated
Round 1 review on PR #75 caught eight issues; this commit addresses
all of them.

1. (review #1) loadPreSQL treated any non-empty PreSQL string as
   "set", including whitespace-only values. Whitespace-only env
   var (`MYSCHEMA_PRE_SQL=`) would trigger the mutually-exclusive
   error against a legitimate --pre-sql-file, and skip the no-op
   short-circuit. Fix: TrimSpace both fields before deciding "set".

2. (review #2) loadPreSQL read --pre-sql-file with os.ReadFile,
   which doesn't support the repo's existing `-` for stdin
   convention used by parser.ReadSQLFile (the desired-SQL file
   args already accept it). Fix: route through parser.ReadSQLFile
   so `--pre-sql-file=-` works.

3. (review #3 + #4) runPreSQL was invoked AFTER connect(), so
   flag-validation errors (both flags set, missing file) could be
   masked by a downstream connection failure, and the client
   opened a DB connection it then threw away. Split into:
   - loadPreSQL: validate / read, no DB contact (called BEFORE connect)
   - execPreSQL: run on conn (called AFTER connect)
   apply.go and plan.go updated accordingly.

4. CLI-level mutual exclusion: tag both PreSQL and PreSQLFile with
   kong's `xor:"pre-sql"` so kong rejects the both-set case at
   parse time before our code sees it. The runtime check in
   loadPreSQL stays in place for programmatic API callers
   (Apply / Plan invoked from Go without going through kong).

5. (review #5 / #6 / #7 / #8) Test reshuffle. The original
   suite over-relied on success-only smoke tests that would have
   silently passed even if pre-SQL were skipped:
   - TestApply_PreSQLString and TestApply_PreSQLMultiStatement
     only checked NoError.
   - TestApply_PreSQLAppliesToSession claimed to be the
     "strongest behavioural pin" but probed nothing — the
     connection it would have probed is closed by Apply's defer.
   - TestPlan_PreSQLString same issue.
   Fix: load-bearing pins now feed INVALID pre-SQL and assert
   the apply / plan aborts with a wrapped "pre-sql" error
   containing the exact failing piece. The multi-statement
   split test now uses a payload with an invalid SECOND piece
   and asserts the error references that piece exactly (proves
   the split happened — without splitting, the driver would
   reject the whole concatenated string with a different error).

6. YAML migration: the new harness field `pre_sql` lets
   inline-payload tests live as testdata/apply/pre_sql_*.yml and
   testdata/plan/pre_sql_*.yml fixtures (3 + 1 = 4 new fixtures).
   File-related and programmatic-API tests stay in Go because
   the YAML harness has no `pre_sql_file` field (would need
   fixture-relative path handling, out of scope).

Coverage is unchanged at the package level; loadPreSQL stays at
100% and execPreSQL is exercised by all the real-execution paths.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread apply.go
Comment thread plan.go
Comment thread pre_sql.go Outdated
winebarrel and others added 2 commits May 5, 2026 11:13
PR #75 round-2 review #3: --pre-sql-file=- and a `-` desired file
argument both read stdin; the second read would hit EOF and
silently truncate either the pre-SQL or the desired SQL. loadPreSQL
now takes the desired file list and fails fast with an explicit
error on the conflict, before any actual stdin read.

Reviews #1 / #2 (USE other_db redirect) deliberately NOT addressed
per direction "USEは拒否しない。ユーザー責任". Pre-SQL is
operator-trusted: myschema runs the SQL as-is without scrubbing,
so a stray USE, destructive DML, or session-state corruption is
the operator's responsibility. AGENTS.md gains a paragraph stating
this explicitly so future readers understand the trust model.

New regression: TestApply_PreSQLStdinConflict pins the stdin-conflict
detection. The pre-existing TestApply_PreSQLFile / FileMissing /
EmptyFile / BothSetError stay as Go tests; YAML harness still
covers the inline / failure / multi-statement paths.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reverses round-2's stdin-conflict detection in favour of an
outright rejection of --pre-sql-file=-. Allowing stdin for both
pre-SQL and desired-SQL files invites the second-read-hits-EOF
silent-truncation footgun the previous commit was trying to catch
— if the only legitimate use of --pre-sql-file=- is "I have a
short SQL snippet to inject", --pre-sql / MYSCHEMA_PRE_SQL covers
that case without the conflict surface.

Changes:
- loadPreSQL: errors with "stdin not supported" when preFile is `-`.
- loadPreSQL: signature reverts to taking just PreSQLOption (no
  desiredFiles parameter — stdin conflict check is no longer needed).
- pre_sql.go: drops parser.ReadSQLFile dependency, switches back to
  os.ReadFile (stdin support was the only reason to use the former).
- AGENTS.md / --pre-sql-file help: documents stdin as unsupported.
- Test: TestApply_PreSQLStdinConflict → TestApply_PreSQLFileStdinRejected,
  asserts the error message contains "stdin".

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread apply_test.go Outdated
Comment thread options.go
…comment

PR #75 round-3 review:

1. TestApply_PreSQLFile's docstring still claimed the file path was
   read via parser.ReadSQLFile and that stdin (-) "would work in
   principle". Both are no longer true — loadPreSQL switched back
   to os.ReadFile and rejects `-` outright. Re-word the comment to
   point at TestApply_PreSQLFileStdinRejected for the stdin pin.

2. PreSQLOption's kong tags (env vars + xor:"pre-sql" group) were
   not directly exercised — only the runtime check in loadPreSQL
   was. A future tag typo (e.g. mismatched xor group names on the
   two fields) would silently disable the CLI guard. Add two new
   options_test.go cases mirroring the existing
   TestDropPolicyKongParse* shape:

   - TestPreSQLOptionKongXor: passing both flags at the CLI must
     fail at parse time.
   - TestPreSQLOptionKongAcceptsEither: 3 sub-tests
     (--pre-sql alone, --pre-sql-file alone, neither set) confirm
     the xor group doesn't over-restrict.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@winebarrel winebarrel merged commit f1e4219 into main May 5, 2026
9 checks passed
@winebarrel winebarrel deleted the add-pre-sql-option branch May 5, 2026 02:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants