Skip to content

options, dump: add --split=<dir> for per-object file output#76

Merged
winebarrel merged 7 commits into
mainfrom
add-dump-split-option
May 5, 2026
Merged

options, dump: add --split=<dir> for per-object file output#76
winebarrel merged 7 commits into
mainfrom
add-dump-split-option

Conversation

@winebarrel

Copy link
Copy Markdown
Owner

Summary

Closes the TODO entry "`--split` for `dump` (one file per table / view)". Lets the operator dump a database into a directory of per-object `.sql` files instead of one concatenated stream — useful for code review (each PR shows only the changed object), git history attribution, or feeding the output to per-object tools.

Design

  • `--split=` is a single flag. Setting it switches dump to split mode; leaving it empty keeps today's concat behaviour.
  • File naming: `.sql` (flat, no `tables/` vs `views/` subdirs). MySQL tables and views share an identifier namespace, so the flat layout is unambiguous.
  • mkdir -p the destination; existing files at the same path are overwritten. Stale files from previous dumps are NOT cleaned up — split is "write this set", not rsync-style sync.
  • Filters apply: `--include` / `--exclude` pre-filter the table / view set the same way concat mode does.
  • stdout: header summary plus a `-- Wrote N file(s) to ` completion notice. The SQL body is NOT written to stdout in split mode.
  • File content: `TableToSQL` / `ViewToSQL` output, terminated with a trailing newline so `cat /*.sql` produces a clean stream.

Implementation

  • `options.go`: `SplitDir string` with help text.
  • `dump.go`: `writeDumpSplit` helper. `Dump()` returns `DumpResult` with empty SQL when `SplitDir` is set so the cmd layer can branch.
  • `cmd/command/dump.go`: skip the SQL body `fmt.Fprintln` when `SplitDir` is set; print the completion notice instead.

Test plan (11 cases)

Layer Scenario Test
Library 2 tables + 1 view → 3 files `TestDump_SplitTablesAndViews`
Library mkdir -p missing path `TestDump_SplitCreatesDir`
Library overwrite existing + leave orphan `TestDump_SplitOverwritesExistingFiles`
Library --exclude pre-filters `TestDump_SplitRespectsFilters`
Library per-file content == concat-mode `TestDump_SplitFileContentMatchesConcatMode`
Library empty schema → empty dir `TestDump_SplitEmptySchema`
cmd stdout = header + notice, no SQL `TestDump_Run_Split`

Coverage

  • `writeDumpSplit` 76.9% (remaining gap = MkdirAll / WriteFile fault-injection paths, out of scope for natural testing).
  • `cmd/command` stays 100%.
  • Root `myschema` 93.7% → 92.5% (small dilution from the new code's defensive error returns).
  • Other packages unchanged. `make lint` clean.

Doc updates

  • `AGENTS.md` "In scope (v1)" gains a paragraph for `--split=`.
  • `TODO.md`: `--split` entry removed; the now-empty "Low — CLI ergonomics" section header is dropped too.

🤖 Generated with Claude Code

Closes the TODO entry "--split for dump (one file per table /
view)". Lets the operator dump a database into a directory of
per-object .sql files instead of one concatenated stream — useful
for code review (each PR shows only the changed object), git
history attribution, or feeding the output to per-object tools.

Design:
- --split=<dir> is a single flag. Setting it switches dump to
  split mode; leaving it empty keeps today's concat behaviour.
- File naming: <name>.sql (flat, no tables/ vs views/ subdirs).
  MySQL tables and views share an identifier namespace, so the
  flat layout is unambiguous (an object name can be either, never
  both).
- mkdir -p the destination; existing files at the same path are
  overwritten. Stale files from previous dumps are NOT cleaned
  up — split is "write this set", not rsync-style sync, so the
  operator decides whether to wipe the directory first.
- Filters apply: --include / --exclude pre-filter the table /
  view set the same way concat mode does.
- stdout: header summary plus a `-- Wrote N file(s) to <dir>`
  completion notice. The SQL body is NOT written to stdout in
  split mode (the whole point — files on disk are the output).
- File content: TableToSQL / ViewToSQL output, terminated with a
  trailing newline so `cat <dir>/*.sql` produces a clean stream.

Implementation:
- options.go: SplitDir string with help text.
- dump.go: writeDumpSplit helper. Dump() returns DumpResult with
  empty SQL when SplitDir is set so the cmd layer can branch.
- cmd/command/dump.go: skip the SQL body fmt.Fprintln when
  SplitDir is set; print the completion notice instead.

Test plan (8 + 2 + 1 = 11 cases):

Library-level Go tests:
- TestDump_SplitTablesAndViews: 2 tables + 1 view → 3 files,
  with content shape spot-checked.
- TestDump_SplitCreatesDir: nested missing path → mkdir -p.
- TestDump_SplitOverwritesExistingFiles: same-name file
  overwritten; orphan file left alone.
- TestDump_SplitRespectsFilters: --exclude excludes that table's
  file from the dir.
- TestDump_SplitFileContentMatchesConcatMode: per-file content
  equals what concat mode emits for that object, trailing newline
  included; concatenated split files reproduce the concat-mode
  body modulo separators.
- TestDump_SplitEmptySchema: zero tables/views → empty dir
  (mkdir runs, no files inside).

cmd-layer:
- TestDump_Run_Split: stdout = header + `Wrote 2 file(s) to ...`
  notice, SQL body absent.

Coverage: writeDumpSplit 76.9% (remaining gap = error paths in
MkdirAll / WriteFile that need fault injection — out of scope for
natural testing). Per-package: cmd/command stays 100%; root
myschema 93.7% → 92.5% (small dilution from the new code's
defensive error returns). All other packages unchanged. make
lint clean.

Doc updates:
- AGENTS.md "In scope (v1)" gains a paragraph for --split=<dir>
  matching the existing --pre-sql / --alter-algorithm style.
- TODO.md: "--split for dump" entry removed; the now-empty
  "Low — CLI ergonomics" section header is dropped too.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings May 5, 2026 02:56
@codecov

codecov Bot commented May 5, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.33%. Comparing base (f1e4219) to head (77627e8).

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #76      +/-   ##
==========================================
+ Coverage   86.21%   86.33%   +0.12%     
==========================================
  Files          29       29              
  Lines        3140     3169      +29     
==========================================
+ Hits         2707     2736      +29     
  Misses        267      267              
  Partials      166      166              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new dump output mode that writes one .sql file per table/view into a target directory, intended to improve reviewability and tooling by avoiding a single concatenated SQL stream.

Changes:

  • Added DumpOptions.SplitDir and split-mode implementation that writes per-object SQL files and returns an empty SQL body in DumpResult.
  • Updated the dump CLI command to omit SQL output on stdout in split mode and print a completion notice instead.
  • Added split-mode library + cmd tests, and updated docs/TODO entries to reflect the new flag.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
TODO.md Removes the completed TODO entry for split dump output.
dump.go Introduces SplitDir option and writeDumpSplit implementation for per-object file output.
dump_test.go Adds unit/integration tests covering split-mode behavior (files created, filtering, overwrite, content match).
cmd/command/dump.go Changes CLI output behavior for split mode (header + “Wrote N…” only).
cmd/command/dump_test.go Adds cmd-layer test verifying stdout excludes SQL in split mode.
AGENTS.md Documents the new --split=<dir> behavior and removes outdated mention elsewhere.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread dump.go
Comment thread dump.go Outdated
- Add `name:"split"` kong tag to DumpOptions.SplitDir so the CLI flag
  surfaces as `--split=<dir>` (matches docs/PR description) instead
  of kong's auto-derived `--split-dir`. Pin with a kong-parse
  regression test.
- Defence-in-depth path-traversal guard in writeDumpSplit: reject
  table/view names that contain '/', '\', os.PathSeparator, or that
  are "" / "." / "..". MySQL itself forbids these in identifiers,
  but split must not write outside the chosen directory if a
  non-MySQL data source ever fed an unsafe name through. Cover the
  validator branches and the os.WriteFile error paths via direct
  WriteDumpSplit unit tests (export_test.go).

Pushes patch coverage on dump.go to 100% on the new helpers and
keeps writeDumpSplit at 100% — fixes the codecov/patch failure on
PR #76.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread dump.go Outdated
Comment thread dump.go Outdated
Comment thread dump.go Outdated
Comment thread dump_test.go Outdated
- Help text on --split now reads `'-- Wrote N file(s) to <dir>'`,
  matching the cmd's actual `-- Wrote ...` notice (was lowercase
  `wrote` — doc/CLI drift the review flagged).
- splitPath also rejects ':' anywhere in the name plus
  filepath.VolumeName != "". On Windows a name like `C:foo` has a
  volume part that filepath.Join would treat as absolute and
  *discard* dir; the literal ':' scan catches the shape on every
  host (filepath.VolumeName is OS-specific — Linux/macOS always
  return ""), and the VolumeName call stays as a belt-and-
  suspenders guard for any Windows shape the byte scan misses.
  New test cases cover `C:foo`, `\\server\share\x`, and
  `weird:name`, exercised on Linux CI via the explicit ':' check.
- Rewrote the comments on splitPath and TestDump_SplitRejectsUnsafeName
  so they describe what the validator actually does (filesystem-
  shaped path-traversal guards) instead of asserting MySQL's
  identifier rules. Names that merely *contain* '.' (e.g. `my.tbl`)
  are accepted — only the bare `.` / `..` segment sentinels are
  unsafe at the filesystem layer.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread dump.go
Comment thread dump.go
Comment thread dump_test.go Outdated
- writeDumpSplit error wraps now use %q instead of %s for the path
  field so whitespace, control characters, or other shell-noise in
  a directory name comes through unambiguously: `dump: mkdir
  "/tmp/has space": ...` instead of `dump: mkdir /tmp/has space: ...`.
  Three sites updated: mkdir, table-side WriteFile, view-side
  WriteFile. Existing assertions on substrings ("mkdir",
  "dump: write", "v.sql", "t.sql") survive — `%q` wraps the value
  in quotes but the path content stays in the rendered error.
- TestDump_SplitFlagName splits its two Parse() calls into separate
  t.Run subtests, each with its own kong.Parser and destination
  struct. kong's Parse mutates the destination, and reusing one
  parser across two Parse() calls is technically supported today
  but couples the test to that semantic. Mirroring the pre-sql
  tests' fresh-parser-per-case pattern keeps the regression test
  robust against a future kong upgrade that tightens the contract.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread dump.go
Comment thread dump_test.go Outdated
Comment thread cmd/command/dump_test.go Outdated
winebarrel and others added 2 commits May 5, 2026 17:06
- --split help text now mentions the rejected character set ("Object
  names containing path separators or ':' are rejected at write
  time"). MySQL backtick-quotes accept identifiers like `foo:bar`
  that splitPath then refuses; the help text steers users with such
  schemas to concat dump (or to rename the table) instead of
  failing without a hint.
- All split-mode tests now build paths with filepath.Join — both
  the per-package dump_test.go and the cmd/command/dump_test.go.
  The string-concat form `dir+"/x.sql"` is non-portable on Windows
  and inconsistent with the rest of the repo. 17 sites converted
  in dump_test.go, 1 in cmd/command/dump_test.go.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous fixup added "Object names containing path separators
or ':' are rejected at write time." to the --split help, addressing
a Copilot review suggestion. The help line was already verbose; the
extra sentence didn't justify the cost. The reject set is still
documented in dump.go's splitPath doc comment for code readers.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread dump.go
Comment thread dump.go
Comment thread dump.go
Copilot review noted that strings.ContainsRune(name, os.PathSeparator)
is strictly redundant: os.PathSeparator is always '/' on Unix and
'\' on Windows, both already in strings.ContainsAny(name, "/\\:").
Drop the redundant ContainsRune call and update the splitPath doc
comment to match. Behaviour unchanged.

The `:` and filepath.VolumeName guards stay — they cover the
Windows volume-escape vector that the literal '/' / '\' branch
doesn't catch.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread dump.go
Comment thread dump.go
Comment thread dump.go

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@winebarrel winebarrel merged commit cdd3769 into main May 5, 2026
13 checks passed
@winebarrel winebarrel deleted the add-dump-split-option branch May 5, 2026 08:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants