Release Notes

Added

• Expanded benchmark suite with 4 new competitor tools:

  • mysqldbsplit (PHP) - Now the fastest tool at 1238 MB/s throughput
  • mysqldump-splitter (Rust/Scoopit) - Hierarchical output with gzip support
  • mysql-dump-splitter (Go/Bekkema) - Include/exclude table filtering
  • extract-mysql-dump (Python/agroff) - Multi-database extraction
  • Total of 10 tools now benchmarked in Docker suite

• JSON schema generation: Auto-generate JSON schemas from Rust types using schemars

  • New schema subcommand to export schemas for all JSON output formats
  • Schemas available for: analyze, convert, graph, merge, redact, sample, shard, split, validate

• Website OG image generation: Dynamic Open Graph images using Satori

  • Automatic OG image generation for documentation pages
  • IndexNow integration for faster search engine indexing

• Profiling and benchmark recipes: New justfile recipes for performance analysis

  • just profile-medium / just profile-large for CPU profiling
  • just bench-baseline for criterion benchmarks

Changed

• Performance: Use stack-allocated buffers in parser to reduce heap allocations
• Error handling: Improved error handling in validation, sample, and compression modules
• Dependencies: Updated multiple dependencies (chrono 0.4.43, jsonschema 0.40, schemars 1.2, clap, serde_json, flate2, indicatif, dirs, rustyline, rand, fake)

Fixed

• CI disk space: Resolved disk space exhaustion in GitHub Actions builds
• Compression: Return Result from wrap_reader for better error handling

Install sql-splitter 1.13.0

Install prebuilt binaries via shell script

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/helgesverre/sql-splitter/releases/download/v1.13.0/sql-splitter-installer.sh | sh

Install prebuilt binaries via powershell script

powershell -ExecutionPolicy Bypass -c "irm https://github.com/helgesverre/sql-splitter/releases/download/v1.13.0/sql-splitter-installer.ps1 | iex"

Install prebuilt binaries via Homebrew

brew install helgesverre/tap/sql-splitter

Download sql-splitter 1.13.0

Added

• Full INSERT/COPY statement rewriting for redact command: The redact command now actively modifies SQL data

  • Previously, redaction strategies were identified but statements passed through unchanged
  • INSERT statements are now parsed, values redacted according to matching strategies, and reconstructed
  • PostgreSQL COPY data blocks are now parsed and redacted with proper \N NULL markers
  • Multi-row INSERT statements are fully supported
  • All 7 redaction strategies now work: null, constant, hash, mask, fake, shuffle, skip

• Dialect-aware value formatting for redact command: Proper SQL escaping per dialect

  • MySQL: Backtick quoting, backslash escape sequences (\', \n, \r, \t)
  • PostgreSQL: Double-quote identifiers, '' escaping, \N for NULL in COPY format
  • SQLite: Double-quote identifiers, '' escaping
  • MSSQL: Square bracket identifiers [column], N'...' for Unicode strings

• ValueRewriter module: New src/redactor/rewriter.rs for INSERT/COPY transformation

  • Converts ParsedValue to RedactValue, applies strategy, formats back to SQL
  • Handles PostgreSQL's two-step COPY pattern (header + data block)
  • Proper stats tracking during actual redaction

• 20 new redact integration tests:

  • 7 MSSQL-specific tests: null, hash, fake, dry-run, bracket quoting, Unicode strings, reproducibility
  • 13 cross-dialect tests: MySQL INSERT, PostgreSQL COPY, SQLite, multi-row, escaping, skip tables

Changed

• Redact command now reports accurate statistics: Row and column counts reflect actual redaction performed

Full Changelog: v1.12.5...v1.12.6

MSSQL Production-Style Output for Test Data Generator

Added

• MSSQL production-style output for test data generator: New CLI flags for production-style MSSQL output

  • --go-separator: Use GO batch separators instead of semicolons
  • --schema-prefix: Use [dbo]. schema prefix on all table references
  • --named-constraints: Use named CONSTRAINT syntax (e.g., CONSTRAINT [PK_users] PRIMARY KEY CLUSTERED)
  • --mssql-production: Enable all three options at once for production-like output
  • RenderConfig::mssql_production() method for programmatic use

• 9 new MSSQL generator integration tests: Comprehensive tests for generated MSSQL fixtures

  • Tests for analyze, split, validate, merge roundtrip, sample command
  • Tests for deterministic generation and production-style output
  • All 38 MSSQL integration tests now pass

Changed

• Updated TEST_DATA_GENERATOR.md documentation: Added MSSQL dialect examples and updated dialect count

Example Usage

# Generate MSSQL with production-style output
gen-fixtures --dialect mssql --scale small --mssql-production

# Or enable individual options
gen-fixtures --dialect mssql --scale medium --go-separator --schema-prefix --named-constraints

Full Changelog: v1.12.4...v1.12.5

Added

• MSSQL dialect support for test data generator: The gen-fixtures CLI and test_data_gen crate now support MSSQL dialect
  • Square bracket [identifier] quoting for table and column names
  • INT IDENTITY(1,1) for auto-increment primary keys
  • NVARCHAR(n) and NVARCHAR(MAX) for Unicode string columns
  • N'...' prefix for Unicode string literals
  • DATETIME2 for timestamp columns
  • BIT for boolean columns
  • DECIMAL(10,2) for monetary values
  • Proper SET ANSI_NULLS ON, SET QUOTED_IDENTIFIER ON, SET NOCOUNT ON header
  • Both streaming mode (--rows) and multi-tenant schema mode (--scale) supported

Usage:

gen-fixtures --dialect mssql --scale small
gen-fixtures --dialect mssql --rows 10000 --tables 5

Full Changelog: v1.12.3...v1.12.4

Fixed

• SQLite AUTOINCREMENT stripping for DuckDB: SQLite dumps with AUTOINCREMENT now import correctly
  • Previously, tables with INTEGER PRIMARY KEY AUTOINCREMENT failed to create in DuckDB
  • Now strips AUTOINCREMENT, WITHOUT ROWID, and STRICT table modifiers
  • SQLite dumps now import all tables (was failing on most tables before)

Added

• New SQLite DuckDB tests: Added 2 tests for SQLite-specific syntax stripping

Full Changelog: v1.12.2...v1.12.3

Full Changelog: v1.12.1...v1.12.2

Fixed

• PostgreSQL COPY performance: Fixed severe performance regression in query command when processing PostgreSQL dumps with COPY data blocks

  • Processing time for pagila-data.sql reduced from 15-27 seconds to ~0.03 seconds
  • Full pagila dump now processes in ~1.9 seconds (previously could hang or take minutes)

• Single-column COPY data: Fixed bug where PostgreSQL COPY data with single columns (no tabs) was incorrectly identified as non-COPY data

• COPY schema prefix handling: Fixed "table does not exist" errors when COPY statements used schema-qualified names (COPY public.tablename)

Added

• 16 new PostgreSQL COPY tests: Comprehensive edge case coverage for COPY blocks with comments, schema prefixes, escape sequences, unicode, and more

• Stress tests: 50k row COPY imports, multi-table stress tests, 10k row INSERT tests

Performance

PostgreSQL dumps are now ~6x faster to query than MySQL/SQLite:

Query Command: SQL Analytics on Dump Files

Run SQL queries directly on dump files using embedded DuckDB — no database required!

Features

• Zero dependencies: DuckDB is bundled and compiled into sql-splitter
• Multi-dialect support: MySQL, PostgreSQL, and SQLite dumps
• Output formats: table (ASCII), JSON, JSONL, CSV, TSV
• Interactive REPL: .tables, .schema, .count, .sample, .export
• Persistent caching: 400x speedup on repeated queries
• Memory management: Auto disk mode for dumps >2GB
• Table filtering: Import only specific tables for faster startup

Usage Examples

# Single query
sql-splitter query dump.sql "SELECT COUNT(*) FROM users"

# Export to JSON
sql-splitter query dump.sql "SELECT * FROM orders" -f json -o results.json

# Interactive REPL
sql-splitter query dump.sql --interactive

# With caching (fast repeated queries)
sql-splitter query dump.sql "SELECT ..." --cache

REPL Commands

• .tables — List all tables
• .schema [table] — Show schema
• .describe <table> — Describe table columns
• .count <table> — Count rows
• .sample <table> [n] — Sample rows
• .format <fmt> — Set output format
• .export <file> <query> — Export results

Cache Management

sql-splitter query --list-cache   # List cached databases
sql-splitter query --clear-cache  # Clear all caches

Testing

• 119 new DuckDB tests covering edge cases and real-world patterns
• Real-world verification tests across MySQL, PostgreSQL, and SQLite dumps

Full Changelog: v1.11.0...v1.12.0

What's New

Graph Command

Generate Entity-Relationship Diagrams (ERD) from SQL dumps:

sql-splitter graph dump.sql -o schema.html        # Interactive HTML
sql-splitter graph dump.sql -o schema.dot         # Graphviz DOT
sql-splitter graph dump.sql -o schema.mmd         # Mermaid erDiagram
sql-splitter graph dump.sql --json                # JSON with full schema

Features:

• ERD-style diagrams showing tables with columns, types, PK/FK markers, nullability
• Interactive HTML with dark/light mode toggle, copy Mermaid button, panzoom
• Table filtering with --tables and --exclude glob patterns
• Focus mode: --table X --transitive (dependencies) or --reverse (dependents)
• Cycle detection using Tarjan's SCC algorithm (--cycles-only)
• Auto-render to PNG/SVG/PDF via Graphviz (--render)
• Tested with large schemas (281 tables, 3104 columns)

Order Command

Reorder SQL dumps in topological FK order for safe imports:

sql-splitter order dump.sql -o ordered.sql        # Safe import order
sql-splitter order dump.sql --check               # Check for cycles
sql-splitter order dump.sql --reverse             # For DROP operations

Installation

cargo install sql-splitter

See CHANGELOG.md for full details.

What's Changed

• Fix mobile code overflow and light/dark mode color variables by @Copilot in #10
• docs: extend roadmap with graph, migrate, parallel, and infer features by @HelgeSverre in #11

New Contributors

• @Copilot made their first contribution in #10
• @HelgeSverre made their first contribution in #11

Full Changelog: v1.10.0...v1.11.0