Implement sql-pipe: CSV-to-SQLite query CLI in Zig by Copilot · Pull Request #1 · vmvarela/sql-pipe

Copilot · 2026-03-02T03:54:18Z

Implements a zero-dependency CLI that pipes CSV from stdin into an in-memory SQLite database and executes a user-provided SQL query, printing results to stdout.

New files

src/main.zig — Core implementation:
- GeneralPurposeAllocator for memory; @cImport(@cInclude("sqlite3.h")) for SQLite bindings
- Reads CSV header line → dynamically generates CREATE TABLE t (col1 TEXT, col2 TEXT, ...)
- All row inserts wrapped in a single BEGIN TRANSACTION/COMMIT; statement prepared once via sqlite3_prepare_v2, values bound per row with sqlite3_bind_text using SQLITE_TRANSIENT
- Query executed via sqlite3_exec with a C callback printing comma-separated rows to stdout
- All resources freed via defer
build.zig — Zig 0.15 build using b.createModule() API; links sqlite3 + libc
build.zig.zon — Package manifest (fingerprint required by Zig 0.15)
.gitignore — Excludes .zig-cache/ and zig-out/

Usage

echo 'name,age
Alice,30
Bob,25' | sql-pipe 'SELECT name FROM t WHERE CAST(age AS INT) > 27'
# Alice

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

ziglang.org
- Triggering command: /usr/bin/curl curl -s REDACTED (dns block)
- Triggering command: /usr/bin/wget wget -q REDACTED -O zig.tar.xz (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Original prompt

Write a highly optimized CLI tool in Zig called sql-pipe. The program should read CSV data from stdin, load it into an in-memory SQLite database, execute a user-provided SQL query passed as a CLI argument, and print the results to stdout.
Please follow these exact steps and constraints:

Setup & Imports: Use std.heap.GeneralPurposeAllocator for memory management. Import SQLite directly using @cImport({ @cInclude("sqlite3.h"); }).

CLI Arguments: Parse the command-line arguments. The first argument after the executable name will be the SQL query. If it's missing, print usage instructions and exit gracefully.

Database Initialization: Open a new SQLite database strictly in memory using sqlite3_open(":memory:", &db).

Schema Generation: Read the first line from stdin (assume it contains comma-separated headers). Dynamically construct and execute a CREATE TABLE t (...) statement where the columns are the headers from the CSV.

Data Ingestion (Batched): Read the rest of stdin line by line. Insert the data into table t. Crucial for performance: Wrap all insertions inside a single transaction (BEGIN TRANSACTION and COMMIT). Prepare the INSERT statement once using sqlite3_prepare_v2 and bind the values for each row using sqlite3_bind_text to avoid SQL injection and parsing overhead.

Query Execution & Output: Execute the user-provided SQL query against the database using sqlite3_exec. Implement a callback function that prints the results to stdout, formatting the output with a comma or tab separator.

Cleanup: Ensure all memory allocations, prepared statements, and the SQLite database are properly freed and closed using defer.

Write the code using modern Zig idioms (compatible with Zig 0.12+ / 0.13+).

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Co-authored-by: vmvarela <11040851+vmvarela@users.noreply.github.com>

Copilot

Pull request overview

Adds sql-pipe, a Zig CLI that ingests CSV from stdin into an in-memory SQLite database and runs a user-provided SQL query, printing results to stdout.

Changes:

Implemented CSV ingestion → dynamic CREATE TABLE → batched INSERT via prepared statement → query execution via sqlite3_exec callback.
Added Zig build configuration linking against system sqlite3 and libc.
Added package manifest and basic build output cache ignores.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 9 comments.

File	Description
src/main.zig	Core CLI implementation: CSV→SQLite ingestion, batched inserts, query execution, stdout printing
build.zig	Zig build script creating the executable and linking `sqlite3` + libc
build.zig.zon	Package manifest (name/version/fingerprint/min Zig version)
.gitignore	Ignores Zig build/cache output directories

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/main.zig

build.zig.zon

Copilot · 2026-03-02T04:14:08Z

src/main.zig

+        try sql.appendSlice(allocator, "CREATE TABLE t (");
+        for (cols.items, 0..) |col, i| {
+            if (i > 0) try sql.appendSlice(allocator, ", ");
+            try sql.appendSlice(allocator, col);
+            try sql.appendSlice(allocator, " TEXT");
+        }


CSV headers are appended directly into the CREATE TABLE statement without quoting/escaping as identifiers. This breaks on headers with spaces/reserved words and enables SQL injection via crafted header values. Quote identifiers (e.g., double quotes) and escape embedded quotes, and reject empty/invalid column names before building SQL.

Copilot · 2026-03-02T04:14:08Z

src/main.zig

+        var errmsg: [*c]u8 = null;
+        if (c.sqlite3_exec(db, sql.items.ptr, null, null, &errmsg) != c.SQLITE_OK) {
+            std.debug.print("CREATE TABLE failed: {s}\n", .{std.mem.span(errmsg)});
+            c.sqlite3_free(errmsg);


If sqlite3_exec fails, errmsg is not guaranteed to be non-null; calling std.mem.span(errmsg) will crash when SQLite doesn't provide an error string. Guard for null (and still return a useful message, e.g., using sqlite3_errmsg(db)).

src/main.zig

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

…ase drafter, workflows)

- Quote CSV column names as SQL identifiers ("col") in CREATE TABLE to prevent SQL injection and support headers with spaces/reserved words - Guard errmsg null check before std.mem.span in CREATE TABLE error path, falling back to sqlite3_errmsg(db) for diagnostics - Fix broken insert loop: remove duplicated variable declarations, add missing sqlite3_step call with error handling, use SQLITE_STATIC (safe since step is called before buffer is freed) - Replace SQLITE_TRANSIENT with SQLITE_STATIC (null) to work around Zig 0.15 alignment constraint on function pointer casts from usize

…ated tools)

- build.zig: -Dbundle-sqlite=true compiles sqlite3.c from lib/ enabling full cross-compilation with no system dependencies - ci.yml: build + integration test on ubuntu/macos/windows - release.yml: 16-target release matrix — Linux: x86_64, aarch64, armv7, riscv64, x86 (musl/static) macOS: x86_64, aarch64 Windows: x86_64, aarch64, x86 (MinGW) FreeBSD: x86_64, aarch64 NetBSD: x86_64, aarch64 OpenBSD: x86_64 DragonFly: x86_64 Attaches binaries + sha256sums.txt to published releases - .gitignore: exclude lib/ and sqlite.zip - README: update build instructions; remove system sqlite3 limitation

Initial plan

86d40b8

Copilot AI assigned Copilot and vmvarela Mar 2, 2026

Copilot started work on behalf of vmvarela March 2, 2026 03:54 View session

Implement sql-pipe CLI tool in Zig

6fddf50

Co-authored-by: vmvarela <11040851+vmvarela@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Add CLI tool sql-pipe for CSV to SQLite querying~~ Implement sql-pipe: CSV-to-SQLite query CLI in Zig Mar 2, 2026

Copilot finished work on behalf of vmvarela March 2, 2026 04:07

vmvarela approved these changes Mar 2, 2026

View reviewed changes

vmvarela marked this pull request as ready for review March 2, 2026 04:10

Copilot AI review requested due to automatic review settings March 2, 2026 04:10

Copilot started reviewing on behalf of vmvarela March 2, 2026 04:10 View session

Copilot AI reviewed Mar 2, 2026

View reviewed changes

vmvarela and others added 8 commits March 2, 2026 05:27

Update src/main.zig

5383d93

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/main.zig

b1723b2

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/main.zig

62fb896

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/main.zig

e72191e

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/main.zig

e3f661e

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/main.zig

5c94c37

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update build.zig.zon

548ef7f

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

chore: add GitHub Scrum scaffolding (.github templates, labeler, rele…

d38102b

…ase drafter, workflows)

github-actions bot added type:feature New functionality type:chore Maintenance, refactoring, tooling type:docs Documentation only labels Mar 2, 2026

vmvarela added 4 commits March 2, 2026 12:32

chore: rename breaking label to type:breaking for consistency

dd61d59

docs: rewrite README with pragmatic style (examples, limitations, rel…

5da9ccb

…ated tools)

vmvarela merged commit 281004f into master Mar 2, 2026
3 of 5 checks passed

vmvarela deleted the copilot/create-sql-pipe-cli-tool branch March 2, 2026 12:04

vmvarela mentioned this pull request Mar 14, 2026

feat(packaging): add Nix derivation and flake #56

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement sql-pipe: CSV-to-SQLite query CLI in Zig#1

Implement sql-pipe: CSV-to-SQLite query CLI in Zig#1
vmvarela merged 14 commits intomasterfrom
copilot/create-sql-pipe-cli-tool

Copilot AI commented Mar 2, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Mar 2, 2026

Uh oh!

Copilot AI Mar 2, 2026

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Copilot AI commented Mar 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

New files

Usage

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Mar 2, 2026 •

edited

Loading