Switch the SQL backend to googlesqlite and harden BigQuery compatibility#448
Merged
Conversation
Replace the SQL backend with the googlesqlite driver (pinned to v0.1.0) and raise the emulator's BigQuery compatibility across the board. Conformance testing: - Add a multi-client conformance suite (test/e2e) that runs the official Python, Ruby, PHP, Node.js and Java client libraries plus the bq CLI against a locally started emulator over a shared query corpus, driven by testcontainers from a dedicated integration GitHub Actions workflow. Query and result correctness: - Encode TIMESTAMP results as numeric epoch values, not datetime strings. - Emit an explicit NULLABLE mode on query result fields. - Honor maxResults paging in jobs.getQueryResults. - Expose the anonymous results table on query jobs (destinationTable). - Map NUMERIC/BIGNUMERIC to Arrow decimal types; migrate to arrow-go/v18. - Treat empty or absent numeric, temporal and string query parameters as typed NULLs. DDL, load and streaming: - Drop views with DROP VIEW, hydrate view schemas, and register both DDL-created and materialized views. - Report unknown fields in tabledata.insertAll. - Honor WRITE_TRUNCATE / WRITE_EMPTY load dispositions. - Infer the schema for autodetect CSV loads. Performance and robustness: - Batch dataset child lookups so per-request metadata cost stays constant. - Release the pooled connection when BeginTx fails. - Build without cgo (the SQL backend is pure Go); the build no longer needs clang or mold, and go.mod targets Go 1.25. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
goccy
added a commit
that referenced
this pull request
May 18, 2026
The SQL backend is now pure Go and the build no longer links native code (see #448), so the mold linker is no longer needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
goccy
added a commit
that referenced
this pull request
May 18, 2026
…461) * Refresh README for the googlesqlite backend and add a feature matrix Reflect the switch to the googlesqlite SQL backend and document, in one place, what the emulator actually supports today. - Add docs/feature-support.md: a MECE (mutually exclusive, collectively exhaustive) BigQuery feature support matrix, organized after the official BigQuery documentation and REST API reference. It covers the full REST surface, job types, table types, ingestion/export formats, the GoogleSQL surface, security, BigQuery ML, the Storage API, integrations and emulator-specific features. - Slim the README Status section down to a short overview that links to that matrix, instead of scattering partial caveats across the README. - Add a Sponsorship section adapted from the googlesqlite project, rewritten from the bigquery-emulator project's perspective, and split the former "Goals and Sponsors" heading into "Goals" and "Sponsorship". - Rename the "Google Standard SQL" section to "GoogleSQL" (its current name) and update the function coverage to googlesqlite v0.1.0's spec-driven support matrix instead of the stale "200+ functions". - Note the multi-client conformance suite under test/e2e. - Replace the cgo-only go-sqlite3 reference in the type conversion description with the generic SQLite driver wording. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci: derive the Go version from go.mod The build/test and release workflows pinned Go 1.21.5, while go.mod now declares go 1.25.0. Replace the hardcoded go-version with go-version-file: go.mod so CI tracks the module's Go version automatically, matching what the integration workflow already does. Also bump actions/setup-go to v5 (v2/v4 predate go-version-file). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci: drop the mold setup step from the release workflow The SQL backend is now pure Go and the build no longer links native code (see #448), so the mold linker is no longer needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci: check out the repo before setup-go in the test job The test job ran setup-go before checkout. That worked while the Go version was hardcoded, but go-version-file: go.mod needs go.mod to be present, so checkout must run first. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the SQL backend with the googlesqlite driver (pinned to v0.1.0)
and raises the emulator's BigQuery compatibility across the board, with a new
multi-client conformance test suite to guard it.
Conformance testing
test/e2esuite runs the official Python, Ruby, PHP, Node.js andJava client libraries, plus the
bqCLI, against a locally startedemulator over a shared query corpus (
test/e2e/cases/cases.json).integrationGitHub Actions workflow.
Query & result correctness
NULLABLEmode emitted on query result fields.maxResultspaging honored injobs.getQueryResults.destinationTable).NUMERIC/BIGNUMERICmapped to Arrow decimal types; Arrow upgraded toarrow-go/v18.typed
NULLs.DDL, load & streaming
DROP VIEW; view schemas hydrated; DDL-created andmaterialized views registered in metadata.
tabledata.insertAll.WRITE_TRUNCATE/WRITE_EMPTYload dispositions honored.Performance & robustness
BeginTxfails.Testing
go test ./...unit suite: green.test/e2econformance suite: green for all six clients.🤖 Generated with Claude Code