Skip to content

v0.7.0: COUNT(*) on empty table returns NULL instead of 0 #468

@vieux

Description

@vieux

What happened?

Starting in v0.7.0, SELECT COUNT(*) against an empty table is returned to the Go BigQuery client as a NULL column instead of the typed INT64 value 0. This breaks every consumer that scans the result into a non-pointer int64:

bigquery: NULL cannot be assigned to field `N` of type int64

Scanning into bigquery.NullInt64 confirms the emulator is actually returning a NULL value ({Int64: 0, Valid: false}), not just a different schema mode.

The same query against v0.6.6 returns 0 correctly.

What did you expect to happen?

SELECT COUNT(*) should return INT64 0 for an empty input, matching v0.6.6 and real BigQuery.

COUNT(*) is non-nullable by the BigQuery spec — it always returns INT64, never NULL, even on empty input. Real BigQuery additionally returns the column mode as REQUIRED.

How can we reproduce it (as minimally and precisely as possible)?

Start both emulator versions side-by-side:

docker run --rm -d --name bq-emu-066 -p 9050:9050 ghcr.io/goccy/bigquery-emulator:0.6.6 --project=test
docker run --rm -d --name bq-emu-070 -p 9051:9050 ghcr.io/goccy/bigquery-emulator:0.7.0 --project=test

main.go:

package main

import (
	"context"
	"fmt"
	"log"
	"time"

	"cloud.google.com/go/bigquery"
	"google.golang.org/api/iterator"
	"google.golang.org/api/option"
)

func main() {
	ctx := context.Background()
	client, err := bigquery.NewClient(
		ctx, "test",
		option.WithEndpoint("http://localhost:9051"), // 9050 for v0.6.6
		option.WithoutAuthentication(),
		option.WithScopes(bigquery.Scope),
	)
	if err != nil {
		log.Fatalf("NewClient: %v", err)
	}
	defer client.Close()

	ds := client.Dataset(fmt.Sprintf("repro_%d", time.Now().UnixMicro()))
	if err := ds.Create(ctx, nil); err != nil {
		log.Fatalf("Dataset.Create: %v", err)
	}
	schema, _ := bigquery.InferSchema(struct {
		Name string `bigquery:"name"`
	}{})
	if err := ds.Table("empty").Create(ctx, &bigquery.TableMetadata{Schema: schema}); err != nil {
		log.Fatalf("Table.Create: %v", err)
	}

	q := client.Query(fmt.Sprintf("SELECT COUNT(*) AS n FROM `test.%s.empty`", ds.DatasetID))
	it, err := q.Read(ctx)
	if err != nil {
		log.Fatalf("Read: %v", err)
	}

	var row struct {
		N int64 `bigquery:"n"`
	}
	if err := it.Next(&row); err != nil && err != iterator.Done {
		fmt.Printf("FAIL: %v\n", err)
		return
	}
	fmt.Printf("OK: n = %d\n", row.N)
}

Run against each emulator by pointing option.WithEndpoint at :9050 (v0.6.6) and then :9051 (v0.7.0).

Output:

v0.6.6 → OK: n = 0
v0.7.0 → FAIL: bigquery: NULL cannot be assigned to field `N` of type int64

Anything else we need to know?

Scope of the regression. I tested a few related shapes to narrow it down:

  • SELECT COUNT(*) FROM <empty>NULL on v0.7.0 ❌
  • SELECT COUNT(*) FROM <table> WHERE FALSENULL on v0.7.0 ❌ (same root cause — empty input to the aggregate)
  • SELECT COALESCE(SUM(1), 0) FROM <empty>0 on v0.7.0 ✅ (COALESCE happens to win)

So this looks specific to COUNT rather than a generic empty-aggregate-row issue.

Likely root cause. The v0.7.0 release notes mention "Emit an explicit NULLABLE mode on query result fields" (#448). My guess is that the new googlesqlite-backed cursor unconditionally marks aggregate output columns as NULLABLE and, for COUNT(*) specifically, encodes "no input rows" as NULL instead of 0. Real BigQuery returns COUNT's output column as REQUIRED INT64 with value 0.

Impact. Anyone scanning a COUNT(*) result into Go's int64 (or Java long, Python int, etc.) breaks on upgrade to v0.7.0 unless they pre-emptively switch to nullable types. Real BigQuery has never required that. We hit this in CI when upgrading to v0.7.0 and worked around it by switching the result struct to bigquery.NullInt64 and treating NULL as 0.

Versions.

  • cloud.google.com/go/bigquery v1.74.0
  • ghcr.io/goccy/bigquery-emulator:0.7.0 (broken)
  • ghcr.io/goccy/bigquery-emulator:0.6.6 (works)

Happy to send a PR if you can point me at where the empty-aggregate-row branch is computed in the new googlesqlite backend.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions