Skip to content

feat!: align with Iceberg REST OpenAPI spec for v1#48

Merged
mandarini merged 2 commits into
mainfrom
feat/iceberg-v1
May 18, 2026
Merged

feat!: align with Iceberg REST OpenAPI spec for v1#48
mandarini merged 2 commits into
mainfrom
feat/iceberg-v1

Conversation

@mandarini

Copy link
Copy Markdown
Collaborator

Summary

Aligns the client with the Apache Iceberg REST Catalog OpenAPI spec ahead of the 1.0.0 release, fixes #32 by replacing the static catalogName flow with the spec-recommended warehouse -> GET /v1/config -> prefix flow, and adds a spec-conformance test layer so we cannot silently drift from the spec going forward.

What changed

Spec alignment (breaking)

  • listNamespaces and listTables now take an options bag and return paginated objects:
    • listNamespaces({ parent?, pageToken?, pageSize? }) -> { namespaces, nextPageToken? }
    • listTables(namespace, { pageToken?, pageSize? }) -> { identifiers, nextPageToken? }
  • updateTable body is the spec-aligned { requirements?, updates } shape. The legacy { properties } / { schema } shorthand is replaced by TableUpdate variants (e.g. { action: 'set-properties', updates: {...} }).
  • commitTable is added as a spec-named alias.

/v1/config + warehouse (closes #32)

  • New warehouse constructor option. On first use the client calls GET /v1/config?warehouse=... once and uses the server-returned overrides.prefix (falling back to defaults.prefix, then to the warehouse string itself if /config is unavailable). Memoized per instance.
  • catalogName stays as a permanent alias for warehouse so existing call sites (notably @supabase/storage-js's StorageAnalyticsClient) keep compiling.
  • New public loadConfig() accessor returns the cached CatalogConfig.

Transport features

  • Idempotency-Key (UUIDv7, generated client-side) is emitted on every POST/DELETE mutation: createNamespace, dropNamespace, updateNamespaceProperties, createTable, updateTable, dropTable.
  • Conditional loadTable(id, { ifNoneMatch }) returns null on 304. Overloaded so loadTable(id) keeps returning a non-nullable TableMetadata.
  • New loadTableResult(id, options?) returns the full spec-shaped LoadTableResult (metadata, metadata-location, config, storage-credentials) plus the captured ETag so callers can feed it back into a future ifNoneMatch.

Type fidelity

  • Full discriminated unions for TableUpdate (23 actions: assign-uuid, upgrade-format-version, add-schema, set-current-schema, add-spec, set-default-spec, add-sort-order, set-default-sort-order, add-snapshot, set-snapshot-ref, remove-snapshots, remove-snapshot-ref, set-location, set-properties, remove-properties, set-statistics, remove-statistics, set-partition-statistics, remove-partition-statistics, remove-partition-specs, remove-schemas, add-encryption-key, remove-encryption-key) and TableRequirement (8 assertions).
  • New types: CatalogConfig, StorageCredential, Snapshot, SnapshotReference, StatisticsFile, PartitionStatisticsFile, EncryptedKey, BlobMetadata, LoadTableResult, LoadTableResultWithEtag, RegisterTableRequest, RenameTableRequest, UpdateNamespacePropertiesRequest, UpdateNamespacePropertiesResponse, ListNamespacesOptions, ListNamespacesResult, ListTablesOptions, ListTablesResult, LoadTableOptions.
  • All re-exported from src/index.ts.

Spec-conformance test layer (test/spec-conformance/)

Three layers verify alignment from different angles:

  1. Path/method matrix (paths.test.ts): exercises every public method against a mock fetch and asserts the emitted (METHOD, path) matches a real spec operation. Also asserts every mutation request carries an Idempotency-Key UUIDv7 header.
  2. Ajv runtime body validation (schemas.test.ts): loads the bundled OpenAPI YAML, registers every component schema with Ajv, and validates request bodies and response fixtures against the live schemas.
  3. openapi-typescript type assignability (types.test.ts): generated types from the YAML are asserted against handwritten types via expectTypeOf().toExtend() so the type checker catches drift.

CI runs pnpm gen:spec-types before pnpm type-check and pnpm test, so a spec change immediately surfaces as a type error or a failing schema/path test.

Docs and examples

  • README rewritten: new constructor options, paginated method docs, ETag/If-None-Match section, expanded TypeScript Types export list, and a "Migrating from 0.x to 1.0" table at the top.
  • examples/basic-usage.ts and examples/supabase-storage-usage.ts updated for the paginated return shape.

Why

The 0.x client diverged from the spec in several places that mattered: namespace/table listings did not surface nextPageToken so any catalog with more than a page of entries silently truncated; Idempotency-Key was never sent so retries could double-write; and the catalogName flow could not address per-warehouse catalogs (Cloudflare R2, Tabular) that need server-side prefix resolution. Issue #32 surfaced the warehouse case from a real user; the rest of the gaps were caught while doing the audit. Cutting 1.0 is a chance to fix all of these together with one well-publicized break, and the conformance test layer keeps us honest from this point on.

Breaking changes

  • listNamespaces and listTables return { items, nextPageToken? } instead of arrays. Callers must destructure or read the .namespaces / .identifiers property.
  • listNamespaces no longer takes a parent identifier directly; it takes { parent, pageToken?, pageSize? }.
  • updateTable body is { requirements?, updates }; the old { properties } shorthand is gone (use { updates: [{ action: 'set-properties', updates: {...} }] }).

catalogName is preserved as a permanent alias for warehouse, and loadTable(id) (no options) keeps its non-null TableMetadata return via TS overloads, to keep the downstream blast radius tight. Confirmed audit of supabase, storage, and supabase-js: only @supabase/storage-js's StorageAnalyticsClient is affected, and the migration there is a single proxy-wrapper update.

Test plan

  • Unit tests pass (pnpm test, 93 tests across 7 files including the new spec-conformance suite)
  • Type check clean (pnpm type-check)
  • Lint clean (pnpm lint)
  • Build succeeds for ESM and CJS with .d.ts (pnpm build)
  • Integration tests pass against tabulario/iceberg-rest (pnpm test:integration:ci, 20/20)
  • Compatibility tests pass for ESM, CJS, TS-ESM, TS-CJS (pnpm test:compatibility)
  • Manual verification of warehouse -> /v1/config -> server-returned prefix flow via the new unit tests in test/catalog/IcebergRestCatalog.test.ts

@mandarini mandarini requested review from a team as code owners May 7, 2026 16:43
@mandarini mandarini merged commit 943aaf3 into main May 18, 2026
4 checks passed
@mandarini mandarini deleted the feat/iceberg-v1 branch May 18, 2026 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request]: Support 'warehouse' property in REST Catalog client for Data Location Override from the server

2 participants