Skip to content

CFSQL-1589 migrations_pattern for configuring recursive migration discovery #14089

Merged
alsuren merged 20 commits into
mainfrom
dlaban/d1-migrations-pattern
Jun 3, 2026
Merged

CFSQL-1589 migrations_pattern for configuring recursive migration discovery #14089
alsuren merged 20 commits into
mainfrom
dlaban/d1-migrations-pattern

Conversation

@alsuren

@alsuren alsuren commented May 28, 2026

Copy link
Copy Markdown
Contributor

Fixes #13257 (CFSQL-1589).

Adds a migrations_pattern config field to D1 bindings so users can
opt their D1 migrations into a nested layout (drizzle's
migrations/*/migration.sql, prisma-style folders, custom
extensions, …) instead of the implicit migrations/*.sql. Without
migrations_pattern, behaviour is unchanged.

Also includes a commit to fix our logging, because it was causing a bunch of test failures as I added more tests (best reviewed with whitespace hidden: 54d293f?w=1 .


A picture of a cute animal (not mandatory, but encouraged)

 ,_     _
 |\\_,-~/
 / _  _ |    ,--.
(  @  @ )   / ,-'
 \  _T_/-._( (
 /         `. \
|         _  \ |
 \ \ ,  /      |
  || |-_\__   /
 ((_/`(____,-'

-- https://www.asciiart.eu/art/64248696ab474307

executeSql and the d1 execute command lower logger.loggerLevel to
"error" when run with --json, so that human-readable logs don't end
up in the JSON output. Previously the level was only restored on the
happy path, so any early return or thrown error left the singleton
logger muted — silencing later logger.warn/log calls (notably from
migration helpers that wrap executeSql and are commonly mocked in
tests).

Wrap the log-level swap in try/finally so the level is always restored.

Also stop using a literal /tmp/my-migrations-go-here path in two
migrate tests; they were leaking state between runs and depending on
the muted-logger bug above. They now build the migrations dir under
the per-test runInTempDir() cwd.
@changeset-bot

changeset-bot Bot commented May 28, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: f9d669e

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 5 packages
Name Type
wrangler Minor
@cloudflare/workers-utils Minor
@cloudflare/vite-plugin Patch
@cloudflare/vitest-pool-workers Patch
@cloudflare/cli-shared-helpers Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions

github-actions Bot commented May 28, 2026

Copy link
Copy Markdown
Contributor

✅ All changesets look good

@ask-bonk

ask-bonk Bot commented May 28, 2026

Copy link
Copy Markdown
Contributor

UnknownError: ProviderInitError

github run

@ask-bonk

ask-bonk Bot commented May 28, 2026

Copy link
Copy Markdown
Contributor

@alsuren Bonk workflow failed. Check the logs for details.

View workflow run · To retry, trigger Bonk again.

@alsuren alsuren force-pushed the dlaban/d1-migrations-pattern branch from 5b8ccb5 to bd8db0b Compare May 28, 2026 18:07
@pkg-pr-new

pkg-pr-new Bot commented May 28, 2026

Copy link
Copy Markdown
create-cloudflare

npm i https://pkg.pr.new/create-cloudflare@14089

@cloudflare/deploy-helpers

npm i https://pkg.pr.new/@cloudflare/deploy-helpers@14089

@cloudflare/kv-asset-handler

npm i https://pkg.pr.new/@cloudflare/kv-asset-handler@14089

miniflare

npm i https://pkg.pr.new/miniflare@14089

@cloudflare/pages-shared

npm i https://pkg.pr.new/@cloudflare/pages-shared@14089

@cloudflare/unenv-preset

npm i https://pkg.pr.new/@cloudflare/unenv-preset@14089

@cloudflare/vite-plugin

npm i https://pkg.pr.new/@cloudflare/vite-plugin@14089

@cloudflare/vitest-pool-workers

npm i https://pkg.pr.new/@cloudflare/vitest-pool-workers@14089

@cloudflare/workers-editor-shared

npm i https://pkg.pr.new/@cloudflare/workers-editor-shared@14089

@cloudflare/workers-utils

npm i https://pkg.pr.new/@cloudflare/workers-utils@14089

wrangler

npm i https://pkg.pr.new/wrangler@14089

@cloudflare/wrangler-bundler

npm i https://pkg.pr.new/@cloudflare/wrangler-bundler@14089

commit: f9d669e

Adds a `migrations_pattern` config field to D1 bindings so users can
opt their D1 migrations into a nested layout (drizzle's
`migrations/*/migration.sql`, prisma-style folders, custom
extensions, …) instead of the implicit `migrations/*.sql`. Without
`migrations_pattern`, behaviour is unchanged.

Rules
-----

When `migrations_pattern` is set:
  - `migrations_dir` must also be set explicitly (no implicit default).
  - The pattern must literally start with `${migrations_dir}/`. This
    keeps the relationship between the two settings obvious and makes
    "name relative to migrations_dir" deterministic via prefix-strip.

Names written to the `d1_migrations` table are always relative to
`migrations_dir` (with forward-slash separators), so the table stays
portable across machines regardless of how the user wrote their
config.

Implementation
--------------

  - `resolveMigrationsConfig({ databaseInfo, configPath })` returns
    a single `MigrationsConfig` object with every field defaulted
    and normalized. Called once per command (`apply`, `list`,
    `create`) and threaded through the helpers. "Parse, don't
    validate" — the only way to get a `MigrationsConfig` is via the
    resolver, and a `MigrationsConfig` is by construction valid.

  - `getMigrationNames` walks `projectPath/migrationsDir` with a
    `Minimatch` for the pattern. Uses minimatch's `partial: true`
    mode to prune subdirectory descents (so a `*.sql` pattern never
    recurses, `*/migration.sql` only descends one level, `**/*.sql`
    recurses unconditionally — patterns opt into walking everything).
    This keeps the cost bounded even if a user has a
    `migrations/node_modules` sitting around.

  - `compareMigrationPaths` is the named ordering: numeric on the
    first path segment's leading integer, lex tiebreaker on the full
    path, numbered files before unnumbered. Preserves the old
    inconsistent-padding behaviour (`1_a, 9_b, 10_c` still sort
    numerically).

  - `getNextMigrationNumber` delegates to `getMigrationNames`, so
    it respects `migrations_pattern` and counts numbered directories
    (drizzle-style) alongside numbered files.

  - When `migrations_pattern` matches nothing but
    `${migrations_dir}/*/migration.sql` (drizzle's layout) matches
    files on disk, the command logs a hint pointing the user at the
    right pattern.

  - `wrangler d1 migrations create` rejects with an actionable error
    before touching the filesystem if the file it would write
    wouldn't match `migrations_pattern` (the migrations dir is no
    longer created as a side effect of a failing command).

Tests
-----

  - 49 unit tests in `helpers.test.ts` covering walk, pattern
    matching, ordering, drizzle-hint output, prune behaviour,
    absolute paths (POSIX + Windows-only), and the
    `resolveMigrationsConfig` rejection paths.

  - 25 end-to-end tests in `migrate.test.ts` covering all three
    commands' interaction with `migrations_pattern`, including
    inline-snapshot tests of every user-facing error and warning.

  - Tests live in the describe block matching their subcommand
    (`create` / `apply` / `list`), not a separate
    `migrations_pattern` block.

CFSQL-1589
@alsuren alsuren force-pushed the dlaban/d1-migrations-pattern branch from bd8db0b to 9bdc7ba Compare May 28, 2026 19:19
@alsuren alsuren changed the title Dlaban/d1 migrations pattern CFSQL-1589 migrations_pattern for configuring recursive migration discovery May 28, 2026
ask-bonk[bot]

This comment was marked as resolved.

@ask-bonk

ask-bonk Bot commented May 28, 2026

Copy link
Copy Markdown
Contributor

Review posted successfully. Here's a summary of what I found:

PR #14089 adds migrations_pattern to D1 bindings for nested migration layouts and fixes a logger-level leak bug. The overall design is solid — validation, file walking with glob-based pruning, and numeric sorting all look correct.

One actionable issue found: getNextMigrationNumber at helpers.ts:417 duplicates the parsing logic from leadingMigrationNumber but omits the radix 10 from parseInt. I posted a suggestion to reuse the existing leadingMigrationNumber helper instead.

github run

Bonk suggestion

Co-authored-by: ask-bonk[bot] <249159057+ask-bonk[bot]@users.noreply.github.com>
@alsuren alsuren marked this pull request as ready for review May 29, 2026 07:49
@alsuren alsuren requested a review from workers-devprod as a code owner May 29, 2026 07:49
@alsuren alsuren requested a review from a team May 29, 2026 07:50
@workers-devprod workers-devprod requested review from a team and james-elicx and removed request for a team May 29, 2026 07:50
@workers-devprod

workers-devprod commented May 29, 2026

Copy link
Copy Markdown
Contributor

Codeowners approval required for this PR:

  • @cloudflare/wrangler
  • ✅ @cloudflare/d1
Show detailed file reviewers
  • .changeset/d1-execute-logger-level.md: [@cloudflare/wrangler]
  • .changeset/d1-migrations-pattern.md: [@cloudflare/wrangler]
  • packages/workers-utils/src/config/environment.ts: [@cloudflare/wrangler]
  • packages/workers-utils/src/config/validation.ts: [@cloudflare/wrangler]
  • packages/workers-utils/src/worker.ts: [@cloudflare/wrangler]
  • packages/wrangler/src/tests/d1/migrate.test.ts: [@cloudflare/wrangler]
  • packages/wrangler/src/tests/d1/migrations/helpers.test.ts: [@cloudflare/wrangler]
  • packages/wrangler/src/tests/d1/utils.test.ts: [@cloudflare/wrangler]
  • packages/wrangler/src/d1/execute.ts: [@cloudflare/wrangler]
  • packages/wrangler/src/d1/migrations/apply.ts: [@cloudflare/wrangler]
  • packages/wrangler/src/d1/migrations/create.ts: [@cloudflare/wrangler]
  • packages/wrangler/src/d1/migrations/helpers.ts: [@cloudflare/wrangler]
  • packages/wrangler/src/d1/migrations/list.ts: [@cloudflare/wrangler]
  • packages/wrangler/src/d1/types.ts: [@cloudflare/wrangler]
  • packages/wrangler/src/d1/utils.ts: [@cloudflare/wrangler]

devin-ai-integration[bot]

This comment was marked as resolved.

Comment thread packages/workers-utils/src/config/environment.ts
Comment thread packages/workers-utils/src/config/environment.ts Outdated
Comment thread packages/wrangler/src/d1/migrations/helpers.ts Outdated
Comment thread packages/wrangler/src/d1/migrations/helpers.ts Outdated
Comment thread packages/wrangler/src/d1/migrations/helpers.ts
Comment thread packages/wrangler/src/d1/migrations/helpers.ts Outdated
Comment thread packages/wrangler/src/d1/migrations/helpers.ts
Comment thread packages/wrangler/src/d1/migrations/helpers.ts
Comment thread packages/wrangler/src/__tests__/d1/migrations/helpers.test.ts
@FlorentCollin

Copy link
Copy Markdown
Member

It was not easy to review, but it's absolutely not your fault. Thanks for taking the time to write it - I really like the direction.

alsuren added a commit to cloudflare/cloudflare-docs that referenced this pull request May 29, 2026
Adds documentation for the new `migrations_pattern` D1 binding
config field landing in wrangler PR
cloudflare/workers-sdk#14089.

Updates:

  - `d1/reference/migrations.mdx` \u2014 `migrations_pattern` shown
    in the example config, plus a new "Nested migration layouts"
    section that explains the default behaviour, shows the
    Drizzle-style nested example, lists the rules (`migrations_dir`
    required, pattern must start with `${migrations_dir}/`,
    recorded names are relative to `migrations_dir`), and notes
    that `wrangler d1 migrations create` only writes top-level
    files so ORM users should use their ORM's generate command.

  - `workers/wrangler/configuration.mdx` \u2014 `migrations_pattern`
    bullet next to the existing `migrations_dir` bullet in the D1
    bindings reference.
@alsuren alsuren enabled auto-merge (squash) June 1, 2026 14:03
Comment thread packages/wrangler/src/d1/migrations/helpers.ts Outdated
Comment thread packages/wrangler/src/d1/execute.ts
Comment thread packages/wrangler/src/d1/migrations/helpers.ts Outdated
Comment thread packages/wrangler/src/d1/migrations/helpers.ts
Comment thread packages/wrangler/src/d1/migrations/helpers.ts
devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

@alsuren alsuren requested a review from james-elicx June 2, 2026 17:34
@workers-devprod

Copy link
Copy Markdown
Contributor

Codeowners approval required for this PR:

  • ✅ @cloudflare/d1
  • ✅ @cloudflare/wrangler
Show detailed file reviewers

@workers-devprod workers-devprod left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codeowners reviews satisfied

@alsuren alsuren merged commit c6c61b5 into main Jun 3, 2026
56 checks passed
@github-project-automation github-project-automation Bot moved this from Untriaged to Approved in workers-sdk Jun 3, 2026
@alsuren alsuren deleted the dlaban/d1-migrations-pattern branch June 3, 2026 09:12
@github-project-automation github-project-automation Bot moved this from Approved to Done in workers-sdk Jun 3, 2026
alsuren added a commit to cloudflare/cloudflare-docs that referenced this pull request Jun 5, 2026
Adds documentation for the new `migrations_pattern` D1 binding
config field landing in wrangler PR
cloudflare/workers-sdk#14089.

Updates:

  - `d1/reference/migrations.mdx` \u2014 `migrations_pattern` shown
    in the example config, plus a new "Nested migration layouts"
    section that explains the default behaviour, shows the
    Drizzle-style nested example, lists the rules (`migrations_dir`
    required, pattern must start with `${migrations_dir}/`,
    recorded names are relative to `migrations_dir`), and notes
    that `wrangler d1 migrations create` only writes top-level
    files so ORM users should use their ORM's generate command.

  - `workers/wrangler/configuration.mdx` \u2014 `migrations_pattern`
    bullet next to the existing `migrations_dir` bullet in the D1
    bindings reference.
elithrar pushed a commit to cloudflare/cloudflare-docs that referenced this pull request Jun 8, 2026
Adds documentation for the new `migrations_pattern` D1 binding
config field landing in wrangler PR
cloudflare/workers-sdk#14089.

Updates:

  - `d1/reference/migrations.mdx` \u2014 `migrations_pattern` shown
    in the example config, plus a new "Nested migration layouts"
    section that explains the default behaviour, shows the
    Drizzle-style nested example, lists the rules (`migrations_dir`
    required, pattern must start with `${migrations_dir}/`,
    recorded names are relative to `migrations_dir`), and notes
    that `wrangler d1 migrations create` only writes top-level
    files so ORM users should use their ORM's generate command.

  - `workers/wrangler/configuration.mdx` \u2014 `migrations_pattern`
    bullet next to the existing `migrations_dir` bullet in the D1
    bindings reference.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

D1 migrations_dir | Support recursive .sql file search

5 participants