Skip to content

Increase maxBuffer for capnp upload#10217

Merged
petebacondarwin merged 2 commits intocloudflare:mainfrom
veggiedefender:main
Aug 14, 2025
Merged

Increase maxBuffer for capnp upload#10217
petebacondarwin merged 2 commits intocloudflare:mainfrom
veggiedefender:main

Conversation

@veggiedefender
Copy link
Copy Markdown
Contributor

@veggiedefender veggiedefender commented Aug 4, 2025 •
edited
Loading

Wrangler breaks when trying to compile a capnp schema large enough that the output is bigger than 1 MB.

In my wrangler.toml I have:

[unsafe.capnp]
base_path = "./schemas/"
source_schemas = ["message.capnp"]

Attempting to deploy the toml results in this error:

✘ [ERROR] spawnSync capnp ENOBUFS

And this stack trace:

Error: spawnSync capnp ENOBUFS
    at Object.spawnSync (node:internal/child_process:1120:20)
    at spawnSync (node:child_process:878:24)
    at handleUnsafeCapnp (REDACTED/node_modules/wrangler/wrangler-dist/cli.js:70137:65)
    at createWorkerUploadForm (REDACTED/node_modules/wrangler/wrangler-dist/cli.js:70564:25)
    at deploy (REDACTED/node_modules/wrangler/wrangler-dist/cli.js:157736:22)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async Object.handler (REDACTED/node_modules/wrangler/wrangler-dist/cli.js:85848:66)
    at async Object.handler (REDACTED/node_modules/wrangler/wrangler-dist/cli.js:82691:7)

This happens because spawnSync has a default buffer size of 1024 * 1024. The internal data team's message.capnp compiles to a little over a megabyte :(

capnp compile -o- schemas/message.capnp | wc -c
 1139904

There's likely a better long term fix (async collecting a stream) but bumping it to 10 MB seems like a fine quick fix.

  • Tests
    • Tests included
    • Tests not necessary because: it's a small option tweak..
  • Public documentation
    • Cloudflare docs PR(s):
    • Documentation not necessary because: this is an undocumented unsafe feature
  • Wrangler V3 Backport

@veggiedefender veggiedefender requested a review from a team as a code owner August 4, 2025 20:08
@changeset-bot
Copy link
Copy Markdown

changeset-bot bot commented Aug 4, 2025 •
edited
Loading

🦋 Changeset detected

Latest commit: 5247394

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages
Name Type
wrangler Patch
@cloudflare/vite-plugin Patch
@cloudflare/vitest-pool-workers Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-project-automation github-project-automation bot moved this to Untriaged in workers-sdk Aug 4, 2025
@pkg-pr-new
Copy link
Copy Markdown

pkg-pr-new bot commented Aug 4, 2025 •
edited
Loading

create-cloudflare

npm i https://pkg.pr.new/create-cloudflare@10217

@cloudflare/kv-asset-handler

npm i https://pkg.pr.new/@cloudflare/kv-asset-handler@10217

miniflare

npm i https://pkg.pr.new/miniflare@10217

@cloudflare/pages-shared

npm i https://pkg.pr.new/@cloudflare/pages-shared@10217

@cloudflare/unenv-preset

npm i https://pkg.pr.new/@cloudflare/unenv-preset@10217

@cloudflare/vite-plugin

npm i https://pkg.pr.new/@cloudflare/vite-plugin@10217

@cloudflare/vitest-pool-workers

npm i https://pkg.pr.new/@cloudflare/vitest-pool-workers@10217

@cloudflare/workers-editor-shared

npm i https://pkg.pr.new/@cloudflare/workers-editor-shared@10217

wrangler

npm i https://pkg.pr.new/wrangler@10217

commit: 5247394

vicb
vicb reviewed Aug 4, 2025
View reviewed changes
vicb
vicb reviewed Aug 4, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@vicb vicb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could please add a patch changeset (using pnpm changeset).

Could you please also describe what are the conditions to trigger this error?

Otherwise the change looks good to me (with minor comments).

Thanks!

@veggiedefender veggiedefender force-pushed the main branch 2 times, most recently from 26cdf1d to 4a49880 Compare August 4, 2025 20:49
@veggiedefender
Copy link
Copy Markdown
Contributor Author

veggiedefender commented Aug 4, 2025

Added a changeset and more details on how to trigger the error!

@veggiedefender veggiedefender requested a review from vicb August 4, 2025 20:58
@veggiedefender veggiedefender mentioned this pull request Aug 5, 2025
Merged
@petebacondarwin petebacondarwin added the skip-pr-description-validation Skip validation of the required PR description format label Aug 5, 2025
@petebacondarwin
Copy link
Copy Markdown
Contributor

petebacondarwin commented Aug 5, 2025

Skipping description validation as it doesn't seem to like the URL to the backport PR.

@veggiedefender veggiedefender force-pushed the main branch 2 times, most recently from c9d932f to 00d7c4c Compare August 5, 2025 16:32
@github-project-automation github-project-automation bot moved this from Untriaged to Approved in workers-sdk Aug 6, 2025
@petebacondarwin petebacondarwin force-pushed the main branch 3 times, most recently from 70a7ae1 to 4c4c3cf Compare August 11, 2025 11:07
@petebacondarwin petebacondarwin self-assigned this Aug 11, 2025
@petebacondarwin petebacondarwin force-pushed the main branch 2 times, most recently from 1e86b9a to 74bfafe Compare August 11, 2025 20:03
@petebacondarwin petebacondarwin merged commit 979984b into cloudflare:main Aug 14, 2025
43 of 46 checks passed
@github-project-automation github-project-automation bot moved this from Approved to Done in workers-sdk Aug 14, 2025
@petebacondarwin
Copy link
Copy Markdown
Contributor

petebacondarwin commented Aug 14, 2025

Finally got this green and landed! Thanks for your patience.

@workers-devprod workers-devprod mentioned this pull request Aug 13, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@petebacondarwin petebacondarwin petebacondarwin approved these changes

@vicb vicb Awaiting requested review from vicb

Assignees

@petebacondarwin petebacondarwin

Labels

skip-pr-description-validation Skip validation of the required PR description format

Projects

workers-sdk
Archived in project

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@veggiedefender @petebacondarwin @vicb @workers-devprod