Skip to content

Make update-deps.py generate bzlmod-compatible output#5691

Merged
npaun merged 6 commits intomainfrom
npaun/draft-bzlmod-update-deps
Dec 18, 2025
Merged

Make update-deps.py generate bzlmod-compatible output#5691
npaun merged 6 commits intomainfrom
npaun/draft-bzlmod-update-deps

Conversation

@npaun
Copy link
Copy Markdown
Member

@npaun npaun commented Dec 13, 2025

What this PR does:

  1. Edit our update-deps.py script to generate output compatible with bzlmod instead of workspace, for our dependencies that are just http_archive. For example, dragonbox, ncrypto, simdutf.
  2. Put a bunch of dependencies that got frozen back under automatic management.

What we can get Claude to do for us:

  • Support dependencies that are taken directly from BCR. For example, ada-url, brotli.
  • Support BCR dependencies with http_archive overrides. For example, sqlite3, highway.

To make it easier for our robot buddy to do this, I've added an unused function called get_bcr_version that I wrote back in August, as a starting point.

With these two elements added, we can add all of our dependency management workflows as of today.

Not addressed in this PR: our "shared_deps" system where the same version is used in edgeworker and workerd needs to be redone. With bzlmod we cannot load these things across repos, though I suspect we can with symlinks. We might need some help here. At worst we can just break the link between these versions and manage them independently.

Future work: I am still not sure if we want this, but it's certainly doable: Instead of using the separate jsonc files and generating Bazel code, we could parse the MODULE.bazel files. The potential advantages are a bit less conceptual complexity and getting ready for renovate when DEVTOOLS makes it available on GitHub. This can be figured out later and doesn't block any of this work.

@npaun npaun requested review from danlapid and fhanau December 13, 2025 07:12
@github-actions
Copy link
Copy Markdown

github-actions bot commented Dec 13, 2025

The generated output of @cloudflare/workers-types matches the snapshot in types/generated-snapshot 🎉

@npaun npaun force-pushed the npaun/draft-bzlmod-update-deps branch from 79a8e25 to 66356e2 Compare December 13, 2025 07:14
Copy link
Copy Markdown
Contributor

@fhanau fhanau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, this looks like it mostly solves our bzlmod auto-update conundrum.
I believe the PR description is partially obsolete since this is sufficient to have auto-updates for bzlmod dependencies, as long as we specify them in json (which works with some of the most important ones like bzlmod). For things already in MODULE.bazel, we could move them back to bzlmod, or implement an improved update mechanism to support them directly, but both should be done in a separate PR.
You should manually delete the old *.bzl auto-generated files since the script no longer cleans them up after this, but otherwise this LGTM.

@npaun
Copy link
Copy Markdown
Member Author

npaun commented Dec 15, 2025

Marking as "ready to review" as there's only a couple of minor things left to fix then we can land this.

@npaun npaun marked this pull request as ready for review December 15, 2025 04:14
@npaun npaun requested review from a team as code owners December 15, 2025 04:14
@fhanau
Copy link
Copy Markdown
Contributor

fhanau commented Dec 15, 2025

Marking as "ready to review" as there's only a couple of minor things left to fix then we can land this.

Still LGTM, just need to address the remaining comments.

@npaun npaun force-pushed the npaun/draft-bzlmod-update-deps branch 4 times, most recently from b096d19 to 18ba84e Compare December 16, 2025 18:31
@npaun
Copy link
Copy Markdown
Member Author

npaun commented Dec 16, 2025

I also have to port over the internal repo before we can merge this

@npaun npaun force-pushed the npaun/draft-bzlmod-update-deps branch 2 times, most recently from 0390949 to ec57240 Compare December 17, 2025 21:40
@jasnell jasnell added the build label Dec 17, 2025
@npaun npaun force-pushed the npaun/draft-bzlmod-update-deps branch from ec57240 to 62cbffc Compare December 17, 2025 22:30
@npaun npaun enabled auto-merge (squash) December 17, 2025 23:48
@npaun npaun force-pushed the npaun/draft-bzlmod-update-deps branch from 76f5cbf to 158af70 Compare December 17, 2025 23:49
@npaun
Copy link
Copy Markdown
Member Author

npaun commented Dec 18, 2025

CI is slower than a sloth on ambien, but I swear I'm merging this

@npaun npaun merged commit 2fbab14 into main Dec 18, 2025
20 of 21 checks passed
@npaun npaun deleted the npaun/draft-bzlmod-update-deps branch December 18, 2025 00:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants