[bazel] Move to rules_rs for crate management#6125
Conversation
|
All contributors have signed the CLA ✍️ ✅ |
|
I have read the CLA Document and I hereby sign the CLA |
26941d3 to
dfd14a8
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #6125 +/- ##
==========================================
- Coverage 70.80% 70.70% -0.11%
==========================================
Files 424 424
Lines 115290 115450 +160
Branches 18751 18770 +19
==========================================
- Hits 81629 81626 -3
- Misses 22429 22590 +161
- Partials 11232 11234 +2 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
018135b to
355efd0
Compare
863b156 to
5857a4a
Compare
|
|
||
| crate = use_extension("@rules_rs//rs:extensions.bzl", "crate") | ||
| crate.annotation( | ||
| additive_build_file_content = """exports_files(["include/lol_html.h"])""", |
There was a problem hiding this comment.
nit: It's easier to define a new target so that we don't need to manually import the lol_html_c_api repo – the branch I have just uses an annotation with extra_aliased_targets = {"lol_html_c_api_include_lol_html_h": "include/lol_html.h"} for that and updates the header target in deps/rust/BUILD.lolhtml. This will require a change in the downstream project, I can help with that later
There was a problem hiding this comment.
Ended up doing this myself in #6302 – if that ends up landing first this PR will need some rebasing, but it makes lol-html much easier to maintain so it's something I wanted to do anyway.
There was a problem hiding this comment.
Ok no problem! I've been focusing on rules_rs fixes for Patrick anyway and then the plan is workerdcxx so this will need a rebase either way:)
| name = "crates_vendor", | ||
| cargo_lock = "//deps/rust:Cargo.lock", | ||
| cargo_toml = "//deps/rust:Cargo.toml", | ||
| platform_triples = RUST_TARGET_TRIPLES, |
There was a problem hiding this comment.
Does rules_rs have the equivalent of strip_internal_dependencies_from_cargo_lockfile here? Not a big change but would make Cargo.lock slightly cleaner.
There was a problem hiding this comment.
rules_rs doesn't have that, it was only added to rules_rust to try to make cargo splicing run less often, because it was a pain for other projects. But the folks who added it at Zipline are also using rules_rs now :)
but also how would strip_internal_dependencies_from_cargo_lockfile help? My understanding was it removed dependencies between first-party crates, when your Cargo.toml represents a workspace with multiple crates in it. That's not the setup here, so I think the lockfile is already refelcting only third-party deps. Am I missing something?
There was a problem hiding this comment.
When I tried this with rules_rust's from_cargo, it ended up removing the direct-cargo-bazel-deps package definition from Cargo.lock – not a big change, but cleaner than before.
| deps = [ | ||
| "//src/rust/cxx-integration", | ||
| "@crates_vendor//:encoding_rs", | ||
| "@crates_vendor//:encoding_rs-0.8.35", |
There was a problem hiding this comment.
This looks odd, and will break upon the next deps update – we should explicitly depend on encoding_rs in Cargo.toml instead, looks like that was missed when copying the dependencies over.
There was a problem hiding this comment.
ack, will fix. I think the deps have also updated a bit since I generated the list, so I'll re-sync them and also carry over the comments/etc
|
|
||
| # equivalent to `cargo update`; use `workspace` or <package> to limit update scope | ||
| update-rust package="full": | ||
| # cd deps/rust && bazel run @rules_rust//tools/upstream_wrapper:cargo -- update |
There was a problem hiding this comment.
nit: Let's actually replace the old command instead of just adding a comment with the new one.
There was a problem hiding this comment.
ah yes, this was a reminder for me to ask yall to double check this does what you expect, can you please check :)
| path = "fake.rs" | ||
|
|
||
| [dependencies] | ||
| # Keep explicit features minimal to bound compile times and binary size. |
There was a problem hiding this comment.
nit: In Cargo.toml, let's copy existing comments instead of rephrasing them. If there turn out to be cases where the comment is outdated/misleading, we can change those.
| lol_html_c_api = { git = "https://github.com/cloudflare/lol-html", tag = "v2.7.1" } | ||
| nix = "0" | ||
| pico-args = "0" | ||
| proc-macro2 = { version = "1", features = ["span-locations"] } |
There was a problem hiding this comment.
This is newly added here – let's add a comment stating that this is for workerd-cxx since it is not used directly in workerd. Ideally, let's keep workerd-cxx dependencies separately at the end of the list so that we can more easily keep track of them.
There was a problem hiding this comment.
Also, to facilitate usage of this list in the internal version and avoid duplicating this list: Is there a way to have Cargo.toml include another Cargo.toml file so that common dependencies can be shared between these? I guess we could set up a crate with the given Cargo.toml file and depend on that crate but that hardly sounds like the most elegant solution.
There was a problem hiding this comment.
I can rearrange workerd-cxx bits to the end + add an explanation. I think the more proper fix is to make workerd's Cargo.toml depend on workerd-cxx's Cargo.toml as a crate, but I think that might require making similar Cargo.toml-ification change to workerd-cxx, so we can either fix that repo first and then circle back to this one, or do this hack as a stopgap, which would you prefer
There was a problem hiding this comment.
Yeah, transitioning to rules_rs in workerd-cxx first might be the better approach
There was a problem hiding this comment.
@fhanau OK, lets give that a shot. Do I need to be added as collaborator there as well so I can trigger CI?
There was a problem hiding this comment.
I'd imagine that you already have the required permissions, otherwise I should be able to run the CI too
There was a problem hiding this comment.
@fhanau looks like I don't have the permissions, mind unblocking cloudflare/workerd-cxx#96 and adding me as collaborator there so I can iterate over the weekend if needed?
There was a problem hiding this comment.
Done, will start looking at that PR soon.
|
|
||
| # rules_rust | ||
| bazel_dep(name = "rules_rust", version = "0.68.1") | ||
| bazel_dep(name = "rules_rs", version = "0.0.40") |
There was a problem hiding this comment.
not actionable, I'm just curious: What's the best way to see what rules_rust version a given rules_rs version maps to?
There was a problem hiding this comment.
https://github.com/dzbarsky/rules_rs/blob/c5ae2dbfd8171d6c973a5bdc426f8bd1b295c0fe/rs/experimental/rules_rust.bzl#L32-L39 contains the pin, https://github.com/dzbarsky/rules_rust tracks upstream very closely (i'm usually rebasing every few days) but carries a lot of fixes on top as well. Not sure if that answers your question?
|
Thanks for the review! I'll update over the weekend, cheers |
6e24811 to
0dad305
Compare
|
|
||
| [dependencies] | ||
| # Import the correct versions of dependencies from workerd-cxx | ||
| workerd-cxx = { package = "third-party", git = "https://github.com/cloudflare/workerd-cxx.git", rev = "c54ffa14aa9f0fc43e701f0ed01f1680be154be2" } |
There was a problem hiding this comment.
This will make workerd-cxx updates inconvenient as they will no longer work through the auto-update mechanism when there are crate dependency changes
@mikea can you think of a better approach here?
|
I started porting this approach to the downstream repo. To make that easier, can you rebase this PR on main, use the latest commit on cloudflare/workerd-cxx#97 and update to rules_rs 0.44.0 here? The latter might require rebasing the patch, we'll need extra_aliased_targets support there too |
|
Landed as part of #6643, which gets rid of the crate files – we're using plain rules_rust for now, unfortunately I kept running into cases where rules_rs is incompatible which makes using it difficult (might revisit in the future, but for now we're mostly interested in not having the vendored crate files, which is easier to do with just rules_rust) |
No description provided.