feat(rustdoc-json): Add optional support for rkyv (de)serialization#153283
Open
LukeMathWalker wants to merge 1 commit intorust-lang:mainfrom
Open
feat(rustdoc-json): Add optional support for rkyv (de)serialization#153283LukeMathWalker wants to merge 1 commit intorust-lang:mainfrom
LukeMathWalker wants to merge 1 commit intorust-lang:mainfrom
Conversation
Collaborator
|
rustdoc-json-types is a public (although nightly-only) API. If possible, consider changing |
Collaborator
|
|
This comment has been minimized.
This comment has been minimized.
6cc4c8b to
e357584
Compare
|
|
||
| #[cfg(feature = "rkyv_0_8")] | ||
| mod rkyv { | ||
| use std::fmt::Debug; |
Member
There was a problem hiding this comment.
These tests don't run. When I applied
diff --git a/src/rustdoc-json-types/tests.rs b/src/rustdoc-json-types/tests.rs
index e878350e43b..258c22304c3 100644
--- a/src/rustdoc-json-types/tests.rs
+++ b/src/rustdoc-json-types/tests.rs
@@ -41,6 +41,11 @@ fn test_union_info_roundtrip() {
#[cfg(feature = "rkyv_0_8")]
mod rkyv {
+ #[test]
+ fn definenly_fails() {
+ panic!("at least the rkyv tests were ran");
+ }
+
use std::fmt::Debug;
use rkyv::Archive;Running ./x test ./src/rustdoc-json-types/ still passed.
The fix (I think) is to enable this feature in bootsrap:
diff --git a/src/bootstrap/src/core/build_steps/test.rs b/src/bootstrap/src/core/build_steps/test.rs
index 88f10775333..ab1d2b8a24b 100644
--- a/src/bootstrap/src/core/build_steps/test.rs
+++ b/src/bootstrap/src/core/build_steps/test.rs
@@ -3302,7 +3302,7 @@ fn run(self, builder: &Builder<'_>) {
builder.kind,
"src/rustdoc-json-types",
SourceType::InTree,
- &[],
+ &["rkyv_0_8".to_owned()],
);
// FIXME: this looks very wrong, libtest doesn't accept `-C` arguments and the quotes are fishy.(CC @jieyouxu, is this ok to do?)
Contributor
Author
There was a problem hiding this comment.
Apologies, I had only tested the crate directly via local cargo test, under the implicit assumption that the testing infrastructure would automatically pick up feature flags for matrix testing.
I've added the feature flag to the bootstrap script, let me know if other changes are needed.
Member
|
@rustbot author |
Collaborator
|
Reminder, once the PR becomes ready for a review, use |
Contributor
Author
|
@rustbot ready |
This comment has been minimized.
This comment has been minimized.
aDotInTheVoid
approved these changes
Mar 7, 2026
Member
There was a problem hiding this comment.
Thanks, I hope this helps perf.
r=me with commits squashed, and when someone from T-bootstrap signs off that 39f7f0d is ok.
40d35e9 to
1d81c50
Compare
Contributor
Author
|
Commits have been squashed @aDotInTheVoid. What's the best way to get a reviewer from T-bootstrap (or is your tag in the thread enough)? |
Member
|
I asked on zulip (#t-infra/bootstrap > Review Request: Adding a feature in a test step.). Seems fine. (Sorry, I should've made that clear here). @bors r+ rollup |
Contributor
aDotInTheVoid
added a commit
to aDotInTheVoid/rustdoc-types-contrib
that referenced
this pull request
Mar 9, 2026
Adds support for rust-lang/rust#153283
JonathanBrouwer
added a commit
to JonathanBrouwer/rust
that referenced
this pull request
Mar 9, 2026
…=aDotInTheVoid feat(rustdoc-json): Add optional support for rkyv (de)serialization ## Motivation The JSON documents produced by `rustdoc-json` are _big_. More often than not, tools need to access a small fraction of that output—e.g. a couple of types from a transitive dependency, or a subset of the fields on a given `rustdoc-json-types` type. Using a binary (de)serialization format and a cache helps to drive down the performance cost of deserialization: you invoke `rustdoc-json` to get the JSON output you need, re-serialize it using a more perfomant format as target (e.g. `bincode` or `postcard`) and thus amortize the cost of future queries that hit the persistent cache rather than `rustdoc-json`. This is _better_, but still not great: the deserialization cost for crates like `std` still shows up prominently in flamegraphs. ## An Alternative Approach: rkyv `rkyv` provides a different opportunity: you avoid paying the deserialization cost _upfront_ thanks to [zero-copy deserialization](https://rkyv.org/zero-copy-deserialization.html). You're often able to determine if you need a certain entry from the JSON document using the archived version of that type, thus incurring the full deserialization cost only for the subset of items you actually need ([example](LukeMathWalker/pavex@d067e7e)). ## The Change This PR adds support for `rkyv` behind a feature flag (`rkyv_0_8`). For most types, it's a straight-forward `derive(rkyv::Archive, rkyv::Serialize, rkyv::Deserialize)` annotation. For co-recursive types, we need to adjust the generated bounds, using the techniques from [`rkyv`'s JSON example](https://github.com/rkyv/rkyv/blob/985b0230a0b9cb9fce4a4ee9facb6af148e27c8e/rkyv/examples/json_like_schema.rs). I have added new round-trip tests to ensure `rkyv` works as expected. r? @aDotInTheVoid
aDotInTheVoid
added a commit
to aDotInTheVoid/rustdoc-types-contrib
that referenced
this pull request
Mar 9, 2026
Adds support for rust-lang/rust#153283
aDotInTheVoid
added a commit
to aDotInTheVoid/rustdoc-types-contrib
that referenced
this pull request
Mar 9, 2026
Adds support for rust-lang/rust#153283
rust-bors bot
pushed a commit
that referenced
this pull request
Mar 9, 2026
…uwer Rollup of 5 pull requests Successful merges: - #153283 (feat(rustdoc-json): Add optional support for rkyv (de)serialization) - #153608 (ast_passes: unsupported arch w/ scalable vectors) - #153616 (Update `sysinfo` version to `0.38.4`) - #153619 (Update books) - #153624 (Ping fmease on parser modifications)
Contributor
Contributor
|
This pull request was unapproved. This PR was contained in a rollup (#153625), which was unapproved. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
The JSON documents produced by
rustdoc-jsonare big. More often than not, tools need to access a small fraction of that output—e.g. a couple of types from a transitive dependency, or a subset of the fields on a givenrustdoc-json-typestype.Using a binary (de)serialization format and a cache helps to drive down the performance cost of deserialization: you invoke
rustdoc-jsonto get the JSON output you need, re-serialize it using a more perfomant format as target (e.g.bincodeorpostcard) and thus amortize the cost of future queries that hit the persistent cache rather thanrustdoc-json.This is better, but still not great: the deserialization cost for crates like
stdstill shows up prominently in flamegraphs.An Alternative Approach: rkyv
rkyvprovides a different opportunity: you avoid paying the deserialization cost upfront thanks to zero-copy deserialization.You're often able to determine if you need a certain entry from the JSON document using the archived version of that type, thus incurring the full deserialization cost only for the subset of items you actually need (example).
The Change
This PR adds support for
rkyvbehind a feature flag (rkyv_0_8).For most types, it's a straight-forward
derive(rkyv::Archive, rkyv::Serialize, rkyv::Deserialize)annotation. For co-recursive types, we need to adjust the generated bounds, using the techniques fromrkyv's JSON example.I have added new round-trip tests to ensure
rkyvworks as expected.r? @aDotInTheVoid