Skip to content

Commit c447e9d

Browse files
alexcrichtonalexcrichton
committed
Add better diagnostics for rebuilding
This commit overhauls how a `Fingerprint` is stored on the filesystem and in-memory to help provide much better diagnostics as to why crates are being rebuilt. This involves storing more structured data on the filesystem in order to have a finer-grained comparison with the previous state. This is not currently surfaced in the output of cargo and still requires `RUST_LOG=cargo::ops::cargo_rustc::fingerprint=info` but if it turns out to be useful we can perhaps surface the output. There are performance considerations here to ensure that a noop build is still quite speedy for a few reasons: 1. JSON decoding is slow (these are just big structures to decode) 2. Each fingerprint stores all recursive fingerprints, so we can't just "vanilla decode" as it would decode O(n^2) items 3. Hashing is actually somewhat nontrivial for this many items here and there, so we still need as much memoization as possible. To ensure that builds are just as speedy tomorrow as they are today, a few strategies are taken: * The same fingerprint strategy is used today as a "first line of defense" where a small text file with a string contains the "total fingerprint" hash. A separately stored file then contains the more detailed JSON structure of the old fingerprint, and that's only decoded if there's a mismatch of the short hashes. The rationale here is that most crates don't need to be rebuilt so we shouldn't decode JSON, but if it does need to be rebuilt then the work of compiling far dwarfs the work of decoding the JSON. * When encoding a full fingerprint as JSON we don't actually include any dependencies, just the resolved u64 of them. This helps the O(n^2) problem in terms of decoding time and storage space on the filesystem. * Short hashes continue to be memoized to ensure we don't recompute a hash if we've already done so (e.g. shared dependencies). Overall, when profiling with Servo, this commit does not regress noop build times, but should help diagnose why crates are being rebuilt hopefully! Closes #2011
1 parent 8524efb commit c447e9d

File tree

5 files changed

+244
-75
lines changed

5 files changed

+244
-75
lines changed

‎Cargo.lock‎

Lines changed: 3 additions & 3 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎src/cargo/ops/cargo_rustc/context.rs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ pub struct Context<'a, 'cfg: 'a> {
3333
pub compilation: Compilation<'cfg>,
3434
pub build_state: Arc<BuildState>,
3535
pub exec_engine: Arc<Box<ExecEngine>>,
36-
pub fingerprints: HashMap<Unit<'a>, Fingerprint>,
36+
pub fingerprints: HashMap<Unit<'a>, Arc<Fingerprint>>,
3737
pub compiled: HashSet<Unit<'a>>,
3838
pub build_config: BuildConfig,
3939
pub build_scripts: HashMap<Unit<'a>, Arc<BuildScripts>>,

0 commit comments

Comments
 (0)