-
-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Have our cake and eat it too derivation metadata #10780
Copy link
Copy link
Open
Labels
breakingChanges we can't make without breaking old expressions, changing hashes, etcChanges we can't make without breaking old expressions, changing hashes, etcderivation designIssues to consider for new versions of the derivation format (major or incremental)Issues to consider for new versions of the derivation format (major or incremental)featureFeature request or proposalFeature request or proposalstoreIssues and pull requests concerning the Nix storeIssues and pull requests concerning the Nix store
Description
Metadata
Metadata
Assignees
Labels
breakingChanges we can't make without breaking old expressions, changing hashes, etcChanges we can't make without breaking old expressions, changing hashes, etcderivation designIssues to consider for new versions of the derivation format (major or incremental)Issues to consider for new versions of the derivation format (major or incremental)featureFeature request or proposalFeature request or proposalstoreIssues and pull requests concerning the Nix storeIssues and pull requests concerning the Nix store
Fields
Give feedbackNo fields configured for issues without a type.
Projects
Status
⚖ To discuss
Is your feature request related to a problem? Please describe.
metafields are not included in derivations because they would cause unnecessary rebuilds.Yet, we want users to be able to analyze this information, which isn't exposed reliably.
This lets us solve the problem of lost
meta.timeoutvalues in the store layer.Describe the solution you'd like
Change derivation output hashing so that a section of the
.drvis excluded from the computation, as well as being made unobservable from the derivation's environment. This causes outputs of equivalent derivations to still happily collide onto the same output, so that output caching remains as effective as it is today.This extra info only needs to be unobservable under what we might call "equivalence under validity", an idea that we already exploit in other areas. For example, it is fundamental to fixed output derivations. An example here is
meta.timeout: if we don't pass it to the build environment, it can not cause successful builds to have different outputs. We might even call it "equivalence under weak validity", because if a different timeout causes one derivation to fail, that's also ok.The idea of making output hashes not depend on the entire derivation is also not new; again fixed output derivations already apply this trick very effectively.
Describe alternatives you've considered
Store such information in
EvalState, so that we don't have to change the output hashing.This causes a problem when the build happens with
nix eval + nix-store -r, so we'd have to persist this info.However, then we can't discern between stale entries from previous evaluations and the entries we actually need.
Also it does not work without forwarding this info to remote builders somehow.
Additional context
Another output hashing change:
Priorities
Add 👍 to issues you find important.