Fix #6572 requires non-existent output#7283
Conversation
requires non-existent output
It occurred when a output of the dependency was already available, so it didn't need rebuilding and didn't get added to the inputDrvOutputs. This process-related info wasn't suitable for the purpose of finding the actual input paths for the builder. It is better to do this in absolute terms by querying the store.
|
Bug was present since impure derivations, Nix 2.8, so backporting to current NixOS stable as well. |
|
I'll fix that in the backport prs. |
thufschmitt
left a comment
There was a problem hiding this comment.
Thanks a lot for digging into this, it's really great to see this issue fixed.
Having to query the store again is a bit sad, but hopefully one day we'll have a proper structured control flow, and we won't have to rely on this any more 🤞
|
Backport failed for Please cherry-pick the changes locally. git fetch origin 2.8-maintenance
git worktree add -d .worktree/backport-7283-to-2.8-maintenance origin/2.8-maintenance
cd .worktree/backport-7283-to-2.8-maintenance
git checkout -b backport-7283-to-2.8-maintenance
ancref=$(git merge-base 0efc314d4d1add3215810f9034b6041759d5175b c279ddb18cf3a34b0f6d4e3adcf9455da5397ad7)
git cherry-pick -x $ancref..c279ddb18cf3a34b0f6d4e3adcf9455da5397ad7 |
|
Successfully created backport PR #7304 for |
|
Successfully created backport PR #7305 for |
|
Successfully created backport PR #7306 for |
| worker.store.printStorePath(drvPath), j, worker.store.printStorePath(depDrvPath)); | ||
| } | ||
| else { | ||
| auto outMap = worker.evalStore.queryDerivationOutputMap(depDrvPath); |
There was a problem hiding this comment.
Is this something expensive that should be evaluated at most once in the loop?
There was a problem hiding this comment.
In the vast majority of cases, inputDrvOutputs will have the result. Otherwise Nix would have been completely unusable since 2.8.
We can just lift it out of the loop, because that would create many more derivation file queries; for each inputDrv, instead for only very rare cases. Even then the chance of reusing the map is fairly low, because derivations typically don't have many outputs, and this only happens for multi output derivations.
It is important to keep in mind though when implementing the suggestion in the comment above, so I'll update that.
We should talk about this at some point because I feel the exact opposite, that the store should be the single source of truth and we should have as little in-memory state as possible. |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/2022-11-14-nix-team-meeting-minutes-8/23452/1 |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/nix-team-report-2022-10-2023-03/27486/1 |
Reproduces #6572
Only
issue_6572_dependent_outputsfails, suggesting that dependencies between the outputs of a multi-output drv are involved.