Get rid of deep realisations entirely by Ericson2314 · Pull Request #15289 · NixOS/nix

Ericson2314 · 2026-02-18T21:13:42Z

Motivation

Fix #11896

Context

Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

src/libstore-tests/dummy-store.cc

roberth · 2026-03-04T17:51:11Z

src/libstore/include/nix/store/outputs-query.hh

+ * Callback type for querying realisations. The callback should return
+ * the realisation for the given DrvOutput, or nullptr if not found.
+ */
+using QueryRealisationFun = std::function<std::shared_ptr<const UnkeyedRealisation>(const DrvOutput &)>;


Could be fun<> now?
Makes your reference to nullptr trivially unambiguous :)

Oh yes this is something to improve post the recent rebase :)

Did you mean std::optional<fun<....>>?

roberth · 2026-03-04T17:52:26Z

src/libstore/include/nix/store/outputs-query.hh

+/**
+ * For internal use only.
+ */


You're teasing me!

Suggested change

/**

* For internal use only.

*/

/**

* For internal use only. Just here to be shared in the implementation of `Store::queryPartialDerivationOutputMap` and `deepQueryPartialDerivationOutputMap`.

*/

roberth · 2026-03-04T22:20:30Z

src/libstore/derivations.cc


 static bool tryResolveInput(
-    Store & store,
+    const StoreDirConfig & store,


Just a storedir, nice.

roberth · 2026-03-04T22:24:49Z

src/libcmd/built-path.cc

Love to see progress on CA. Would be great to add a release note!

xokdvium · 2026-03-16T17:52:58Z

src/libstore/local-store.cc

Why are we removing this? What if one enables and then disables CA drvs?

The check was moved to Store::queryRealisation, so it affects all implementations / no implementation can forget to check.

This change was done in its own commit btw, it might be easier to read that way.

lovesegfault

one thing not in the diff — nix-build.cc:593 still uses store->queryPartialDerivationOutputMap(inputDrv). line 537 got migrated to deepQuery* but this one didn't. if inputDrv is a CA drv that needs resolution, i think *o at 596 could throw now that deep realisations are gone? pretty niche (nix-shell + __structuredAttrs + CA input) so maybe fine to punt.

not blocking — overall this looks good, happy to see deep realisations go

src/libstore/store-api.cc

src/libstore/include/nix/store/outputs-query.hh

src/libstore/outputs-query.cc

src/libstore/local-store.cc

src/libstore/include/nix/store/outputs-query.hh

xokdvium · 2026-03-16T20:29:01Z

src/nix/copy.cc

        }

-        copyPaths(*srcStore, *dstStore, stuffToCopy, NoRepair, checkSigs, substitute);
+        copyClosure(*srcStore, *dstStore, stuffToCopy, NoRepair, checkSigs, substitute);


Good question, maybe I did this because I needed to compute the shallow realisation build closure to send it over.

If so, that should be factored out. And longer term, we should stop implicitly copying realisations anyways.

This makes the --no-recursive option ignored.

xokdvium · 2026-03-16T20:29:46Z

tests/functional/ca/build-cache.sh

+    # copyClosure copies unsigned input-addressed closure deps into the
+    # binary cache, so mark the substituter as trusted to skip the
+    # signature check in substitution-goal.cc.


Do we actually want this?

We don't want to only have build traces for objects we have in the store, anymore.

tests/functional/ca/build-cache.sh

domenkozar · 2026-03-23T13:47:03Z

pretty niche (nix-shell + __structuredAttrs + CA input) so maybe fine to punt.

not that niche in my experience

xokdvium · 2026-03-23T18:46:50Z

src/libstore/build/derivation-goal.cc

-            if (!drv->type().isImpure()) {
-                Realisation newRealisation{
-                    realisation,
-                    {
-                        .drvPath = drvPath,
-                        .outputName = wantedOutput,
-                    }};
-                newRealisation.signatures.clear();
-                worker.store.signRealisation(newRealisation);
-                worker.store.registerDrvOutput(newRealisation);
-            }
-


Why is this removed?

Yup, this is the spot where it created deep realizations before.

xokdvium · 2026-03-23T18:48:46Z

tests/functional/ca/build-cache.sh

    clearStore
    # Check nothing gets built.
-    buildAttr "$derivationPath" 1 --option substituters "file://$cacheDir" --no-require-sigs |& grepQuietInverse " will be built:"
+    buildAttr "$derivationPath" 1 --option substituters "file://$cacheDir?trusted=1" --no-require-sigs -j0


Why do we need this?

Discussed. Some weird existing problem that made ?trusted=1 needed, but it is just helping along with the existing --no-require-sigs flag. Not adding fundamentally new untrustedness.

xokdvium · 2026-03-23T18:53:28Z

src/libstore/store-api.cc

+        std::visit(
+            overloaded{
+                [&](const Realisation & realisation) {
+                    experimentalFeatureSettings.require(Xp::CaDerivations);
+                    realisations.push_back(&realisation);
+                    storePaths.insert(realisation.outPath);
+                },
+                [&](const OpaquePath & op) { storePaths.insert(op.path); },
+            },
+            path.raw);


What's the effect of this?

We no longer copy over out paths implicitly. If the thing wants to copy a realisation, we just copy that.

This morally goes with removing the foreign keys in the previous commit. It is necessary for copying a (shallow) realisation closure without also making it necessary to keep the build closure around.

Eventually I think we will get rid of RealisedPath::Set because it is just better to manage the build trace explicitly and separately.

Ericson2314 requested a review from edolstra as a code owner February 18, 2026 21:13

github-actions bot added documentation new-cli Relating to the "nix" command with-tests Issues related to testing. PRs with tests have some priority store Issues and pull requests concerning the Nix store repl The Read Eval Print Loop, "nix repl" command and debugger labels Feb 18, 2026

Ericson2314 force-pushed the fix-11896 branch 5 times, most recently from 9c3eabb to b7a9a14 Compare February 19, 2026 15:56

Ericson2314 force-pushed the fix-11896 branch 3 times, most recently from 719c8e9 to 99353ea Compare March 3, 2026 07:38

amaanq force-pushed the fix-11896 branch 8 times, most recently from e69a5a0 to 134f1b4 Compare March 4, 2026 05:32

roberth reviewed Mar 4, 2026

View reviewed changes

amaanq force-pushed the fix-11896 branch from 134f1b4 to 5676838 Compare March 11, 2026 21:37

github-actions bot added the c api Nix as a C library with a stable interface label Mar 11, 2026

Ericson2314 force-pushed the fix-11896 branch 2 times, most recently from 9971ea9 to 30a9106 Compare March 16, 2026 16:45

xokdvium reviewed Mar 16, 2026

View reviewed changes

Ericson2314 force-pushed the fix-11896 branch from 30a9106 to a9f6755 Compare March 16, 2026 18:04

lovesegfault reviewed Mar 16, 2026

View reviewed changes

xokdvium reviewed Mar 16, 2026

View reviewed changes

amaanq force-pushed the fix-11896 branch from a9f6755 to 0362341 Compare March 18, 2026 18:07

This was referenced Mar 19, 2026

tests: RAII guard for experimental feature settings #15527

Merged

libstore: check ca-derivations xp in Store::queryRealisation #15528

Merged

amaanq force-pushed the fix-11896 branch from 0362341 to 9629b17 Compare March 19, 2026 21:21

Remove BuildTrace <-> ValidPaths foreign key

694721e

We don't want to only have build traces for objects we have in the store, anymore.

Ericson2314 commented Mar 19, 2026

View reviewed changes

tests/functional/ca/build-cache.sh Outdated Show resolved Hide resolved

amaanq force-pushed the fix-11896 branch 4 times, most recently from ece4def to 8ab8977 Compare March 19, 2026 22:37

Get rid of deep realisations entirely

57c83a1

amaanq force-pushed the fix-11896 branch from 8ab8977 to 57c83a1 Compare March 19, 2026 22:39

roberth mentioned this pull request Mar 23, 2026

feat: add CA derivation FFI bindings (validPathInfoCA, getDerivationOutputIds, queryRealisation) hercules-ci/hercules-ci-agent#670

Open

xokdvium reviewed Mar 23, 2026

View reviewed changes

xokdvium approved these changes Mar 23, 2026

View reviewed changes

Ericson2314 added this pull request to the merge queue Mar 23, 2026

Merged via the queue into NixOS:master with commit e6ac69f Mar 23, 2026
16 checks passed

Ericson2314 deleted the fix-11896 branch March 23, 2026 20:12

Uh oh!

Conversation

Ericson2314 commented Feb 18, 2026

Motivation

Context

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

roberth Mar 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Ericson2314 Mar 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lovesegfault left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

domenkozar commented Mar 23, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

roberth Mar 4, 2026 •

edited

Loading

Ericson2314 Mar 16, 2026 •

edited

Loading