Skip to content

Scrap ParsedDerivation for parts#12410

Merged
Mic92 merged 2 commits intoNixOS:masterfrom
obsidiansystems:derivation-options-2
Apr 15, 2025
Merged

Scrap ParsedDerivation for parts#12410
Mic92 merged 2 commits intoNixOS:masterfrom
obsidiansystems:derivation-options-2

Conversation

@Ericson2314
Copy link
Copy Markdown
Member

@Ericson2314 Ericson2314 commented Feb 3, 2025

Motivation

This is a follow up to #12292 doing more work cleaning up derivation options. Please review each commit separately.

The end goal is to have these fields on Derivation itself:

  • DerivationOptions options
  • std::variant<StructuredAttrs, StringMap> structuredAttrsOrEnv

Context

Depends on #12292
Depends on #13024

Preparing the way for #10760, and the issues it addresses.

Progress on #9846


Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

};
}

StringSet DerivationOptions::getRequiredSystemFeatures(const BasicDerivation & drv) const
Copy link
Copy Markdown
Contributor

@haenoe haenoe Feb 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do these methods really belong to DerivationOptions, or is this just an artifact from the fact that they were previously in ParsedDerivation? Would it make sense to move them to BasicDerivation?

Edit: that is the endgoal anyway -- I forgot :D

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think I care too much where these methods lie --- apologies if I said otherwise before.

auto get_ = [&](const std::string & name) -> std::optional<StringSet> {
if (auto i = get(output, name)) {
StringSet res;
for (auto j = i->begin(); j != i->end(); ++j) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also use getStringSet here, right?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point! And it shouldn't be converted to store paths actually because that will break CA derivations using placeholders.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a silly question, but where do we convert it to store paths right now?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would break CA derivations, in which is it still placeholders (and indeed for that reason the one place I did convert to store paths is a mistake)

We can, however, parse the placeholders and convert to SingleDerivedPath

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh the question was where. It is in when parsing the structured attrs / environment variables.

checks.allowedRequisites = get_("allowedRequisites");
checks.disallowedReferences = get_("disallowedReferences").value_or(StringSet{});
checks.disallowedRequisites = get_("disallowedRequisites").value_or(StringSet{});
;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oopsie

* attributes give to the builder. The set of paths in the original JSON
* is replaced with a list of `PathInfo` in JSON format.
*/
std::map<std::string, StorePathSet> exportReferencesGraph;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this field also have to be (de)serialized into JSON. If yes, then we also need to put it into the (de)serializer respectively.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yes, I just forgot about that 😓

@dpulls
Copy link
Copy Markdown

dpulls bot commented Feb 17, 2025

🎉 All dependencies have been resolved !

@Ericson2314 Ericson2314 force-pushed the derivation-options-2 branch from f81c89f to 313ec2f Compare March 16, 2025 22:50
@Ericson2314 Ericson2314 force-pushed the derivation-options-2 branch from 313ec2f to 1e1eede Compare April 13, 2025 22:32
@Ericson2314 Ericson2314 marked this pull request as draft April 13, 2025 22:32
Ericson2314 and others added 2 commits April 14, 2025 15:46
This moves us towards getting rid of `ParsedDerivation` and just having
`DerivationOptions`.

Co-Authored-By: HaeNoe <git@haenoe.party>
Only a much smaller `StructuredAttrs` remains, the rest is is now moved
to `DerivationOptions`.

This gets us quite close to `std::optional<StructuredAttrs>` and
`DerivationOptions` being included in `Derivation` as fields.
@Ericson2314 Ericson2314 force-pushed the derivation-options-2 branch from 1e1eede to d8be4f6 Compare April 14, 2025 20:42
@github-actions github-actions bot added the with-tests Issues related to testing. PRs with tests have some priority label Apr 14, 2025
@Ericson2314 Ericson2314 marked this pull request as ready for review April 15, 2025 03:53
@dpulls
Copy link
Copy Markdown

dpulls bot commented Apr 15, 2025

🎉 All dependencies have been resolved !

@Ericson2314 Ericson2314 changed the title More work relating to DerivationOptions Scrap ParsedDerivation for parts Apr 15, 2025
@Mic92 Mic92 merged commit cf6da92 into NixOS:master Apr 15, 2025
15 checks passed
@Ericson2314 Ericson2314 deleted the derivation-options-2 branch April 15, 2025 16:41
@vcunat
Copy link
Copy Markdown
Member

vcunat commented Jul 25, 2025

For reference, I think this is the change that causes nix to newly reject null values inside disallowedReferences. I don't think it's a problem really, just an incompatible change and potentially surprising. /cc NixOS/nixpkgs#426071

@Ericson2314
Copy link
Copy Markdown
Member Author

@vcunat We can make it still work. We did another relaxation for compat already surrounding this.

@vcunat
Copy link
Copy Markdown
Member

vcunat commented Jul 25, 2025

It's hard for me to guess how much this hits users. (including outside nixpkgs itself)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

with-tests Issues related to testing. PRs with tests have some priority

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants