Skip to content

stdenv: Replace "// optionalAttrs" with nullable attr name#430969

Merged
infinisil merged 1 commit intoNixOS:masterfrom
tweag:remove-attrs-merges-from-mkderivation
Aug 7, 2025
Merged

stdenv: Replace "// optionalAttrs" with nullable attr name#430969
infinisil merged 1 commit intoNixOS:masterfrom
tweag:remove-attrs-merges-from-mkderivation

Conversation

@YorikSar
Copy link
Copy Markdown
Contributor

@YorikSar YorikSar commented Aug 4, 2025

As in #430132, it saves a lot of set allocations and merge operations, but makes code harder to read.

I've tried introducing a function like this to make code cleaner, but it increases number of envs and space taken by them significantly:

  optionalAttr = cond: name: if cond then name else null;

I've also tried applying this logic to the section with isDarwin, but it makes things measurably worse for x86_64-linux eval.

Here are significant metrics gathered for x86_64-linux eval on my machines:

metric mean_before mean_after mean_diff mean_%_change p_value t_stat
cpuTime 38.1802 37.8518 -0.3284 -1.2641 0.5332 -0.6454
envs.bytes 893432387.6364 866216152.7273 -27216234.9091 -3.2010 0.0001 -6.4520
envs.elements 65051632.1818 63350762.0909 -1700870.0909 -2.7357 0.0001 -6.4520
envs.number 46627416.2727 44926257.0000 -1701159.2727 -3.8583 0.0001 -6.4519
gc.cycles 10.9091 10.7273 -0.1818 -1.2554 0.1669 -1.4907
gc.heapSize 3793414330.1818 3808570647.2727 15156317.0909 0.3018 0.7473 0.3313
gc.totalBytes 6934161339.6364 6792827748.3636 -141333591.2727 -2.1639 0.0001 -6.5707
nrFunctionCalls 42352816.8182 40651657.5455 -1701159.2727 -4.2193 0.0001 -6.4519
nrOpUpdateValuesCopied 112653866.8182 108331752.7273 -4322114.0909 -4.0958 0.0001 -6.5699
nrOpUpdates 5010432.6364 4033244.9091 -977187.7273 -19.3867 0.0001 -6.4689
nrThunks 56828714.1818 55127844.0909 -1700870.0909 -3.1665 0.0001 -6.4520
sets.bytes 2586801697.4545 2535201988.3636 -51599709.0909 -2.1743 0.0001 -6.5502
sets.elements 151548136.5455 148559516.3636 -2988620.1818 -2.1445 0.0001 -6.5267
sets.number 10126969.5455 9890607.9091 -236361.6364 -2.6579 0.0000 -6.8169
time.cpu 38.1802 37.8518 -0.3284 -1.2641 0.5332 -0.6454
time.gc 8.1961 7.7776 -0.4185 -2.8841 0.4146 -0.8511
time.gcFraction 0.2630 0.2505 -0.0125 -2.2664 0.2787 -1.1456
values.bytes 1919952840.0000 1879131957.8182 -40820882.1818 -2.3468 0.0001 -6.4520
values.number 79998035.0000 78297164.9091 -1700870.0909 -2.3468 0.0001 -6.4520

Things done


Add a 👍 reaction to pull requests you find important.

@nixpkgs-ci nixpkgs-ci bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. 6.topic: stdenv Standard environment labels Aug 4, 2025
@YorikSar YorikSar force-pushed the remove-attrs-merges-from-mkderivation branch from 1aa44ad to 571ec7c Compare August 4, 2025 15:44
Copy link
Copy Markdown
Member

@infinisil infinisil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Damn, those are some really impressive savings without any loss of functionality! 🚀

@nixpkgs-ci nixpkgs-ci bot added the 12.approvals: 1 This PR was reviewed and approved by one person. label Aug 6, 2025
Copy link
Copy Markdown
Contributor

@philiptaron philiptaron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Future: This function itself needs to be on the chopping block -- spot its use down below, where it can be inlined into the two uses.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don’t think inlining it would yield much benefit - it’s called from three separate places with different arguments and there is no repetition in them. But doesn’t hurt to try :)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This condition would definitely be more clear if it were extracted to a variable in the let block, ideally near the definition of hardeningDisable', defaultHardeningFlags, etc.

Unrelated to this PR, but hardeningDisable' has a unique which looks performance intensive and would be great to remove if it can be done without incident. I suspect the order of this list doesn't matter.

@philiptaron
Copy link
Copy Markdown
Contributor

@YorikSar, I'm very OK merging this as-is then doing updates in some future PR. There's a good deal more micro-optimizations to be done, I think.

@nixpkgs-ci nixpkgs-ci bot added 12.approvals: 2 This PR was reviewed and approved by two persons. and removed 12.approvals: 1 This PR was reviewed and approved by one person. labels Aug 6, 2025
Copy link
Copy Markdown
Member

@infinisil infinisil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving definitions to a let block is better for a separate PR, so we can independently evaluate the performance difference

As in NixOS#430132, it saves a lot of
set allocations and merge operations, but makes code harder to read.

I've tried introducing a function like this to make code cleaner, but it
increases number of envs and space taken by them significantly:

  optionalAttr = cond: name: if cond then name else null;

I've also tried applying this logic to the section with isDarwin, but
it makes things measurably worse for x86_64-linux eval.
@YorikSar YorikSar force-pushed the remove-attrs-merges-from-mkderivation branch from 571ec7c to 1d4489d Compare August 7, 2025 11:40
Copy link
Copy Markdown
Contributor Author

@YorikSar YorikSar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've addressed the comment about the contend addressed comment. I want to address the rest of them in a separate PR.

@infinisil infinisil enabled auto-merge August 7, 2025 11:41
@infinisil infinisil merged commit d3256e5 into NixOS:master Aug 7, 2025
24 of 27 checks passed
@infinisil infinisil deleted the remove-attrs-merges-from-mkderivation branch August 7, 2025 11:45
@github-project-automation github-project-automation bot moved this to Done in Stdenv Aug 7, 2025
Copy link
Copy Markdown
Contributor Author

@YorikSar YorikSar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’ve opened another PR optimising this part at #431756, please take a look.
@philiptaron I addressed a couple of your comments there, although I hoped to cover more of them.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don’t think inlining it would yield much benefit - it’s called from three separate places with different arguments and there is no repetition in them. But doesn’t hurt to try :)

@philiptaron
Copy link
Copy Markdown
Contributor

Thanks! I'm really excited by your work here. I deeply appreciate it, and its ability to make the experience for literally every single package in Nixpkgs better.

qweered added a commit to qweered/nixpkgs that referenced this pull request Apr 4, 2026
…tr names

Follow-up to NixOS#430969. Replace remaining `// optionalAttrs` patterns with
`${if cond then "name" else null} = value;` to avoid per-derivation
closure allocations and `//` merge operations.

Changes:
- make-derivation.nix: inline Darwin, Windows, outputChecks blocks and
  the `env'`/`__structuredAttrs` patterns in mkDerivationSimple
- trivial-builders: inline optionalAttrs in runCommandWith and symlinkJoin
mkg20001 pushed a commit to mkg20001/nixpkgs that referenced this pull request Apr 5, 2026
…tr names

Follow-up to NixOS#430969. Replace remaining `// optionalAttrs` patterns with
`${if cond then "name" else null} = value;` to avoid per-derivation
closure allocations and `//` merge operations.

Changes:
- make-derivation.nix: inline Darwin, Windows, outputChecks blocks and
  the `env'`/`__structuredAttrs` patterns in mkDerivationSimple
- trivial-builders: inline optionalAttrs in runCommandWith and symlinkJoin
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: stdenv Standard environment 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. 12.approvals: 2 This PR was reviewed and approved by two persons.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants