nixosTests: handleTest -> runTest, batch 1 by Prince213 · Pull Request #410569 · NixOS/nixpkgs

Prince213 · 2025-05-24T15:50:37Z

Partially resolves: #386873

These changes are automated by a sed script, and the changed files are formatted to ensure good style and to make sure it doesn't cause syntax error.

Remaining uses of handleTest are around 220.

I'm willing to work for the remaining uses after this gets merged.

To replicate the changes on local machine:

rg -e '= handleTest \./(.+?\.nix) \{ \};' -o -r '$1' all-tests.nix > files.lst
for file in $(cat files.lst); do
  sed -i -e '/^import \.\.\?\/make-test-python.nix ($/d' -e '$s/^)$//' $file
  nixfmt $file
done
rm files.lst
sed -i 's|handleTest \(\./.\+\?\.nix\) { };|runTest \1;|' all-tests.nix

Things done

Add a 👍 reaction to pull requests you find important.

roberth

all-tests.nix has reordered a few attributes.
I assume that's because of rebasing, so let's fix that afterwards.

nixos/tests/all-tests.nix

roberth · 2025-05-26T12:10:24Z

Oh, it has conflicts right now. Could you fix the reordered attributes?

Prince213 · 2025-05-26T12:11:26Z

Sure.

Reference: NixOS#386873

Prince213 · 2025-05-26T12:47:12Z

Rebased the commit against master and fixed a few previously out-of-order attributes.

jian-lin · 2025-05-26T13:29:33Z

FYI, a new CI check keep-sorted was introduced recently. We can use it for all-tests.nix.

roberth · 2025-05-27T13:34:04Z

Thank you!

mweinelt · 2025-05-27T13:50:00Z

I'll make a guess and say this change broke at least the eval of unstable-small.

{UNKNOWN}: aggregate job ‘tested’ failed with the error: nixos.tests.latestKernel.login.x86_64-linux: does not exist
nixos.tests.latestKernel.login.aarch64-linux: does not exist
nixos.tests.login.x86_64-linux: error:
       … while calling the 'head' builtin
         at /nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/lib/attrsets.nix:1534:13:
         1533|           if length values == 1 || pred here (elemAt values 1) (head values) then
         1534|             head values
             |             ^
         1535|           else

       … while calling the 'head' builtin
         at /nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/lib/attrsets.nix:1534:13:
         1533|           if length values == 1 || pred here (elemAt values 1) (head values) then
         1534|             head values
             |             ^
         1535|           else

       … while calling the 'deepSeq' builtin
         at /nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/lib/customisation.nix:479:35:
          478|     in
          479|     if drv == null then null else deepSeq drv' drv';
             |                                   ^
          480|

       … while evaluating the option `nodes.machine.system.build.toplevel':

       … while evaluating definitions from `/nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/nixos/modules/system/activation/top-level.nix':

       … while evaluating the option `nodes.machine.assertions':

       … while evaluating definitions from `/nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/nixos/modules/system/boot/kernel.nix':

       … while evaluating the option `nodes.machine.boot.kernelPackages':

       … while evaluating definitions from `/nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/nixos/tests/login.nix':

       … while evaluating the module argument `latestKernel' in "/nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/nixos/tests/login.nix":

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: attribute 'latestKernel' missing
       at /nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/lib/modules.nix:652:66:
          651|       extraArgs = mapAttrs (
          652|         name: _: addErrorContext (context name) (args.${name} or config._module.args.${name})
             |                                                                  ^
          653|       ) (functionArgs f);
nixos.tests.login.aarch64-linux: error:
       … while calling the 'head' builtin
         at /nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/lib/attrsets.nix:1534:13:
         1533|           if length values == 1 || pred here (elemAt values 1) (head values) then
         1534|             head values
             |             ^
         1535|           else

       … while calling the 'deepSeq' builtin
         at /nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/lib/customisation.nix:479:35:
          478|     in
          479|     if drv == null then null else deepSeq drv' drv';
             |                                   ^
          480|

       … while evaluating the option `nodes.machine.system.build.toplevel':

       … while evaluating definitions from `/nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/nixos/modules/system/activation/top-level.nix':

       … while evaluating the option `nodes.machine.assertions':

       … while evaluating definitions from `/nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/nixos/modules/system/boot/kernel.nix':

       … while evaluating the option `nodes.machine.boot.kernelPackages':

       … while evaluating definitions from `/nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/nixos/tests/login.nix':

       … while evaluating the module argument `latestKernel' in "/nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/nixos/tests/login.nix":

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: attribute 'latestKernel' missing
       at /nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/lib/modules.nix:652:66:
          651|       extraArgs = mapAttrs (
          652|         name: _: addErrorContext (context name) (args.${name} or config._module.args.${name})
             |                                                                  ^
          653|       ) (functionArgs f);
nixos.tests.containers-imperative.x86_64-linux: error:
       … while calling the 'head' builtin
         at /nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/lib/attrsets.nix:1534:13:
         1533|           if length values == 1 || pred here (elemAt values 1) (head values) then
         1534|             head values
             |             ^
         1535|           else

       … while calling the 'head' builtin
         at /nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/lib/attrsets.nix:1534:13:
         1533|           if length values == 1 || pred here (elemAt values 1) (head values) then
         1534|             head values
             |             ^
         1535|           else

       … while calling the 'deepSeq' builtin
         at /nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/lib/customisation.nix:479:35:
          478|     in
          479|     if drv == null then null else deepSeq drv' drv';
             |                                   ^
          480|

       … while evaluating the option `nodes.machine.virtualisation.qemu.options':

       … while evaluating definitions from `/nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/nixos/modules/virtualisation/qemu-vm.nix':

       … while evaluating the option `nodes.machine.virtualisation.additionalPaths':

       … while evaluating definitions from `/nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/nixos/tests/containers-imperative.nix':

       … while evaluating the module argument `pkgs' in "/nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/nixos/modules/services/hardware/bluetooth.nix":

       … while evaluating the option `nixpkgs.localSystem':

       … while evaluating definitions from `/nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/nixos/tests/containers-imperative.nix':

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: attribute 'localSystem' missing
       at /nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/nixos/tests/containers-imperative.nix:35:52:
           34|             modules = lib.singleton {
           35|               nixpkgs = { inherit (config.nixpkgs) localSystem; };
             |                                                    ^
           36|
nixos.tests.containers-imperative.aarch64-linux: error:
       … while calling the 'head' builtin
         at /nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/lib/attrsets.nix:1534:13:
         1533|           if length values == 1 || pred here (elemAt values 1) (head values) then
         1534|             head values
             |             ^
         1535|           else

       … while calling the 'deepSeq' builtin
         at /nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/lib/customisation.nix:479:35:
          478|     in
          479|     if drv == null then null else deepSeq drv' drv';
             |                                   ^
          480|

       … while evaluating the option `nodes.machine.virtualisation.qemu.options':

       … while evaluating definitions from `/nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/nixos/modules/virtualisation/qemu-vm.nix':

       … while evaluating the option `nodes.machine.virtualisation.additionalPaths':

       … while evaluating definitions from `/nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/nixos/tests/containers-imperative.nix':

       … while evaluating the module argument `pkgs' in "/nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/nixos/modules/services/hardware/bluetooth.nix":

       … while evaluating the option `nixpkgs.localSystem':

       … while evaluating definitions from `/nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/nixos/tests/containers-imperative.nix':

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: attribute 'localSystem' missing
       at /nix/store/fbg634nk478h8hjgfrzfa8q3sy8rk5wl-source/nixos/tests/containers-imperative.nix:35:52:
           34|             modules = lib.singleton {
           35|               nixpkgs = { inherit (config.nixpkgs) localSystem; };
             |                                                    ^
           36|
 at /nix/store/alr2c0g777fq0gxshg6zmrqgbphii8da-hydra-perl-deps/lib/perl5/site_perl/5.40.0/Catalyst/Model/DBIC/Schema.pm line 526

mweinelt · 2025-05-27T19:49:56Z

We lost the following jobs on nixos-unstable. https://hydra.nixos.org/eval/1815634#tabs-removed

Prince213 · 2025-05-28T03:35:20Z

Okay I'm working on a fix.

SuperSandro2000 · 2025-05-28T13:41:51Z

I just wanted to note that I am currently rebasing a PR for paperless against this PR and it is a bit more tricky than I would like it to be 😓

roberth · 2025-05-28T14:23:49Z

Due to the indentation change? Maybe mergiraf could be of help.

Prince213 · 2025-05-28T14:33:51Z

I suppose it's #385637? This PR only changes the first and the last line, and other lines only have indent changes. So you should be able to just apply your changes first, then remove the two lines, then format.

SuperSandro2000 · 2025-05-28T15:21:22Z

mergiraf

That supports nix 🤔 I need to look into that. Thanks for the tip!

I suppose it's #385637?

yep

This PR only changes the first and the last line, and other lines only have indent changes. So you should be able to just apply your changes first, then remove the two lines, then format.

That's what I ended up doing 😅 shame that git couldn't resolve it on its own

trofi · 2025-05-30T21:23:05Z

Bisect claims f34483b nixosTests: handleTest -> runTest, batch 1 broke the eval of discourse.discourseAllPlugins.tests. Attempted at fixing it as:

discourse.discourseAllPlugins.tests: fix the eval #412479

This is a partial backport of f34483b from NixOS#410569 to the release-25.05 branch. Specifically, this is a backport of only the Syncthing-related changes from that commit. The main motivation behind this change is to make it easier to backport some other commits that depend on changes from f34483b. I’m backporting those other commits so that I can prevent syncthing.tests.syncthing-many-devices from failing to build after Syncthing is updated to version 1.29.7. Co-authored-by: Jason Yundt <jason@jasonyundt.email>

github-actions bot added the 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS label May 24, 2025

Prince213 added the 2.status: work-in-progress label May 24, 2025

github-actions bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels May 24, 2025

Prince213 force-pushed the handleTest-runTest branch from 0d9b62c to bc8329e Compare May 24, 2025 16:34

Prince213 changed the title ~~nixosTests: handleTest -> runTest~~ nixosTests: handleTest -> runTest, batch 1 May 24, 2025

Prince213 added 8.has: clean-up This PR removes packages or removes other cruft 11.by: nixpkgs-member and removed 2.status: work-in-progress labels May 24, 2025

Prince213 marked this pull request as ready for review May 24, 2025 16:42

nix-owners bot requested a review from joachifm May 24, 2025 16:44

Prince213 requested review from infinisil, lucasew and roberth May 24, 2025 16:44

Prince213 force-pushed the handleTest-runTest branch 2 times, most recently from 65bc844 to 40688bc Compare May 26, 2025 03:31

roberth approved these changes May 26, 2025

View reviewed changes

nixos/tests/all-tests.nix Outdated Show resolved Hide resolved

nixos/tests/all-tests.nix Outdated Show resolved Hide resolved

nixosTests: handleTest -> runTest, batch 1

f34483b

Reference: NixOS#386873

Prince213 force-pushed the handleTest-runTest branch from 40688bc to f34483b Compare May 26, 2025 12:40

Prince213 added the 12.approvals: 1 This PR was reviewed and approved by one person. label May 26, 2025

jian-lin requested a review from roberth May 26, 2025 13:30

roberth approved these changes May 27, 2025

View reviewed changes

roberth merged commit d5226c8 into NixOS:master May 27, 2025
17 of 18 checks passed

roberth mentioned this pull request May 27, 2025

nixosTests.{login,containers-imperative}: fix eval #411421

Merged

13 tasks

gepbird mentioned this pull request May 27, 2025

{,nixos/}pocket-id: 0.53.0 -> 1.1.0 #411229

Merged

13 tasks

Prince213 mentioned this pull request May 28, 2025

nixosTests: fix eval #411634

Merged

13 tasks

Yarny0 mentioned this pull request May 29, 2025

tsm-client: 8.1.25.0 -> 8.1.27.0, drop old migration code #387363

Merged

15 tasks

This was referenced Jun 6, 2025

Tracking: migrate NixOS handleTest -> runTest #386873

Open

nixosTests: handleTest -> runTest, batch 2 #414711

Merged

Prince213 deleted the handleTest-runTest branch June 27, 2025 04:55

Jayman2000 mentioned this pull request Dec 20, 2025

[25.05] syncthing: Prevent test from failing & 1.29.5 → 1.30.0 #472564

Closed

13 tasks

Uh oh!

Conversation

Prince213 commented May 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Things done

Uh oh!

roberth left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

roberth commented May 26, 2025

Uh oh!

Prince213 commented May 26, 2025

Uh oh!

Prince213 commented May 26, 2025

Uh oh!

jian-lin commented May 26, 2025

Uh oh!

Uh oh!

roberth commented May 27, 2025

Uh oh!

mweinelt commented May 27, 2025

Uh oh!

mweinelt commented May 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Prince213 commented May 28, 2025

Uh oh!

SuperSandro2000 commented May 28, 2025

Uh oh!

roberth commented May 28, 2025

Uh oh!

Prince213 commented May 28, 2025

Uh oh!

SuperSandro2000 commented May 28, 2025

Uh oh!

trofi commented May 30, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Prince213 commented May 24, 2025 •

edited

Loading

mweinelt commented May 27, 2025 •

edited

Loading