What

• Change nixos/all-tests.nix to use runTest instead of handleTest.
• Change the tests to be modules.

Why

Benefits of runTest and runTestOn:

• Define values for any test option
• Use imports to compose tests
• Access the module arguments like hostPkgs and config.node.pkgs
• Portable to other VM hosts, specifically Darwin
• Faster evaluation, using a single pkgs instance
• Documented for external use too

How

Many tests are trivial to migrate, but here's the complete guide, also available in a comment in all-tests.nix.
Testing the darwin support is optional and largely unnecessary.

Changes required to migrate to runTest:

• Remove any import ../make-test-python.nix or similar calls, leaving only
  the callback function.

• Convert the function header to make it a module.
  Packages can be taken from the following. For VM host portability, use

  • config.node.pkgs.<name> or config.nodes.foo.nixpkgs.pkgs.<name> to refer
    to the Nixpkgs used on the VM guest(s).
  • hostPkgs.<name> when invoking commands on the VM host (e.g. in Python
    os.system("foo"))

• Since the runTest argument is a module instead of a function, arguments
  must be passed as option definitions.
  You may declare explicit options for the test parameter(s), or use the
  less explicit _module.args.<name> to pass arguments to the module.

  Example call with arguments:

  runTest {
    imports = [ ./test.nix ];
    _module.args.getPackage = pkgs: pkgs.foo_1_2;
  }
  

• If your test requires any definitions in nixpkgs.* options, set
  node.pkgsReadOnly = false in the test configuration.

Test matrix

Pass the runTest function into the test file and use it instead of make-test-python.nix or similar.

Context

Written as a Nixpkgs maintainer, nothing special.

• nixos: Document handleTest deprecation #383870