Skip to content

nixosTests.installer: Port installer and ZFS test to python#78670

Merged
flokli merged 4 commits intoNixOS:masterfrom
tfc:port-installer-test
Feb 8, 2020
Merged

nixosTests.installer: Port installer and ZFS test to python#78670
flokli merged 4 commits intoNixOS:masterfrom
tfc:port-installer-test

Conversation

@tfc
Copy link
Copy Markdown
Contributor

@tfc tfc commented Jan 28, 2020

Motivation for this change

#72828

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

cc @flokli @worldofpeace

@tfc
Copy link
Copy Markdown
Contributor Author

tfc commented Jan 28, 2020

@GrahamcOfBorg test installer zfs

@ofborg ofborg bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels Jan 28, 2020
@tfc tfc mentioned this pull request Jan 28, 2020
Closed
@worldofpeace worldofpeace added this to the 20.03 milestone Jan 30, 2020
@worldofpeace
Copy link
Copy Markdown
Contributor

worldofpeace commented Jan 31, 2020

@tfc please reword your commit for the zfs test, it's already ported 40396a7.

@tfc
Copy link
Copy Markdown
Contributor Author

tfc commented Jan 31, 2020

@tfc please reword your commit for the zfs test, it's already ported 40396a7.

Done @worldofpeace

@tfc tfc force-pushed the port-installer-test branch from 7cc4a1d to ab649e2 Compare January 31, 2020 08:15
@worldofpeace
Copy link
Copy Markdown
Contributor

worldofpeace commented Jan 31, 2020

@tfc please reword your commit for the zfs test, it's already ported 40396a7.

Done @worldofpeace

Umm I don't see a different commit message for ab649e2?

@tfc tfc force-pushed the port-installer-test branch from ab649e2 to 380134d Compare February 1, 2020 08:30
@tfc
Copy link
Copy Markdown
Contributor Author

tfc commented Feb 1, 2020

@tfc please reword your commit for the zfs test, it's already ported 40396a7.

Done @worldofpeace

Umm I don't see a different commit message for ab649e2?

Sorry, my fault. Please have a look again @worldofpeace

flokli
flokli reviewed Feb 3, 2020
View reviewed changes
nixos/lib/test-driver/test-driver.py Outdated
Copy link
Copy Markdown
Member

@flokli flokli Feb 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason we have this?

Looking at 510ada7, these could probably also just use copy_from_host…

Copy link
Copy Markdown
Contributor Author

@tfc tfc Feb 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At first i am using copy_from_host, but with the machine configs that follow later in the script, it doesn't work any longer as the VMs do not seem to support this. For that reason i added the shell variant as an addition.

Copy link
Copy Markdown
Member

@flokli flokli Feb 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume this is because /tmp/shared from nixos/modules/virtualisation/qemu-vm.nix isn't setup in the installer.

Can we just invoke the mount command manually before running copy_from_host?

Copy link
Copy Markdown
Contributor Author

@tfc tfc Feb 3, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could, but i really don't like the implications of this: Currently, tests are agnostic of how this is all implemented and this allows for transparently changing the virt backend etc (apart from the create_machine parameters which are pretty much qemu-specific, but these are also used by only a minority of tests).

As soon as tests begin doing such things, they would become testdriver-implementation specific and i am pretty sure we don't want that. I'd rather have two different functions that communicate which preconditions they assume and let the user choose from them, before users hard-code testdriver-details into their tests.

Copy link
Copy Markdown
Member

@flokli flokli Feb 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We had an offline discussion on that. Agreed on not leaking implementation-specific details about how things are shared into the actual test code. Ideally, this should be encapsulated behind the corresponding methods.

flokli
flokli reviewed Feb 3, 2020
View reviewed changes
nixos/lib/test-driver/test-driver.py Outdated
Copy link
Copy Markdown
Member

@flokli flokli Feb 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Prefer copy_from_host for large files.
Prefer copy_from_host whenever possible.

This implies it's okay to use it for small files.

In fact, we should use copy_from_host whenever possible, and only use the base64 and shell dance if we have to IMHO.

@tfc tfc force-pushed the port-installer-test branch 3 times, most recently from 7a47b7c to 11de1ae Compare February 3, 2020 17:23
@flokli
Copy link
Copy Markdown
Member

flokli commented Feb 3, 2020

@tfc The zfs tests are still missing from all-tests.nix. Can you add them there?

@tfc tfc force-pushed the port-installer-test branch from 11de1ae to c2864b8 Compare February 3, 2020 17:33
@tfc
Copy link
Copy Markdown
Contributor Author

tfc commented Feb 3, 2020

@GrahamcOfBorg test zfs

@flokli
Copy link
Copy Markdown
Member

flokli commented Feb 4, 2020

@GrahamcOfBorg build nixosTests.installer.simpleUefiSystemdBoot

@flokli
Copy link
Copy Markdown
Member

flokli commented Feb 4, 2020

@tfc the above test seems to fail: https://logs.nix.ci/?attempt_id=0cfe37cd-ae01-49d3-9997-aa34ffe60ae9&key=nixos%2Fnixpkgs.78670

@worldofpeace
Copy link
Copy Markdown
Contributor

worldofpeace commented Feb 4, 2020

the above test case also fails on master btw https://hydra.nixos.org/build/111477951, but I can confirm it currently fails for an related reasons to that one.

@flokli
Copy link
Copy Markdown
Member

flokli commented Feb 4, 2020

As the log indicates, this currently fails in the black validator, so it's more broken than before. We should at least fix that.

@flokli
Copy link
Copy Markdown
Member

flokli commented Feb 4, 2020

Also not sure if the installer tests block channel releases, so we should be extra careful here.

@tfc
Copy link
Copy Markdown
Contributor Author

tfc commented Feb 5, 2020

As the log indicates, this currently fails in the black validator, so it's more broken than before. We should at least fix that.

Don't worry, i am on it. It's just that i had no extra time yesterday.

@tfc tfc force-pushed the port-installer-test branch from c2864b8 to 0495eae Compare February 5, 2020 17:05
@tfc
Copy link
Copy Markdown
Contributor Author

tfc commented Feb 5, 2020

@GrahamcOfBorg test installer.simpleUefiSystemdBoot

@disassembler disassembler modified the milestones: 20.03, 20.09 Feb 5, 2020
@tfc
Copy link
Copy Markdown
Contributor Author

tfc commented Feb 5, 2020

So the current state is that the simpleUefiSystemdBoot test sometimes succeeds, sometimes doesn't (on both x86_64 and aarch64). When it fails, it fails like this:

  • nixos-install calls systemd bootctl and that produces:
machine # copying channel...
machine # installing the boot loader...
machine # setting up /etc...
machine # Initializing machine ID from random generator.
machine # Failed to open file system "/dev/block/253:1": No such file or directory
machine # Traceback (most recent call last):
machine #   File "/nix/store/4lzpcmzwdi73ma34vx6hiff80an1r050-systemd-boot-builder.py", line 240, in <module>
machine #     main()
machine #   File "/nix/store/4lzpcmzwdi73ma34vx6hiff80an1r050-systemd-boot-builder.py", line 199, in main
machine #     subprocess.check_call(["/nix/store/c54w4pb6g627ix9d7pyrqfv9qvgjbg6n-systemd-243.4/bin/bootctl", "--path=/boot", "--no-variables", "install"])
machine #   File "/nix/store/4c4c5ig32lqhsgmcncnb72c3a4bwj66n-python3-3.7.6/lib/python3.7/subprocess.py", line 363, in check_call
machine #     raise CalledProcessError(retcode, cmd)
machine # subprocess.CalledProcessError: Command '['/nix/store/c54w4pb6g627ix9d7pyrqfv9qvgjbg6n-systemd-243.4/bin/bootctl', '--path=/boot', '--no-variables', 'install']' returned non-zero exit status 1.
machine: output:
error: command `nixos-install < /dev/null >&2` failed (exit code 1)
cleaning up
killing machine (pid 9)
(0.00 seconds)
builder for '/nix/store/7jad11fkfs1rqql86zx5c84i1dqf7b1m-vm-test-run-installer-simpleUefiSystemdBoot.drv' failed with exit code 1
error: build of '/nix/store/7jad11fkfs1rqql86zx5c84i1dqf7b1m-vm-test-run-installer-simpleUefiSystemdBoot.drv' failed

@tfc tfc force-pushed the port-installer-test branch from 0495eae to 81172e1 Compare February 5, 2020 22:20
@tfc
Copy link
Copy Markdown
Contributor Author

tfc commented Feb 5, 2020

@GrahamcOfBorg test installer.btrfsSimple installer.luksroot-format1 installer.simpleClone installer.swraid installer.btrfsSubvolDefault installer.luksroot-format2 installer.simpleLabels installer.zfsroot installer.btrfsSubvols installer.lvm installer.simpleProvided installer.encryptedFSWithKeyfile installer.separateBoot installer.simpleUefiGrub installer.grub1 installer.separateBootFat installer.simpleUefiGrubClone installer.luksroot installer.simple installer.simpleUefiSystemdBoot

@tfc tfc force-pushed the port-installer-test branch from 81172e1 to 30d8d1d Compare February 6, 2020 07:22
@tfc
Copy link
Copy Markdown
Contributor Author

tfc commented Feb 6, 2020

@GrahamcOfBorg test installer-btrfsSimple installer-luksroot-format1 installer-simpleClone installer-swraid installer-btrfsSubvolDefault installer-luksroot-format2 installer-simpleLabels installer-zfsroot installer-btrfsSubvols installer-lvm installer-simpleProvided installer-encryptedFSWithKeyfile installer-separateBoot installer-simpleUefiGrub installer-grub1 installer-separateBootFat installer-simpleUefiGrubClone installer-luksroot installer-simple installer-simpleUefiSystemdBoot

@tfc tfc force-pushed the port-installer-test branch from 30d8d1d to 942f0fd Compare February 7, 2020 12:24
@tfc
Copy link
Copy Markdown
Contributor Author

tfc commented Feb 7, 2020

@GrahamcOfBorg test installer

@flokli
Copy link
Copy Markdown
Member

flokli commented Feb 8, 2020

Judging from hydra, simpleUefiSystemdBoot seems to have been flaky in the perl test driver as well, so merging this in.

@flokli flokli merged commit dd5f92f into NixOS:master Feb 8, 2020
@worldofpeace
Copy link
Copy Markdown
Contributor

worldofpeace commented Feb 8, 2020

Good for me as well.

dtzWill pushed a commit to dtzWill/nixpkgs that referenced this pull request Feb 8, 2020
@flokli @dtzWill
Merge pull request NixOS#78670 from tfc/port-installer-test
c43a45f 
nixosTests.installer: Port installer and ZFS test to python

(cherry picked from commit dd5f92f)
@tfc tfc deleted the port-installer-test branch February 9, 2020 22:18
This was referenced Feb 18, 2020
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@flokli flokli flokli left review comments

@blitz blitz Awaiting requested review from blitz

@adisbladis adisbladis Awaiting requested review from adisbladis

@jonringer jonringer Awaiting requested review from jonringer

Assignees

No one assigned

Labels

6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux.

Projects

None yet

Milestone

20.09

Development

Successfully merging this pull request may close these issues.

4 participants

@tfc @worldofpeace @flokli @disassembler