Skip to content

nixos: kubernetes fixes#37218

Merged
cstrahan merged 1 commit intoNixOS:masterfrom
cstrahan:kube-test-fix
Apr 4, 2018
Merged

nixos: kubernetes fixes#37218
cstrahan merged 1 commit intoNixOS:masterfrom
cstrahan:kube-test-fix

Conversation

@cstrahan
Copy link
Copy Markdown
Contributor

@cstrahan cstrahan commented Mar 17, 2018

  • Fix reference CNI plugins

    • The plugins were split out of the upstream cni repo around version
      0.6.0

  • Fix RBAC and DNS tests

    • Fix broken apiVersion fields
    • Change plugin linking to look in ${package}/bin rather than
      ${package.plugins}

  • Initial work towards a working e2e test

    • Test still fails, but at least the expression evaluates now

Continues @srhb's work in #37199

Fixes #37199

Motivation for this change
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

@cstrahan cstrahan mentioned this pull request Mar 17, 2018
Closed
8 tasks
@GrahamcOfBorg GrahamcOfBorg added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: module (update) This PR changes an existing module in `nixos/` 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels Mar 17, 2018
@cstrahan cstrahan force-pushed the kube-test-fix branch 2 times, most recently from d69358f to e359258 Compare March 17, 2018 06:12
@cstrahan
Copy link
Copy Markdown
Contributor Author

cstrahan commented Mar 17, 2018

@GrahamcOfBorg test kubernetes.dns kubernetes.rbac

@GrahamcOfBorg
Copy link
Copy Markdown

GrahamcOfBorg commented Mar 17, 2018

Success on x86_64-linux (full log)

Attempted: tests.kubernetes.dns, tests.kubernetes.rbac

No log is available.

@GrahamcOfBorg
Copy link
Copy Markdown

GrahamcOfBorg commented Mar 17, 2018

Success on aarch64-linux (full log)

Attempted: tests.kubernetes.dns, tests.kubernetes.rbac

No log is available.

@cstrahan
Copy link
Copy Markdown
Contributor Author

cstrahan commented Mar 17, 2018

@grahamc Any idea why OfBorg is reporting success, while the logs show a 404 served up by nginx?

@grahamc
Copy link
Copy Markdown
Member

grahamc commented Mar 17, 2018

...yikes... this is ugly!

@grahamc
Copy link
Copy Markdown
Member

grahamc commented Mar 17, 2018

The problem is these evaluate to nothing:

[nix-shell:~/projects/nixpkgs]$ HOME=/homeless-shelter NIX_PATH=nixpkgs=$(pwd) nix-instantiate ./nixos/release.nix -A tests.kubernetes.rbac --option restrict-eval true --option build-timeout 1800 --arg supportedSystem '["x86_64-linux"]' --show-trace

[nix-shell:~/projects/nixpkgs]$ HOME=/homeless-shelter NIX_PATH=nixpkgs=$(pwd) nix-instantiate ./nixos/release.nix -A tests.kubernetes.dns --option restrict-eval true --option build-timeout 1800 --arg supportedSystem '["x86_64-linux"]' --show-trace

so evaluation "Succeeds" but then there is nothing to do. I'll write a test and fix this!

@grahamc grahamc mentioned this pull request Mar 17, 2018
Open
grahamc
grahamc reviewed Mar 17, 2018
View reviewed changes
nixos/release.nix Outdated
Copy link
Copy Markdown
Member

@grahamc grahamc Mar 17, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note callSubTestsOnTheseSystems might need recurseForDerivations applied to it to make this work.

@cstrahan
Copy link
Copy Markdown
Contributor Author

cstrahan commented Mar 30, 2018

@GrahamcOfBorg test kubernetes.dns.multinode kubernetes.dns.singlenode kubernetes.rbac.multinode kubernetes.rbac.singlenode

@GrahamcOfBorg
Copy link
Copy Markdown

GrahamcOfBorg commented Mar 30, 2018

Failure on x86_64-linux (full log)

Partial log (click to expand)

Merge failed

@GrahamcOfBorg
Copy link
Copy Markdown

GrahamcOfBorg commented Mar 30, 2018

Failure on aarch64-linux (full log)

Partial log (click to expand)

Merge failed

@cstrahan
Copy link
Copy Markdown
Contributor Author

cstrahan commented Mar 30, 2018

@GrahamcOfBorg test kubernetes.dns.multinode kubernetes.dns.singlenode kubernetes.rbac.multinode kubernetes.rbac.singlenode

@GrahamcOfBorg
Copy link
Copy Markdown

GrahamcOfBorg commented Mar 30, 2018

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: tests.kubernetes.dns.multinode, tests.kubernetes.dns.singlenode, tests.kubernetes.rbac.multinode, tests.kubernetes.rbac.singlenode

Partial log (click to expand)


Cannot nix-instantiate `tests.kubernetes.dns.singlenode' because:
�[31;1merror:�[0m undefined variable 'callSubTestsOnTheseSystems' at �[1m/var/lib/gc-of-borg/nix-test-rs-25/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-25/nixos/release.nix�[0m:299:26

Cannot nix-instantiate `tests.kubernetes.rbac.multinode' because:
�[31;1merror:�[0m undefined variable 'callSubTestsOnTheseSystems' at �[1m/var/lib/gc-of-borg/nix-test-rs-25/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-25/nixos/release.nix�[0m:302:27

Cannot nix-instantiate `tests.kubernetes.rbac.singlenode' because:
�[31;1merror:�[0m undefined variable 'callSubTestsOnTheseSystems' at �[1m/var/lib/gc-of-borg/nix-test-rs-25/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-25/nixos/release.nix�[0m:302:27

@GrahamcOfBorg
Copy link
Copy Markdown

GrahamcOfBorg commented Mar 30, 2018

No attempt on x86_64-linux (full log)

The following builds were skipped because they don't evaluate on x86_64-linux: tests.kubernetes.dns.multinode, tests.kubernetes.dns.singlenode, tests.kubernetes.rbac.multinode, tests.kubernetes.rbac.singlenode

Partial log (click to expand)


Cannot nix-instantiate `tests.kubernetes.dns.singlenode' because:
error: undefined variable 'callSubTestsOnTheseSystems' at /var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-0-gustav.ewr1.nix.ci/nixos/release.nix:299:26

Cannot nix-instantiate `tests.kubernetes.rbac.multinode' because:
error: undefined variable 'callSubTestsOnTheseSystems' at /var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-0-gustav.ewr1.nix.ci/nixos/release.nix:302:27

Cannot nix-instantiate `tests.kubernetes.rbac.singlenode' because:
error: undefined variable 'callSubTestsOnTheseSystems' at /var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-0-gustav.ewr1.nix.ci/nixos/release.nix:302:27

@cstrahan
nixos: kubernetes fixes
709b6f6 
* Fix reference CNI plugins
  * The plugins were split out of the upstream cni repo around version
    0.6.0

* Fix RBAC and DNS tests
  * Fix broken apiVersion fields
  * Change plugin linking to look in ${package}/bin rather than
    ${package.plugins}

* Initial work towards a working e2e test
  * Test still fails, but at least the expression evaluates now

Continues @srhb's work in NixOS#37199

Fixes NixOS#37199
@cstrahan
Copy link
Copy Markdown
Contributor Author

cstrahan commented Mar 30, 2018

@GrahamcOfBorg test kubernetes.dns.multinode kubernetes.dns.singlenode kubernetes.rbac.multinode kubernetes.rbac.singlenode

@GrahamcOfBorg
Copy link
Copy Markdown

GrahamcOfBorg commented Mar 30, 2018

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: tests.kubernetes.dns.multinode, tests.kubernetes.dns.singlenode, tests.kubernetes.rbac.multinode, tests.kubernetes.rbac.singlenode

Partial log (click to expand)


Cannot nix-instantiate `tests.kubernetes.dns.singlenode' because:
�[31;1merror:�[0m attribute 'singlenode' in selection path 'tests.kubernetes.dns.singlenode' not found

Cannot nix-instantiate `tests.kubernetes.rbac.multinode' because:
�[31;1merror:�[0m attribute 'multinode' in selection path 'tests.kubernetes.rbac.multinode' not found

Cannot nix-instantiate `tests.kubernetes.rbac.singlenode' because:
�[31;1merror:�[0m attribute 'singlenode' in selection path 'tests.kubernetes.rbac.singlenode' not found

@GrahamcOfBorg
Copy link
Copy Markdown

GrahamcOfBorg commented Mar 30, 2018

Failure on x86_64-linux (full log)

Attempted: tests.kubernetes.dns.multinode, tests.kubernetes.dns.singlenode, tests.kubernetes.rbac.multinode, tests.kubernetes.rbac.singlenode

Partial log (click to expand)

cannot build derivation '/nix/store/pkm7dq6b6qnsb8k46iywxqsjs7xcidgp-nixos-vm.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/aawf1ly7j262rrlrxidrwvxh0p9mj283-nixos-test-driver-kubernetes-dns-multinode.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/jx3kc668x3ydr120k48c4w3d46yj09sx-nixos-test-driver-kubernetes-dns-singlenode.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/wgv64py0y1rwclvv0jg3qcyfkbbyfsc2-nixos-test-driver-kubernetes-rbac-multinode.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/wj6qhqrqxy20lbbgfw2pcrdxdwjs61l3-nixos-test-driver-kubernetes-rbac-singlenode.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/dc8zhjfjqylhmhqwkm7klnq5dhq33nzr-vm-test-run-kubernetes-dns-multinode.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/9k0ckg7fchcp0lscja73lamrpnprqr4m-vm-test-run-kubernetes-dns-singlenode.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/ykzh4p0ax100cm4clx77w563jzcsfq0r-vm-test-run-kubernetes-rbac-multinode.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/kdbwqf3ynzlbrb6vqc9h39sywii6cbaz-vm-test-run-kubernetes-rbac-singlenode.drv': 1 dependencies couldn't be built
�[31;1merror:�[0m build of '/nix/store/9k0ckg7fchcp0lscja73lamrpnprqr4m-vm-test-run-kubernetes-dns-singlenode.drv', '/nix/store/dc8zhjfjqylhmhqwkm7klnq5dhq33nzr-vm-test-run-kubernetes-dns-multinode.drv', '/nix/store/kdbwqf3ynzlbrb6vqc9h39sywii6cbaz-vm-test-run-kubernetes-rbac-singlenode.drv', '/nix/store/ykzh4p0ax100cm4clx77w563jzcsfq0r-vm-test-run-kubernetes-rbac-multinode.drv' failed

@cstrahan
Copy link
Copy Markdown
Contributor Author

cstrahan commented Mar 30, 2018

@grahamc Do you know why OfBorg is failing here?

qemu-system-x86_64: failed to initialize KVM: Permission denied

@7c6f434c
Copy link
Copy Markdown
Member

7c6f434c commented Mar 30, 2018 via email

@cstrahan
Copy link
Copy Markdown
Contributor Author

cstrahan commented Apr 4, 2018

The tests passed locally, so I'm going to stop sitting on this and merge.

@cstrahan cstrahan merged commit 5c066e2 into NixOS:master Apr 4, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@grahamc grahamc grahamc left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: module (update) This PR changes an existing module in `nixos/` 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@cstrahan @GrahamcOfBorg @grahamc @7c6f434c