xen: 4.15 -> {4.16,4.17,4.18,4.19} by SigmaSquadron · Pull Request #324693 · NixOS/nixpkgs

SigmaSquadron · 2024-07-04T22:39:28Z

Description of changes

Required by #324911.

Updates Xen to 4.16.6, 4.17.4, 4.18.2 and 4.19.0 on nixpkgs/master.
Marks a couple of packages as broken, as they are outdated, and their older versions do not seem to support Xen 4.19.

Things done

Built on platform(s)
- x86_64-linux
Built with sandboxing enabled.
- sandbox = true
Tested, as applicable:
- pkg-config test passes successfully.
Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD".
Tested basic functionality of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

Closes #320335, closes #26899.

Add a 👍 reaction to pull requests you find important.

pkgs/applications/qubes/qubes-core-vchan-xen/default.nix

SigmaSquadron · 2024-07-05T00:46:09Z

Fixed the unpatched binaries.

Keep in mind XKCD 1513 as you review.

ulrikstrid · 2024-07-05T19:41:18Z

Would you be willing to cherry-pick the maintainers and ocamlPackages.xenstore commits to a separate PR and add me as reviewer?

Then we can try to merge bits and pieces of this so that only the gnarly stuff that I'm not comfortable with is left 😄

SigmaSquadron · 2024-07-30T15:49:11Z

This will currently fail to build due to #331107.

Kreyren · 2024-07-30T16:48:12Z

This will currently fail to build due to #331107. -- @SigmaSquadron (#324693 (comment))

Is there anything else that blocks review for merge?

SigmaSquadron · 2024-07-30T17:04:16Z

Is there anything else that blocks review for merge?

So far, nobody who reviewed feels comfortable enough with Xen to approve this PR, so the fact this has no approvals is a possible blocker.

Kreyren · 2024-07-30T18:02:52Z

So far, nobody who reviewed feels comfortable enough with Xen to approve this PR, so the fact this has no approvals is a possible blocker. -- @SigmaSquadron (#324693 (comment))

I meant implementation-wise like is there anything you still feel like needs to be handled or once the referenced issue is solved do you feel like it's ready for merge?

SigmaSquadron · 2024-07-30T18:35:02Z

Yes, I do believe this is ready to merge, but we have an issue where there is an abundance of reviewers who are knowledgeable about Nix, and an abundance of reviewers that are knowledgeable about Xen, but basically no active reviewers who are knowledgeable about both.

pkgs/applications/virtualization/xen/generic.nix

Kreyren · 2024-07-30T19:40:46Z

Yes, I do believe this is ready to merge, but we have an issue where there is an abundance of reviewers who are knowledgeable about Nix, and an abundance of reviewers that are knowledgeable about Xen, but basically no active reviewers who are knowledgeable about both. -- @SigmaSquadron (#324693 (comment))

I don't think that is an issue, rather the blocking issue seems to be security considering the nature of Xen (also some people like andy above seem to be working on infra adjustments), but that (to me at least) seems like is sufficiently managed for NixOS standards.
That said i would prefer ~3 days of delay so that people who are concerned with Xen security can test it more, but my main concern now is just the 4,19 release that will stay in unstable anyway so that seems like good management to me.

radhus

Great work! Don't know if my approval mean much - but I think this overall makes Xen way more easily maintainable now!

As there aren't any automated dom0 tests, if the code looks fine from both a Xen and Nixpkgs perspective, and you're able to manually test dom0 with some domU's, shouldn't we just consider merging and let any issues be fixed in upcoming PRs, given that you're the maintainer going forward? 🤗

Kreyren · 2024-07-30T19:47:05Z

@andyhhp Considering your reply in #324693 (comment) are you planing to co-maintain this package to address thomas's concern to make the package more reliable?

Optionally i am willing to help with maintenance if none's against it as NiXium's projected to use Xen on majority of it's systems.

EDIT: sorry i though you meant cleaning NixOS's CIs, feel free to ignore my question then.

Kreyren · 2024-07-30T19:53:32Z

As there aren't any automated dom0 tests, if the code looks fine from both a Xen and Nixpkgs perspective, and you're able to manually test dom0 with some domU's, shouldn't we just consider merging and let any issues be fixed in upcoming PRs -- @radhus (#324693 (review))

Yea, the lack of automated tests is kinda a problem too for a production environment, but in terms of management it's probably better to merge in the current state and add them later as this is already getting kinda complex to review and i am not sure what should we be checking in the nixos context beyond the obvious once.

SigmaSquadron · 2024-07-30T20:06:44Z

Yea, the lack of automated tests is kinda a problem too for a production environment, but in terms of management it's probably better to merge in the current state and add them later

In my opinion, it's next to impossible to add more tests, at least without rewriting the Nixpkgs test infrastructure. Xen requires a bare-metal host to be a dom0 in, which isn't the case with NixOS tests, much less package tests

andyhhp · 2024-07-30T20:08:50Z

@andyhhp Considering your reply in #324693 (comment) are you planing to co-maintain this package to address thomas's concern to make the package more reliable?

Sorry, I'm an upstream Xen developer, and can advise on that side of things, but I don't have time (or indeed the expertise) to advise on the NixOS side of things.

Kreyren · 2024-07-30T22:12:32Z

Yea, the lack of automated tests is kinda a problem too for a production environment, but in terms of management it's probably better to merge in the current state and add them later

In my opinion, it's next to impossible to add more tests, at least without rewriting the Nixpkgs test infrastructure. Xen requires a bare-metal host to be a dom0 in, which isn't the case with NixOS tests, much less package tests -- @SigmaSquadron (#324693 (comment))

That sounds like an over-reaction, the infra should have the tools needed, but it needs to b researched to figure out how to implement them else RFC

Sorry, I'm an upstream Xen developer, and can advise on that side of things, but I don't have time (or indeed the expertise) to advise on the NixOS side of things. -- @andyhhp (#324693 (comment))

My apologies i confused you with nix developer and realized my mistake shortly after.

SigmaSquadron · 2024-07-30T22:32:17Z

That sounds like an over-reaction, the infra should have the tools needed, but it needs to b researched to figure out how to implement them else RFC

Xen requires a bare-metal host to run. It's not a question of tooling, it's a question of not running tests inside containers/VMs, which isn't supported by Nixpkgs, and would require a tremendous amount of resources to effectively implement.

SigmaSquadron · 2024-07-31T00:04:37Z

Dropped the dependency that wasn't compiling. I doubt anyone would actually use it.
Dropped unused dependencies per an upstream review.
Now passes nixpkgs-hammering xen -e missing-patch-comment -e explicit-phases.

Hopefully the final nixpkgs-review.

Result of nixpkgs-review pr 324693 run on x86_64-linux 1

2 packages marked as broken and skipped:

libvmi
qubes-core-vchan-xen

45 packages built:

qemu_xen (qemu_xen_4_19)
qemu_xen.debug (qemu_xen_4_19.debug)
qemu_xen.ga (qemu_xen_4_19.ga)
qemu_xen_4_16
qemu_xen_4_16.debug
qemu_xen_4_16.ga
qemu_xen_4_17
qemu_xen_4_17.debug
qemu_xen_4_17.ga
qemu_xen_4_18
qemu_xen_4_18.debug
qemu_xen_4_18.ga
xen
xen-guest-agent
xen-slim
xen-slim.boot
xen-slim.dev
xen-slim.man
xen.boot
xen.dev
xen.man
xenPackages.xen_4_16
xenPackages.xen_4_16-slim
xenPackages.xen_4_16-slim.boot
xenPackages.xen_4_16-slim.dev
xenPackages.xen_4_16-slim.man
xenPackages.xen_4_16.boot
xenPackages.xen_4_16.dev
xenPackages.xen_4_16.man
xenPackages.xen_4_17
xenPackages.xen_4_17-slim
xenPackages.xen_4_17-slim.boot
xenPackages.xen_4_17-slim.dev
xenPackages.xen_4_17-slim.man
xenPackages.xen_4_17.boot
xenPackages.xen_4_17.dev
xenPackages.xen_4_17.man
xenPackages.xen_4_18
xenPackages.xen_4_18-slim
xenPackages.xen_4_18-slim.boot
xenPackages.xen_4_18-slim.dev
xenPackages.xen_4_18-slim.man
xenPackages.xen_4_18.boot
xenPackages.xen_4_18.dev
xenPackages.xen_4_18.man

Kreyren

As discussed in nixos devel chat and with nixos security team representative for the approach to get this implemented and maintained.

Refer to the linked discussion for usage notes in production/mission critical environment.

There are too many changes to list between these versions, but here's what's important for Nixpkgs: - Enabled xen.efi building, allowing the nixos/xen module to support EFI booting. - Renamed xen-light to xen-slim, and removed the old xen-slim package, as qemu-traditional has been removed per an upstream recommendation. - xen-slim (formerly xen-light) no longer builds iPXE. - You can now use the OVMF from Xen's repos. - Further generalised generic.nix. Maintaining three versions at once is finally doable. - Removed as many recursions as possible. - Split the package output. Signed-off-by: Fernando Rodrigues <alpha@sigmasquadron.net>

The update script is interactive, not automated, and is meant to run with human intervention in order to verify Xen's code signature. It produces default.nix files for all security-supported branches. Signed-off-by: Fernando Rodrigues <alpha@sigmasquadron.net>

xen-light was dropped in favour of xen and xen-slim Signed-off-by: Fernando Rodrigues <alpha@sigmasquadron.net> Reviewed-by: Matei Dibu <contact@mateidibu.dev>

In function 'libvchan__check_domain_alive': error: unknown type name 'xc_dominfo_t'; did you mean 'xc_meminfo_t'? error: implicit declaration of function 'xc_domain_getinfo'; did you mean 'xc_domain_getvnuma'? [-Werror=implicit-function-declaration] error: request for member 'domid' in something not a structure or union error: request for member 'dying' in something not a structure or union Signed-off-by: Fernando Rodrigues <alpha@sigmasquadron.net> Reviewed-by: Joachim Ernst <mail-maintainer@0x4A6F.dev>

In file included from ./driver/xen/xen_private.h:42, error: unknown type name 'xc_dominfo_t'; did you mean 'xc_meminfo_t'? PR NixOS#328873 reverts this commit and updates LibVMI. Signed-off-by: Fernando Rodrigues <alpha@sigmasquadron.net>

emilazy · 2024-08-01T17:32:13Z

Congrats on this huge achievement!

NixOSInfra added the 12.first-time contribution This PR is the author's first one; please be gentle! label Jul 4, 2024

This comment was marked as outdated.

Sign in to view

SigmaSquadron commented Jul 4, 2024

View reviewed changes

pkgs/applications/qubes/qubes-core-vchan-xen/default.nix Outdated Show resolved Hide resolved

SigmaSquadron force-pushed the xen-4.18 branch 2 times, most recently from 8dbcbe0 to 80dcdc6 Compare July 5, 2024 00:34

SigmaSquadron marked this pull request as ready for review July 5, 2024 00:34

SigmaSquadron requested review from JulienMalka and ulrikstrid as code owners July 5, 2024 00:34

This comment was marked as outdated.

Sign in to view

ofborg bot requested review from 0x4A6F and matdibu July 5, 2024 01:43

SigmaSquadron force-pushed the xen-4.18 branch from 80dcdc6 to 2823edd Compare July 5, 2024 02:18

SigmaSquadron force-pushed the xen-4.18 branch from 2823edd to eb884ff Compare July 5, 2024 22:10

github-actions bot removed the 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS label Jul 5, 2024

SigmaSquadron force-pushed the xen-4.18 branch from 102fd16 to 4ded80c Compare July 30, 2024 13:17

SigmaSquadron changed the title ~~xen: 4.15 -> {4.16,4.17,4.18}~~ xen: 4.15 -> {4.16,4.17,4.18,4.19} Jul 30, 2024

SigmaSquadron force-pushed the xen-4.18 branch from 4ded80c to a4b06ca Compare July 30, 2024 15:01

SigmaSquadron mentioned this pull request Jul 30, 2024

Build failure: ocf-resource-agents #331107

Closed

andyhhp reviewed Jul 30, 2024

View reviewed changes

pkgs/applications/virtualization/xen/generic.nix Outdated Show resolved Hide resolved

radhus approved these changes Jul 30, 2024

View reviewed changes

Kreyren approved these changes Jul 31, 2024

View reviewed changes

SigmaSquadron added 5 commits July 31, 2024 05:57

xen: rename xen-light to xen-slim

3e1ca12

xen-light was dropped in favour of xen and xen-slim Signed-off-by: Fernando Rodrigues <alpha@sigmasquadron.net> Reviewed-by: Matei Dibu <contact@mateidibu.dev>

libvmi: mark broken

8d4151a

In file included from ./driver/xen/xen_private.h:42, error: unknown type name 'xc_dominfo_t'; did you mean 'xc_meminfo_t'? PR NixOS#328873 reverts this commit and updates LibVMI. Signed-off-by: Fernando Rodrigues <alpha@sigmasquadron.net>

This was referenced Aug 2, 2024

xen: documentation and build fixes #331867

Merged

freeshow: init at 1.2.1 #332658

Draft

SigmaSquadron mentioned this pull request Dec 2, 2024

citrix-workspace: 24.5.0.76 -> 24.8.0.98 #360106

Merged

13 tasks

SigmaSquadron mentioned this pull request Aug 9, 2025

[2024-2025] Nixpkgs committer nominations NixOS/nixpkgs-committers#50

Closed

Uh oh!

Conversation

SigmaSquadron commented Jul 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of changes

Things done

Closes #320335, closes #26899.

Uh oh!

This comment was marked as outdated.

Uh oh!

SigmaSquadron commented Jul 5, 2024

Uh oh!

This comment was marked as outdated.

ulrikstrid commented Jul 5, 2024

Uh oh!

SigmaSquadron commented Jul 30, 2024

Uh oh!

Kreyren commented Jul 30, 2024

Uh oh!

SigmaSquadron commented Jul 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Kreyren commented Jul 30, 2024

Uh oh!

SigmaSquadron commented Jul 30, 2024

Uh oh!

Uh oh!

Kreyren commented Jul 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

radhus left a comment

Choose a reason for hiding this comment

Uh oh!

Kreyren commented Jul 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Kreyren commented Jul 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SigmaSquadron commented Jul 30, 2024

Uh oh!

andyhhp commented Jul 30, 2024

Uh oh!

Kreyren commented Jul 30, 2024

Uh oh!

SigmaSquadron commented Jul 30, 2024

Uh oh!

SigmaSquadron commented Jul 31, 2024

Uh oh!

Kreyren left a comment

Choose a reason for hiding this comment

Uh oh!

emilazy commented Aug 1, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

14 participants

SigmaSquadron commented Jul 4, 2024 •

edited

Loading

SigmaSquadron commented Jul 30, 2024 •

edited

Loading

Kreyren commented Jul 30, 2024 •

edited

Loading

Kreyren commented Jul 30, 2024 •

edited

Loading

Kreyren commented Jul 30, 2024 •

edited

Loading