Skip to content

nixos/systemd-boot: Fix installed version regexp#140046

Merged
flokli merged 5 commits intoNixOS:stagingfrom
jrobsonchase:systemd-boot/fix-regexp
Nov 6, 2021
Merged

nixos/systemd-boot: Fix installed version regexp#140046
flokli merged 5 commits intoNixOS:stagingfrom
jrobsonchase:systemd-boot/fix-regexp

Conversation

@jrobsonchase
Copy link
Copy Markdown
Contributor

Motivation for this change

The regexp was only matching numbers and not the '.', so everyone using
systemd-boot would always see could not find any previously installed systemd-boot on a nixos-rebuild.

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
      - [ ] x86_64-darwin
      - [ ] aarch64-darwin
      - [ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip" <- Not sure if this is relevant?
    - [ ] Tested execution of all binary files (usually in ./result/bin/)
  • 21.11 Release Notes (or backporting 21.05 Release notes)
    - [ ] (Package updates) Added a release notes entry if the change is major or breaking
    - [ ] (Module updates) Added a release notes entry if the change is significant
    - [ ] (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

@github-actions github-actions bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: module (update) This PR changes an existing module in `nixos/` labels Sep 30, 2021
@jrobsonchase jrobsonchase changed the title systemd-boot: Fix installed version regexp nixos/systemd-boot: Fix installed version regexp Sep 30, 2021
@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels Sep 30, 2021
@jrobsonchase jrobsonchase force-pushed the systemd-boot/fix-regexp branch 3 times, most recently from aa30ae6 to b8ad1c3 Compare September 30, 2021 15:39
@jrobsonchase
Copy link
Copy Markdown
Contributor Author

Changed the 0-9 to \d and added [^)]+ outside of the capture group. Ready for another review pass.

I'm still on the fence about capturing vs not capturing the extra suffix like in the arch case. Could be swayed either way.

I'm also not sure that this check is actually correct. Should it be "always upgrade and never downgrade" or "make it match the version the module expects?" If the later, an equivalence check would be better than the string order.

Copy link
Copy Markdown
Member

@sugar700 sugar700 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That doesn't seem right. Later in this code the result of this check is used for string comparison. For instance, if a current version of systemd is 249.9, and we are going to install 249.10 then the comparison will fail.

>>> "249.10" > "249.9"
False

(to be fair, the code below was wrong previously, however it likely wouldn't cause issues until release of systemd 1000)

@jrobsonchase
Copy link
Copy Markdown
Contributor Author

Yeah, that's true. We'd need to parse the version as either a float if we assume it'll only ever contain one ., or a semver parser otherwise.

wouldn't cause issues until release of systemd 1000

That assumes that it ever actually got to that check, which it wouldn't 😛

Is there any reason we shouldn't just capture the entire version string and ensure that it always completely matches the version from nixpkgs? It might even be prudent to add a hash to the end of our version to make sure that it's actually the nixos binary and not just one that shares the same version.

@danielfullmer
Copy link
Copy Markdown
Contributor

Just to add a bit of context here since I originally wrote this code: I did incorrectly assume the version string would always just be a series of digits, however, if it was originally installed by NixOS then it would be. From a closer look at the code (boot/efi/boot.c line 28), we see that it is set to whatever GIT_VERSION is set to:

static const char _used_ _section_(".sdmagic") magic[] = "#### LoaderInfo: systemd-boot " GIT_VERSION " ####";

GIT_VERSION is set via src/version/version.h.in from @VCS_TAG@, which can be set via meson using -Dversion-tag. (This is how arch sets their custom version tag).

Another thing is that bootctl update itself will only update the boot loader if it detects that its version is newer than the one present. (see src/boot/bootctl.c, and track how verb_install will indirectly call copy_file_with_version_check, which calls down to compare_version. Earlier this year it looks like they replaced the simple version comparator with a more complicated one (strversmp_improved).

Since bootctl update will already do the version comparison for us, and duplicating that functionality in our code would be error prone, perhaps we can just always run bootctl update? I don't recall my original reasoning for duplicating the version comparison in systemd-boot-builder.py, but if it's not needed then we should try to get rid of it.

@jrobsonchase
Copy link
Copy Markdown
Contributor Author

jrobsonchase commented Oct 1, 2021

Since bootctl update will already do the version comparison for us, and duplicating that functionality in our code would be error prone, perhaps we can just always run bootctl update? I don't recall my original reasoning for duplicating the version comparison in systemd-boot-builder.py, but if it's not needed then we should try to get rid of it

That sounds like the right approach to me. At worst, the bootctl version check will get it wrong and install an older version from nixpkgs, which is arguably more correct anyway.

Edit: Hmm, after playing with it, I might understand the original rationale. The version check in bootctl update only skips copying if the installed one is newer. It seems to always update even if it's the same version. It's also rather noisy, though that could probably be suppressed.

@danielfullmer
Copy link
Copy Markdown
Contributor

Looks like this commit would solve that issue, but I don't believe it's in the current systemd used in NixOS yet:
systemd/systemd@e5a8b4b
And we should probably also add --graceful to the options.

@colemickens
Copy link
Copy Markdown
Member

colemickens commented Oct 9, 2021

This is interesting, this change has bigger ramifications:

Ignore failure when the EFI System Partition cannot be found, when EFI variables cannot be written, or a different or newer boot loader is already installed. Currently only applies to random seed and update operations.

I think this would actually fix a case of NixOS installs failing on systems like... say the raspberry pi4, with u-boot ->efi-> systemd-boot. Currently that's not an option because without --graceful, the install fails because it can't access UEFI variables.

It seems like graceful might be ... too graceful?

I really don't get why systemd doesn't default to:

  • warn and skip if the installed version is the same or older
  • have a --reinstall flag to force reinstall
  • leave --graceful the way it is

Seems like it would've allowed more appropriate, granular behavior.

@sternenseemann
Copy link
Copy Markdown
Member

Edit: Hmm, after playing with it, I might understand the original rationale. The version check in bootctl update only skips copying if the installed one is newer. It seems to always update even if it's the same version. It's also rather noisy, though that could probably be suppressed.

Instead of comparing if the new version is greater, I think it would then be sufficient to just check if the version strings are different: If the new version is lower, bootctl update will do nothing (as before); if it is the newer, we'll install the new version.

@jrobsonchase
Copy link
Copy Markdown
Contributor Author

Instead of comparing if the new version is greater, I think it would then be sufficient to just check if the version strings are different

This is exactly what I suggested here, though I was unaware that bootctl had a version check to prevent downgrades. I'm fine with either approach ¯\_(ツ)_/¯

@jrobsonchase jrobsonchase force-pushed the systemd-boot/fix-regexp branch 3 times, most recently from 74bfb31 to 5e2ea28 Compare October 15, 2021 14:17
@jrobsonchase
Copy link
Copy Markdown
Contributor Author

@SuperSandro2000 Is there anything left blocking this from being merged?

@SuperSandro2000
Copy link
Copy Markdown
Member

@SuperSandro2000 Is there anything left blocking this from being merged?

me not knowing anything about systemd-boot.

@nixos-discourse
Copy link
Copy Markdown

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/could-not-find-any-previously-installed-systemd-boot/15627/3

@anund
Copy link
Copy Markdown
Contributor

anund commented Oct 31, 2021

The change works 👍. Working systemd-boot post ~248 exposes some users to systemd/systemd#19191 (see #143847). The? documented way to rollback the bootloader is also broken #144007.

@xaverdh
Copy link
Copy Markdown
Contributor

xaverdh commented Nov 1, 2021

The change works +1. Working systemd-boot post ~248 exposes some users to systemd/systemd#19191 (see #143847). The? documented way to rollback the bootloader is also broken #144007.

Should we patch our systemd with systemd/systemd@e98d271 before merging this?

@xaverdh
Copy link
Copy Markdown
Contributor

xaverdh commented Nov 1, 2021

cc @flokli (re patching systemd)

@flokli
Copy link
Copy Markdown
Member

flokli commented Nov 1, 2021

cc @flokli (re patching systemd)

I replied on the issue for that (#143847 (comment)). Let's do the discussion about patching or not there.

@jrobsonchase jrobsonchase force-pushed the systemd-boot/fix-regexp branch from 5e2ea28 to 51725b4 Compare November 1, 2021 14:56
@arianvp arianvp self-requested a review November 2, 2021 14:55
@flokli
Copy link
Copy Markdown
Member

flokli commented Nov 5, 2021

I just merged #144422 into staging.

Could this be rebased on top of (and target) staging as well? I'd like to avoid this making bootctl install work again, and potentially breaking machines affected by #143847 until the fix from staging ends up in master.

This should fail since the regexp in systemd-boot-builder.py won't match
the '.' in the version string.
The regexp was only matching numbers and not the '.', so everyone using
systemd-boot would always see `could not find any previously installed
systemd-boot` on a `nixos-rebuild`.
bootctl does it as a part of its update process anyway, so we're just
duplicating code.
@jrobsonchase jrobsonchase force-pushed the systemd-boot/fix-regexp branch from 51725b4 to 3efc2de Compare November 5, 2021 16:11
@jrobsonchase jrobsonchase requested a review from dasJ as a code owner November 5, 2021 16:11
@jrobsonchase jrobsonchase changed the base branch from master to staging November 5, 2021 16:12
@jrobsonchase
Copy link
Copy Markdown
Contributor Author

Could this be rebased on top of (and target) staging as well?

Done!

@flokli flokli merged commit 6ae2715 into NixOS:staging Nov 6, 2021
@flokli
Copy link
Copy Markdown
Member

flokli commented Nov 6, 2021

Thanks!

@flokli
Copy link
Copy Markdown
Member

flokli commented Nov 6, 2021

Those should now both end in the staging-next cycle after #144730.

@Profpatsch
Copy link
Copy Markdown
Member

WHY IS THIS PARSED ANYWAY

I’m sorry but why the heck are we parsing display output of bootctl? Is there no machine-readable output format?

@Janik-Haag Janik-Haag added the 12.first-time contribution This PR is the author's first one; please be gentle! label Jun 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: module (update) This PR changes an existing module in `nixos/` 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. 12.first-time contribution This PR is the author's first one; please be gentle!

Projects

None yet

Development

Successfully merging this pull request may close these issues.

systemd-boot-builder.py can't deal with versions of the form major.minor