Skip to content

nixos/radicle: init services#314440

Merged
SuperSandro2000 merged 1 commit intoNixOS:masterfrom
ju1m:radicle
Jul 17, 2024
Merged

nixos/radicle: init services#314440
SuperSandro2000 merged 1 commit intoNixOS:masterfrom
ju1m:radicle

Conversation

@ju1m
Copy link
Copy Markdown
Contributor

@ju1m ju1m commented May 25, 2024

Description of changes

  • Add radicle-node.service
  • Add radicle-httpd.service
    • Add radicle-httpd to pkgs.radicle-node radicle-httpd has been put in a separate package in another PR.
  • Add nixosTests.radicle
    • Add test for radicle-httpd

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@github-actions github-actions bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: module (update) This PR changes an existing module in `nixos/` labels May 25, 2024
@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels May 25, 2024
@ju1m ju1m force-pushed the radicle branch 2 times, most recently from 0ebff93 to 9ea3197 Compare May 26, 2024 00:36
@h7x4 h7x4 added the 8.has: module (new) This PR adds a module in `nixos/` label May 26, 2024
@ju1m ju1m force-pushed the radicle branch 2 times, most recently from 24b5fb5 to c9f82a0 Compare May 28, 2024 01:01
@ju1m ju1m requested a review from lorenzleutgeb May 28, 2024 01:12
@ofborg ofborg bot requested a review from amesgen May 28, 2024 01:41
@ofborg ofborg bot added 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-darwin: 1 This PR causes 1 package to rebuild on Darwin. and removed 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. labels May 28, 2024
Copy link
Copy Markdown
Member

@gador gador left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for all your work towards radicle in NixOS! I like how you structured the module. I only have a few remarks.
When adding the test to all-tests.nix the nixosTests pass without issues 👍

@lorenzleutgeb
Copy link
Copy Markdown
Member

@ju1m thanks a lot for picking this up. I am actually in Berlin with the Radicle team this week, which is why I couldn't review this today. Hopefully tomorrow! :)

@lorenzleutgeb
Copy link
Copy Markdown
Member

lorenzleutgeb commented May 28, 2024

How about splitting into more packages? The web stuff is not necessary to run and administer a node. I have sketched that here: #309050 (comment)

@ju1m
Copy link
Copy Markdown
Contributor Author

ju1m commented May 28, 2024

How about splitting into more packages? The web stuff is not necessary to run and administer a node. I have sketched that here: #309050 (comment)

Sure, though not being used to Rust packaging I went to the most expedient in this PR. I think the splitting can go in another PR, I'll amend this PR to introduce services.radicle.httpd.{enable,package}.

@ju1m
Copy link
Copy Markdown
Contributor Author

ju1m commented May 28, 2024

@ju1m thanks a lot for picking this up. I am actually in Berlin with the Radicle team this week, which is why I couldn't review this today. Hopefully tomorrow! :)

@lorenzleutgeb, no hurry, thanks to the team for making such a promising tool, and thanks to you for pushing its integration into Nixpkgs, that's what caught my attention over it.
I'm not sure I'll commit to using radicle yet due to the "high" level of network I/O and CPU cycles it seems to require to run in sync with the public peers. But the tool is young, so maybe it will get lightweight enough for me at some point. In any case I'm adding myself as a maintainer for now to keep being notified if some help is needed in the future for that service.

@lorenzleutgeb
Copy link
Copy Markdown
Member

Sure, though not being used to Rust packaging I went to the most expedient in this PR. I think the splitting can go in another PR, I'll amend this PR to introduce services.radicle.httpd.{enable,package}.

There even is talk to move radicle-httpd (and I guess also rad-web) into a separate repo. When that happens, at the latest, the justification for the split would be more obvious. I wanted to anticipate this, and keep the changes to Nix users at a minimum (fomr radicle-cli to radicle-node, and then to radicle-web is a bit noisy).

Then there's also the question whether it should be services.radicle-{httpd,node}.* or services.radicle.{httpd,node}.*. I don't know whether there is a clear policy. For example, there is services.buildbot-{master,worker} but also services.salt.{master,minion}.

@lorenzleutgeb, no hurry, thanks to the team for making such a promising tool, and thanks to you for pushing its integration into Nixpkgs, that's what caught my attention over it. I'm not sure I'll commit to using radicle yet due to the "high" level of network I/O and CPU cycles it seems to require to run in sync with the public peers. But the tool is young, so maybe it will get lightweight enough for me at some point. In any case I'm adding myself as a maintainer for now to keep being notified if some help is needed in the future for that service.

Sweet! Yeah, there was some breakage in the protocol recently, and a nodes are now announcing the version they are running, to make upgrades smoother.

Note that high I/O and CPU is being worked on:

@lorenzleutgeb
Copy link
Copy Markdown
Member

lorenzleutgeb commented May 29, 2024

In my module, I implemented a check to catch invalid configuration in settings. Any particular reason not to have this?

https://github.com/radicle-nix/radicle-nix/blob/9c1adbb7d85d12df6236e006c12ac089cb18341b/os/module/services/radicle.nix#L34-L48

Here with a more obviously useless key:

https://github.com/radicle-nix/radicle-nix/blob/9c1adbb7d85d12df6236e006c12ac089cb18341b/hm/module/programs/radicle.nix#L45-L59

@ju1m
Copy link
Copy Markdown
Contributor Author

ju1m commented Jun 27, 2024

@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. and removed 10.rebuild-darwin: 1 This PR causes 1 package to rebuild on Darwin. 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. labels Jun 27, 2024
@ju1m
Copy link
Copy Markdown
Contributor Author

ju1m commented Jun 28, 2024

  • Drop DynamicUser= for the sake of clarity, since the same level of systemd-analyze security can be achieved without it. Note that therefore the state directory is no longer /var/lib/private/radicle but /var/lib/radicle.

@ju1m
Copy link
Copy Markdown
Contributor Author

ju1m commented Jul 5, 2024

  • Rebase after conflict in nixos/doc/manual/release-notes/rl-2411.section.md following the merge of 28512d1

@lorenzleutgeb
Copy link
Copy Markdown
Member

lorenzleutgeb commented Jul 17, 2024

I'd really like to see this merged. Any ideas on how we can get someone with write permissions on board? @SuperSandro2000 could you do another review or merge please? 😃

@SuperSandro2000 SuperSandro2000 merged commit 51fcc2c into NixOS:master Jul 17, 2024
@ju1m ju1m deleted the radicle branch July 18, 2024 20:50
@dpc
Copy link
Copy Markdown
Contributor

dpc commented Jul 24, 2024

I just spent a lot of time debugging what came down to:

   services.radicle = {
     enable = true;
     privateKeyFile = "/run/secrets/radicle/seednode";
-    publicKeyFile = "/run/secrets/radicle/seednode-pub";
+    publicKeyFile = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINIxyouLgGi/RGEeBycs47NuntkAWn6ZAynImFVYsGN6 dpc@ren";
     httpd.enable = true;
     httpd.nginx.serverName = "radicle.dpc.pw";
   };

Without this fix, the service would fail with:

Jul 24 18:16:55 hetz radicle-httpd[600666]: 2024-07-24T18:16:55.284465Z ERROR Fatal: ssh keygen: length invalid: length invalid: length invalid

apparently because it was trying to parse the path to the pubkey file, instead of its content.

It seems like a gotcha or a bug that privateKeyFile and publicKeyFile are named the same way, but have different semantics. No idea if it's some NixOS convention or not, just reporting for reference, search engines, etc. @ju1m @SuperSandro2000

@ju1m
Copy link
Copy Markdown
Contributor Author

ju1m commented Jul 24, 2024

@dpc, I'm sorry this bug made you loose your time.
Both path and str types should have worked because services.radicle currently uses:

"${if isPath cfg.publicKeyFile then cfg.publicKeyFile else pkgs.writeText "radicle.pub" cfg.publicKeyFile}:${env.RAD_HOME}/keys/radicle.pub"

But using isPath is wrong here, because it does not consider "/run/secrets/radicle/seednode-pub" as a path:

$ nix repl
nix-repl> builtins.isPath "/run/secrets/radicle/seednode-pub"
false
nix-repl> builtins.isPath /run/secrets/radicle/seednode-pub  
true

Instead, types.path.check should have been used to also consider as a path any string prefixed by "/":

check = x: isStringLike x && builtins.substring 0 1 (toString x) == "/";

See #329731 for a fix.

@dpc
Copy link
Copy Markdown
Contributor

dpc commented Jul 24, 2024

I'm happy to help improve it. :)

BTW. Is there any existing Wiki page/guide? After getting my node set up, I have no idea how to get it to do things for me (like seed my repos).

@ju1m
Copy link
Copy Markdown
Contributor Author

ju1m commented Jul 24, 2024

@dpc, you can rad-system seed $rid,
where rad-system is installed by services.radicle to execute rad inside the namespaces of radicle-node.service,
and $rid is given by rad . in your git repository.

@dpc
Copy link
Copy Markdown
Contributor

dpc commented Jul 25, 2024

Had connectivity issues because the app wouldn't bind to ipv6.

Setting:

+    node.listenAddress = "[::0]";

fixed it.

@dpc
Copy link
Copy Markdown
Contributor

dpc commented Jul 25, 2024

BTW. It would be great if:

+    settings = {
+        "web" = {
+          "pinned" = {
+            "repositories" = [
+              "rad:z2eeB9LF8fDNJQaEcvAWxQmU7h2PG"
+            ];
+          };
+        };
+    };

could just have its own key. What do you think?

@ju1m
Copy link
Copy Markdown
Contributor Author

ju1m commented Jul 25, 2024

could just have its own key. What do you think?

@dpc, you mean to add an alias to settings.web.pinned.repositories? what problem would it solve?
Note that we previously reverted from type-checking settings.

@dpc
Copy link
Copy Markdown
Contributor

dpc commented Jul 25, 2024

@ju1m I wish i could just:

  services.radicle = {
    enable = true;
    # ..
    pinned = [ "rad:z2eeB9LF8fDNJQaEcvAWxQmU7h2PG" ];
  };

as it seems rather common need, and having to figure out and mimick config-file structure
seems like an unnecessary chore. :D

@ju1m
Copy link
Copy Markdown
Contributor Author

ju1m commented Jul 25, 2024

@dpc, you're raising a discoverability concern on NixOS but I will argue that it belongs to Radicle. Using settings and its config-mimicking, instead of inventing ad-hoc options like NixOS used to do before the settings framework existed, is what enables NixOS to let the burden of discoverability/maintainability onto Radicle's documentation/code where it's best addressed. Yet, in NixOS, we can address the common need you raised to set pinned repositories by setting an example for settings.
See https://github.com/NixOS/nixpkgs/pull/330021/files#diff-47ed1acddaad94538b9ee7995ffa8d7cc1376f9667350acee9cec912cec6a3bfR183

@dpc
Copy link
Copy Markdown
Contributor

dpc commented Jul 25, 2024

Thanks! Fine with me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: changelog This PR adds or changes release notes 8.has: documentation This PR adds or changes documentation 8.has: module (new) This PR adds a module in `nixos/` 8.has: module (update) This PR changes an existing module in `nixos/` 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants