shouldn't it be possible to completely remove the Cargo.lock now that it's checked into git upstream?

Oh drat, I'm sorry. I'm already running testing and I merged #265820 as I wanted bcachefs-tools, I didn't realize this PR also included it. This otherwise looks great; I'm happy to sign off on it and merge it if you don't mind pulling that one out. Thank you!

Done. Went ahead and added an update script and updated to 1.3.3.

@thoughtpolice was that PR needed with my PR waiting in staging?

I feel a little bit blind since I'm not being mentioned in some of the maintenance of the code base, especially since I did a lot of work on the PR, I was hoping my PR would get through quicker, maybe I should cherry pick the commits that add me as a maintainer and merge them to master?

@Madouura we probably need to consider snapshots at some point as well, just been sleeping on it till my PR is merged. How did you get it working on your current system, did you endure the recompile times or something else?

Feel free to @ me on NixOS matrix as woobilicious I think most talk is happening in the System Programming area. My sleep schedule is chaotic so I'll try to get reply when I can.

@Madouura we probably need to consider snapshots at some point as well, just been sleeping on it till my PR is merged. How did you get it working on your current system, did you endure the recompile times or something else?

I'm not sure exactly what you're referencing, but I just have my nixos config flake set to this PR's branch and it works.

maybe I should cherry pick the commits that add me as a maintainer and merge them to master?

I think that would be the best option. linux_bcachefs_testing is now deprecated with this PR so I would suggest taking everything except changes to that and the util-linux changes and put it into master.
IIRC the util-linux changes are the blocking factor, and I imagine the reason why it's not in master yet is because of the cryptsetup testing issues.
Speaking of which, when is util-linux getting official bcachefs support? It's in linux-next now so I can't imagine it's too far from now...

@Madouura Oh I thought you had my PR running, the ordering of your comments had me thinking that.

There's a few misconceptions you seem to have:

1. cryptsetup breaking is a bug triggered by building and running the cryptsetup tests on a bcachefs file system, moving /tmp off bcachefs, or use another system to build it fixed the issue for me.

2. updating util-linux fixes the failing test, somewhere between 23.05 and 2.11 util-linux was updated, with broken bcachefs UUID support, nixos-generate-config output started generating /dev/disks/by-uuid paths, but a bug in util-linux only detects UUIDs with a freshly formatted FS (when running nixos-generate-config), once the superblock has grown (after installation) util-linux fails to read the UUID from the super block and setup /dev/disks/by-uuid. That is why the test fails, because the kernel can't find the device, and panics (panic on error).

3. The changes in util-linux and the changes in bcachefs.nix are both required for unlocking an encrypted multi-disk system using UUID= syntax, by-uuid symlinks don't work for multi device systems, the current tests fail to account for this scenario, and we have no boot.initrd.luks.devices.<name>.device equivalent for bcachefs file systems, the included dynamic script that works directly on the UUID using the patched util-linux, is also far more robust to reboot reordered device paths (something driving me nuts for ages) and adding/removing devices from the bcachefs file system.

4. The util-linux patches are official patches that are from the 2.39 branch, they plan on releasing a .3 release with a few other bug fixes.

5. This PR was pushed in to staging due to the fact that util-linux is a system wide dependency, and requires a huge subset of Nixpkgs to be rebuilt.

6. The reason why it works on your system is because you're using a single device, and ran nixos-generate-config with an old util-linux.

@Madouura Oh I thought you had my PR running, the ordering of your comments had me thinking that.

There's a few misconceptions you seem to have:

1. `cryptsetup` breaking is a bug triggered by building and running the cryptsetup tests on a bcachefs file system, moving /tmp off bcachefs, or use another system to build it fixed the issue for me.

2. updating `util-linux` **fixes the failing test**, somewhere between 23.05 and 2.11 util-linux was updated, with broken bcachefs UUID support, nixos-generate-config output  started generating  `/dev/disks/by-uuid` paths, but a bug in util-linux _only_ detects UUIDs with a freshly formatted FS (when running nixos-generate-config), once the superblock has grown (after installation) `util-linux` fails to read the UUID from the super block and setup `/dev/disks/by-uuid`. That is why the test fails, because the kernel can't find the device, and panics (panic on error).

3. The changes in `util-linux` and the changes in `bcachefs.nix` are both required for unlocking an encrypted multi-disk system using UUID= syntax, by-uuid symlinks don't work for multi device systems, the current tests fail to account for this scenario, and we have no `boot.initrd.luks.devices.<name>.device` equivalent for bcachefs file systems, the included dynamic script that works directly on the UUID using the patched util-linux, is also far more robust to reboot reordered device paths (something driving me nuts for ages) and adding/removing devices from the bcachefs file system.

4. The `util-linux` patches are official patches that are from the `2.39` branch, they plan on releasing a `.3` release with a few other bug fixes.

5. This PR was pushed in to staging due to the fact that `util-linux` is a system wide dependency, and requires a huge subset of Nixpkgs to be rebuilt.

6. The reason why it works on your system is because you're using a single device, and ran nixos-generate-config  with an old util-linux.

I appreciate the reply, that answered a few questions/misconceptions I had.

The reason why it works on your system is because you're using a single device, and ran nixos-generate-config with an old util-linux.

I'm not sure where you got this from. My setup is multi-device and encrypted.
My config also regenerates the (hardware) config every time I run make, so I don't think that's the case, since if it was, I would have been unable to boot several iterations ago.

To be clear, before this PR I actually was running your PR, as a test.

@Madouura Oh you must be using colon separated multi device system? I think I incorrectly assumed that wasn't possible on master for some reason. I moved away from that due to devices constantly swapping order and 5 by-id devices being too long for systemd to handle.

Were you using an old version? or perhaps only specific parts of my PR?

I'm now curious why it doesn't break, do you mind sharing?

@Madouura Oh you must be using colon separated multi device system? I think I incorrectly assumed that wasn't possible on master for some reason. I moved away from that due to devices constantly swapping order and 5 by-id devices being too long for systemd to handle.

Were you using an old version? or perhaps only specific parts of my PR?

I'm now curious why it doesn't break, do you mind sharing?

There's not much to say, I cherry-picked your PR's commits into a branch I used to have on my nixpkgs fork, disabled cryptsetup tests, did the correct overlays in my config, and it ran.

@thoughtpolice was that PR needed with my PR waiting in staging?

Perhaps it was not; I had not checked staging recently, which was my fault — I instead searched for open bcachefs patches but missed yours since it would have already been merged. I apologize for jumping the gun here! I should have left it to y'all.

I feel a little bit blind since I'm not being mentioned in some of the maintenance of the code base, especially since I did a lot of work on the PR, I was hoping my PR would get through quicker, maybe I should cherry pick the commits that add me as a maintainer and merge them to master?

Yes, my apologies. Please do add yourself as a maintainer so you can be CC'd on all this!

Staging finally got merged into master. 🎉
All tests pass. Running on my machine. 👍

All tests pass. Is this set of commits satisfactory?

/nix/store/xsrjql2kq7wiwr3l8k95q8nwxiqn5gn9-vm-test-run-bcachefs
/nix/store/26cwv970fxslanvwkj7jw2pg42k1cf78-vm-test-run-installer-bcachefs-simple
/nix/store/swglwmhv5l8yp8if33disv75zv2nh9h3-vm-test-run-installer-bcachefs-multi
/nix/store/59l2yyyyqyz2ypxdr2kzkigs203xrams-vm-test-run-installer-bcachefs-encrypted
/nix/store/57d15zga689bi56sndzjgid130c45ldy-vm-test-run-installer-bcachefs-linux-testing
/nix/store/3r5kfnp6mbqg0ylbjdxvpxswr30vp6pb-vm-test-run-installer-bcachefs-upgrade-to-linux-testing

The error in this image is likely fixed by koverstreet/bcachefs@19cf4df.