Skip to content

lib/generate.sh: Fix EROFS file UIDs and use compression#176

Merged
pothos merged 3 commits intomainfrom
erofs-fix
Aug 7, 2025
Merged

lib/generate.sh: Fix EROFS file UIDs and use compression#176
pothos merged 3 commits intomainfrom
erofs-fix

Conversation

@pothos
Copy link
Copy Markdown
Member

@pothos pothos commented Aug 4, 2025

The EROFS mkfs invocation missed the --all-root flag to make sure that the files are owned by root regardless of the current UID during image generation. By default EROFS also doesn't compress and we can use LZ4 as a lightweight compresson option (LZMA would also work). Add the --all-root flag and the compression flag to mkfs.erofs to use UID 0 and compress the contents.

How to use

./bakery.sh create --format erofs wasmtime v35.0.0

And then copy via ssh to a Flatcar VM, move the file under /etc/extensions/ and reload the sysexts.

Testing done

The wasmtime binary is now owned by root and the resulting image is half the size thanks to compression.

@pothos pothos requested a review from a team as a code owner August 4, 2025 08:36
@pothos
Copy link
Copy Markdown
Member Author

pothos commented Aug 4, 2025

Also added a patch to enable btrfs compression and add a note that something like --all-root does not exist for mkfs.btrfs and the resulting files may have unwanted UIDs set.

lib/generate.sh Outdated
;;
erofs)
mkfs.erofs "${fname}" "${basedir}"
mkfs.erofs -zlz4 --all-root "${fname}" "${basedir}"
Copy link
Copy Markdown

@hsiangkao hsiangkao Aug 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest using -zlz4hc,12 instead of -zlz4 unless the build time is sensitive, also
it's preferable to use -C65536 -Efragments to match squashfs default configuration at least.

@pothos
Copy link
Copy Markdown
Member Author

pothos commented Aug 4, 2025

I suggest using -zlz4hc,12 instead of lz4 unless the build time is sensitive.

Thank you for the comment! Now I've checked the https://github.com/erofs/erofs-utils README and I see there, too. I also thought about LZMA because a 5.16 kernel could be a low enough requirement. What would be better in that case, lz4hc or LZMA? Also considering long term plans?

@hsiangkao
Copy link
Copy Markdown

hsiangkao commented Aug 4, 2025

I suggest using -zlz4hc,12 instead of lz4 unless the build time is sensitive.

Thank you for the comment! Now I've checked the https://github.com/erofs/erofs-utils README and I see there, too

I may need to refine the README in the later version and prefer the website
because README seems always outdated, the latest mkfs guide is at https://erofs.docs.kernel.org/en/latest/mkfs.html

I also thought about LZMA because a 5.16 kernel could be a low enough requirement. What would be better in that case, lz4hc or LZMA? Also considering long term plans?

It depends on the specific use cases, for Android use cases, almost all users use lz4hc and 4k pclusters for the best random runtime performance.
For livecd use cases, they use lzma and even 1m pclusters (+ fragments) for the smallest images.

@pothos pothos force-pushed the erofs-fix branch 2 times, most recently from 2d20a65 to 5e0ee8b Compare August 4, 2025 10:45
pothos added 3 commits August 4, 2025 18:04
The EROFS mkfs invocation missed the --all-root flag to make sure that
the files are owned by root regardless of the current UID during image
generation. By default EROFS also doesn't compress and we can use LZ4
as a lightweight compresson option (LZMA would also work). Another
problem is that the random filesystem UUID breaks reproducibility.
Add the --all-root flag and the compression flag to mkfs.erofs to use
UID 0, compress the contents following upstream recommendations (e.g.
same compression blocksize as squashfs), and set a zero UUID (maybe we
could also calculate one based on hashing the name and the version).

Signed-off-by: Kai Lueke <kailuke@microsoft.com>
Since some time mkfs.btrfs supports compression. For cases where it is
used instead of squashfs or erofs it makes sense to enable it for size
savings. Also add a note that the resulting files won't be owned as
root if the image is created by another user.

Signed-off-by: Kai Lueke <kailuke@microsoft.com>
The service failed to start because the binary lacked execution
permissions.

Signed-off-by: Kai Lueke <kailuke@microsoft.com>
@pothos
Copy link
Copy Markdown
Member Author

pothos commented Aug 4, 2025

Thanks, updated the mkfs.erofs options with the recommendations from the guide.

I've also noticed that by default it's not reproducible, and I had to add -Uclear to clear the filesystem UUID (maybe we could also calculate a hash based on the name and the version).

Edit: We already set SOURCE_DATE_EPOCH

@hsiangkao
Copy link
Copy Markdown

Thanks, updated the mkfs.erofs options with the recommendations from the guide.

I've also noticed that by default it's not reproducible, and I had to add -Uclear to clear the filesystem UUID (maybe we could also calculate a hash based on the name and the version).

-Uclear -T0 --mkfs-time will make images reproducible. @pothos

@pothos
Copy link
Copy Markdown
Member Author

pothos commented Aug 5, 2025

By the way, while testing the compression flags in mkfs.erofs I had the idea that all mkfs tools (other filesystems have the same problems in this regard) could print what minimal kernel version (or kernel config option) the resulting image is compatible with. Maybe it's complicated to implement, but it would help against accidentally raising the kernel requirements when tweaking the options.

@hsiangkao
Copy link
Copy Markdown

By the way, while testing the compression flags in mkfs.erofs I had the idea that all mkfs tools (other filesystems have the same problems in this regard) could print what minimal kernel version (or kernel config option) the resulting image is compatible with. Maybe it's complicated to implement, but it would help against accidentally raising the kernel requirements when tweaking the options.

There is no such unique interface, but I could add the minimal kernel version to the output of dump.erofs super information.

@pothos
Copy link
Copy Markdown
Member Author

pothos commented Aug 6, 2025

@flatcar/flatcar-integrations Can you review the changes?

@danzatt
Copy link
Copy Markdown
Contributor

danzatt commented Aug 7, 2025

Looks good. The only slight worry I had was if systemd won't get confused if there are multiple sysexts with the same (zero) UUID. I've tested this on latest alpha and it works as expected, so I guess that's not an issue.

EDIT: I've just realized squashfs does not support UUID at all, so this should be absolutely fine.

@pothos
Copy link
Copy Markdown
Member Author

pothos commented Aug 7, 2025

Thanks, yes, I should have written that I've also tested this and that squashfs doesn't have one either and a zero UUID won't show up in /dev/disk/by-uuid/.

@pothos pothos merged commit ca2f7b6 into main Aug 7, 2025
1 check passed
@pothos pothos deleted the erofs-fix branch August 7, 2025 14:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants