Skip to content

Changes for merged-usr profile#34

Merged
krnowak merged 10 commits intoflatcar-masterfrom
krnowak/merged-usr
Sep 5, 2024
Merged

Changes for merged-usr profile#34
krnowak merged 10 commits intoflatcar-masterfrom
krnowak/merged-usr

Conversation

@krnowak
Copy link
Copy Markdown
Member

@krnowak krnowak commented Aug 27, 2024

Please read the commit messages for detailed information.

The high level goals of this PR and it's parent PR in scripts (flatcar/scripts#2068) is to convert SDK from split-usr to merged-usr layout and finish that process for generic images (merged-bin).

High level changes:

  • categorize directories to share, bin, lib and etc
  • split etc baselayout tmpfiles (so ebuild can remove unnecessary config files instead of editing them)
  • add layout installation target to Makefile (used by ebuild in pkg_preinst)
    • it installs libdirs, bindirs and symlinks, also for debug directories
  • add generation of libdirs baselayout
  • add temporary, dump and simple tmpfiles processor (replacement for systemd-tmpfiles)

CI: http://jenkins.infra.kinvolk.io:8080/job/container/job/sdk/1697/cldsv/ (it passed once, but I made some more changes, so rerunning it to make sure nothing broke)

It creates library directories (through passed LIB_DIRS variable from
outside), the binary directories and necessary symlinks (sbin to bin,
or lib to /usr/lib and the likes).

The rest of the basic filesystem directories (/dev, /proc and so on)
will be created by tmpfiles.
"sbin" is about to become a symlink to its "bin" counterpart, so
having "sbin" in PATH is pointless since its "bin" counterparts are
already there.
This is so that the dumb tmpfiles processor will know how to create a
parent directory of `/home/core`.
@krnowak krnowak force-pushed the krnowak/merged-usr branch from 8ec52c7 to 65af4cd Compare August 29, 2024 09:56
It's a temporary solution, which will be dropped once we migrate to
Gentoo's baselayout. It is quite a dubm and simple processor, because
it is very limited in its functionality compared to systemd's tmpfiles
processor, and it requires the user and group to be passed up front
through the environment variables. This is because the script is
tailored to be used only in the ebuild, when installing baselayout in
src_install and pkg_preinst phases.

In src_install the script will be used to install the files and
symlinks, together with their parent directories, thus excluding the
empty directories. These will get created in the pkg_preinst phase.

The point of adding the script was to stop using systemd's tmpfiles
processor, which requires user and group information to exist in
/etc/passwd or /etc/group. Obviously it would be even better to not
abuse the tmpfiles for the package installation purposes, but I think
that this will come with the switch to Gentoo's baselayout.
Sort the files and directories that the Makefile installs into
categories, so that everything that gets installed to `/etc` is inside
`etc` directory. Same for `/usr/bin` and `bin`, `/usr/lib` and `lib`,
and `/usr/share` and `share`.

The files in `scripts` (one script and one systemd unit) were moved to
`bin` and to `share`. For SDK builds these will be removed before
installation, which is why the `bin` part is made optional in
Makefile.
These seem to be unwanted by either SDK (flatcar-profile.sh, vimrc) or
generic image (issue). Avoiding installing them can be done in the
ebuild by just removing the file in src_prepare phase now, instead of
fiddling with sed to edit the tmpfiles.
@krnowak krnowak force-pushed the krnowak/merged-usr branch from 65af4cd to d95d5ca Compare August 29, 2024 10:03
@krnowak krnowak marked this pull request as ready for review August 29, 2024 13:02
@krnowak krnowak requested a review from a team August 29, 2024 13:02
@chewi
Copy link
Copy Markdown

chewi commented Aug 29, 2024

I don't understand why you've added this "dumb" tmpfiles implementation. The ebuild used to call systemd-tmpfiles itself, and this was fine? The dependencies were wrong though, sys-apps/systemd should have been under BDEPEND, not DEPEND, and I found the latter caused cyclic issues. I had a fix for this lined up in my arm64 SDK branch. With your changes, it would probably need to go in IDEPEND too. I know that baselayout is initially installed with --nodeps by Catalyst, but this is into a ROOT, so you'd still have systemd-tmpfiles available.

@krnowak
Copy link
Copy Markdown
Member Author

krnowak commented Aug 30, 2024

Thanks for the review, will fix the issues and get back to you. I just added Gentoo baselayout to SDK too just to see what the structure it actually installs. Should have done that in the beginning.

I don't understand why you've added this "dumb" tmpfiles implementation. The ebuild used to call systemd-tmpfiles itself, and this was fine?

It wasn't entirely fine - some files were either installed with wrong user or not installed at all. I think it was mostly the former, so stuff ended up being installed as root. So the script requires passing this infos through env vars instead. This also required some special casing for installing home directory for core user (was done differently for SDK and differently for generic images). I also don't like using tmpfiles as means for doing installation of files and directories in src_install and pkg_preinst (but I'm still doing it though ;) ). In general, it's only temporary, I want to get rid of it once we move to Gentoo baselayout and move the tmpfiles and other stuff to some extra package.

The dependencies were wrong though, sys-apps/systemd should have been under BDEPEND, not DEPEND, and I found the latter caused cyclic issues. I had a fix for this lined up in my arm64 SDK branch. With your changes, it would probably need to go in IDEPEND too. I know that baselayout is initially installed with --nodeps by Catalyst, but this is into a ROOT, so you'd still have systemd-tmpfiles available.

I don't think sys-apps/systemd is really needed at all, so I dunno if it should be a dependency of baselayout with my changes. But I didn't tinker with dependencies at all, left the as they were thinking that this will need a cleanup anyway later.

@krnowak krnowak force-pushed the krnowak/merged-usr branch from f92cc16 to 8e203be Compare August 30, 2024 12:30
@krnowak krnowak requested a review from chewi August 30, 2024 13:34
The Makefile is already aware of LIBDIRS so use it to generate the
tmpfiles config file for the /lib to usr/lib symlinks.
Those are always going to look like this, so no point in having those
generated in the ebuild or Makefile.
`/usr/local/sbin` is not a symlink, it should be a separate
directory. Also `/sbin` should be a symlink directly to
`usr/bin`. Similar fixes were needed for the debug directories and
symlinks.
@krnowak krnowak merged commit 3fe2f92 into flatcar-master Sep 5, 2024
@krnowak krnowak deleted the krnowak/merged-usr branch September 5, 2024 12:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants