Skip to content

homedir: remove idtools and libcontainer's user package dependencies#39975

Merged
thaJeztah merged 1 commit intomoby:masterfrom
tiborvass:homedir-less-deps
Sep 26, 2019
Merged

homedir: remove idtools and libcontainer's user package dependencies#39975
thaJeztah merged 1 commit intomoby:masterfrom
tiborvass:homedir-less-deps

Conversation

@tiborvass
Copy link
Contributor

About github.com/opencontainers/runc/libcontainer/user:

According to opencontainers/runc@195d8d5
this package has two functions:

  • Have a static implementation of user lookup, which is now supported in the
    os/user stdlib package with the osusergo build tag, but wasn't at the time.
  • Have extra functions that os/user doesn't have, but none of those are used
    in homedir.

Since #11287, homedir depended directly on
libcontainer's user package for CurrentUser().
This is being replaced with os/user.Current(), because all of our static
binaries are compiled with the osusergo tag, and for dynamic libraries it
is more correct to use libc's implementation than parsing /etc/passwd.

About github.com/docker/docker/pkg/idtools:

Only dependency was from GetStatic() which uses idtools.LookupUID(uid).
The implementation of idtools.LookupUID just calls to
github.com/opencontainers/runc/libcontainer/user.LookupUid or fallbacks
to exec-ing to getent (since #27599).

This patch replaces calls to homedir.GetStatic by homedir.Get(), opting out
of supporting nss lookups in static binaries via exec-ing to getent for
the homedir package.

If homedir package users need to support nss lookups, they are advised
to compile dynamically instead.

Signed-off-by: Tibor Vass tibor@docker.com

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

update the comment above?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@dmcgowan
Copy link
Member

This looks reasonable to me. Are there any other options for statically compiling?

ping @jonjohnsonjr

Copy link
Contributor

@kolyshkin kolyshkin Sep 24, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no longer homedir.GetStatic(), this comment needs to be updated.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kolyshkin Should I move ensureHomeIfIAmStatic to gcplogging.go ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nevermind i'm just removing GetStatic from the comment.

Copy link
Contributor

@kolyshkin kolyshkin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM except for nitpick about a leftover comment

About github.com/opencontainers/runc/libcontainer/user:

According to opencontainers/runc@195d8d5
this package has two functions:
- Have a static implementation of user lookup, which is now supported in the
  os/user stdlib package with the osusergo build tag, but wasn't at the time.
- Have extra functions that os/user doesn't have, but none of those are used
  in homedir.

Since moby#11287, homedir depended directly on
libcontainer's user package for CurrentUser().
This is being replaced with os/user.Current(), because all of our static
binaries are compiled with the osusergo tag, and for dynamic libraries it
is more correct to use libc's implementation than parsing /etc/passwd.

About github.com/docker/docker/pkg/idtools:

Only dependency was from GetStatic() which uses idtools.LookupUID(uid).
The implementation of idtools.LookupUID just calls to
github.com/opencontainers/runc/libcontainer/user.LookupUid or fallbacks
to exec-ing to getent (since moby#27599).

This patch replaces calls to homedir.GetStatic by homedir.Get(), opting out
of supporting nss lookups in static binaries via exec-ing to getent for
the homedir package.

If homedir package users need to support nss lookups, they are advised
to compile dynamically instead.

Signed-off-by: Tibor Vass <tibor@docker.com>
@kolyshkin
Copy link
Contributor

Actually, this kludge of setting $HOME might no longer be needed since osusergo is set. Let me take a look....

@tiborvass
Copy link
Contributor Author

@kolyshkin i'm pretty sure it's not needed anymore, it would just prevent reading from /etc/passwd again. Let me know if you want me to remove it entirely, though I'd love @AkihiroSuda's input too.

@kolyshkin
Copy link
Contributor

OK, in case osusergo is set but $HOME is not, user.Current() returns an error which is probably not what we want, so this has to stay

@kolyshkin
Copy link
Contributor

Hmm. This is all chicken-and-egg, we use user.Current() to get homedir which gets it from os.Getenv("HOME"), and then do os.Setenv("HOME=xxx"). This no longer makes sense.

I think we can now remove the whole ensureHomeIfIAmStatic() kludge.

@kolyshkin
Copy link
Contributor

it would just prevent reading from /etc/passwd again

user.Current() does not do re-reading for quite some time, it only does it once and caches the result

@tiborvass
Copy link
Contributor Author

@kolyshkin

we use user.Current() to get homedir which gets it from os.Getenv("HOME")

That is incorrect. It looks up /etc/passwd first via lookupUserId in https://golang.org/src/os/user/lookup_stubs.go?h=func+current%28%29#L25 and only if that fails it uses os.UserHomeDir() which does a os.Getenv("HOME").

Copy link
Contributor

@kolyshkin kolyshkin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Removing ensureHomeIfIAmStatic() can be done separately

Copy link
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@thaJeztah thaJeztah merged commit 2c6d368 into moby:master Sep 26, 2019
@andrewhsu
Copy link
Contributor

i noticed flaky test DockerSwarmSuite.TestSwarmLockUnlockCluster failed on this PR check, but passed after merge to master

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants