Skip to content

c8d: Implement Children by comparing diff ids#45265

Merged
thaJeztah merged 2 commits intomoby:masterfrom
vvoland:c8d-children-upstream
Apr 11, 2023
Merged

c8d: Implement Children by comparing diff ids#45265
thaJeztah merged 2 commits intomoby:masterfrom
vvoland:c8d-children-upstream

Conversation

@vvoland
Copy link
Contributor

@vvoland vvoland commented Apr 4, 2023

Implement Children method for containerd image store which makes the ancestor filter work for docker ps.

Checking if image is a children of other image is implemented by comparing their rootfs diffids because containerd image store doesn't have a concept of image parentship like the graphdriver store. The child is expected to have more layers than the parent and should start with all parent layers.

- How to verify it

$ docker run -it alpine
/ # apk add fish
...
OK: 22 MiB in 24 packages

$ docker commit 8b1ea669e23e alpine-plus-fish
sha256:776b08f8a0cb27796cced75fcbe783bce0f2cbb0a91c38faadea44b519509c29

$ docker create alpine-plus-fish
c8e1caffdbbabdd3d67dbb9768d67a66ef98d26004eb4e10d7a880dd5da0870a

$ docker create busybox
7276114c22057a45964c93fcada3e23d2856e9021f1fdcc4959d410b8dce8eab

$ docker ps -a
CONTAINER ID   IMAGE              COMMAND     CREATED              STATUS                     PORTS     NAMES
7276114c2205   busybox            "sh"        About a minute ago   Created                              angry_kare
c8e1caffdbba   alpine-plus-fish   "/bin/sh"   2 minutes ago        Created                              brave_mcnulty
8b1ea669e23e   alpine             "/bin/sh"   2 minutes ago        Exited (0) 2 minutes ago             nifty_kilby

$ docker ps -a --filter ancestor=busybox
CONTAINER ID   IMAGE     COMMAND   CREATED         STATUS    PORTS     NAMES
7276114c2205   busybox   "sh"      2 minutes ago   Created             angry_kare
$ docker ps -a --filter ancestor=alpine-plus-fish
CONTAINER ID   IMAGE              COMMAND     CREATED         STATUS    PORTS     NAMES
c8e1caffdbba   alpine-plus-fish   "/bin/sh"   2 minutes ago   Created             brave_mcnulty
$ docker ps -a --filter ancestor=alpine
CONTAINER ID   IMAGE              COMMAND     CREATED         STATUS                     PORTS     NAMES
c8e1caffdbba   alpine-plus-fish   "/bin/sh"   2 minutes ago   Created                              brave_mcnulty
8b1ea669e23e   alpine             "/bin/sh"   3 minutes ago   Exited (0) 2 minutes ago             nifty_kilby

- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)

@vvoland vvoland added status/2-code-review area/images Image Distribution containerd-integration Issues and PRs related to containerd integration labels Apr 4, 2023
@thaJeztah thaJeztah added this to the 24.0.0 milestone Apr 5, 2023
// rootfs of the given image ID, excluding images with exactly the same rootfs.
// Called from list.go to filter containers.
func (i *ImageService) Children(id image.ID) []image.ID {
ctx := context.TODO()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add the context as a parameter?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good to be able to cancel this operation because it can be heavy if there are a lot of images.
However, the interface function is used in other places in the graphdriver image store implementation which would also require context passing, which are not related to this particular implementation.
So I'd leave it for a follow up (handling context in bunch of other graphdriver image store code or refactoring the interface to make it possible to pass context only to this one).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm actually it might be simpler. Let me try.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know for (some of) the graph driver implementations we discussed that we are ok with only adding the context, but "ignoring it" / "not using it" if we can't (so only changing the interface, and add a _ context.Context where applicable)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated - I initially misread Children method from image.Store as the one from ImageService 😄

moby/image/store.go

Lines 18 to 27 in 58c027a

type Store interface {
Create(config []byte) (ID, error)
Get(id ID) (*Image, error)
Delete(id ID) ([]layer.Metadata, error)
Search(partialID string) (ID, error)
SetParent(id ID, parent ID) error
GetParent(id ID) (ID, error)
SetLastUpdated(id ID) error
GetLastUpdated(id ID) (time.Time, error)
Children(id ID) []ID

return false
}

for i := 0; i < len(parent.DiffIDs); i++ {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
for i := 0; i < len(parent.DiffIDs); i++ {
for i := 0; i < parentLen; i++ {

vvoland added 2 commits April 6, 2023 14:27
Implement Children method for containerd image store which makes the
`ancestor` filter work for `docker ps`. Checking if image is a children
of other image is implemented by comparing their rootfs diffids because
containerd image store doesn't have a concept of image parentship like
the graphdriver store. The child is expected to have more layers than
the parent and should start with all parent layers.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
This only makes the containerd ImageService implementation respect
context cancellation though.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
@vvoland vvoland force-pushed the c8d-children-upstream branch from 1321504 to e0f36f9 Compare April 6, 2023 12:28
Copy link
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

func platformRootfs(ctx context.Context, store content.Store, desc ocispec.Descriptor, platform ocispec.Platform) (ocispec.RootFS, error) {
empty := ocispec.RootFS{}

log := logrus.WithField("desc", desc.Digest).WithField("platform", platforms.Format(platform))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably okay for a follow-up, but I recall Cory had some changes elsewhere for things like this (as multiple WithField() could cause quite some work to be done, even if the logger isn't called?) as this is called for every image, we may want to look if it should be optimised.

At least could use WithFields;

log := logrus.WithFields(logrus.Fields{"desc": desc.Digest, "platform": platforms.Format(platform)})`

But maybe we need to consider duplicating the code, and only constructing this when we need it.

return []image.ID{}
}

is := i.client.ImageService()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit; is looks to be only used at line 46, so perhaps we don't need the variable (or if we want to have a variable, move it closer to where it's used)

Comment on lines +21 to +22
logrus.WithError(err).Error("failed to get parent image")
return []image.ID{}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Starting to wonder if we should just (wrap, and) return these errors, and let the caller either log, or handle them.

(can be a follow-up)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/images Image Distribution containerd-integration Issues and PRs related to containerd integration status/2-code-review

Projects

Development

Successfully merging this pull request may close these issues.

3 participants