Skip to content

c8d: Fix re-pull of an image when the snapshotter is changed#45647

Merged
neersighted merged 1 commit intomoby:masterfrom
rumpl:fix-snapshotter-change
May 30, 2023
Merged

c8d: Fix re-pull of an image when the snapshotter is changed#45647
neersighted merged 1 commit intomoby:masterfrom
rumpl:fix-snapshotter-change

Conversation

@rumpl
Copy link
Copy Markdown
Member

@rumpl rumpl commented May 30, 2023

- What I did

Changing the snapshotter would result in docker re-pulling the image because the PrepareSnapshot method would return an error. We make sure that the image we want to prepare a snapshot for is indeed unpacked for the current snapshotter before preparing it.

Relates to #45273

- How I did it

- How to verify it

  • start dockerd with the overlayfs snapshotter
  • run an image docker run --rm hello-world
  • restart dockerd and change the snapshotter (use native for example)
  • run docker images, you should see the hello-world image in the list
  • run the image: docker run --rm hello-world, docker should just run the image without re-pulling it

- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)

@rumpl rumpl added area/images Image Distribution containerd-integration Issues and PRs related to containerd integration labels May 30, 2023
@rumpl rumpl changed the title Unpack by hand when pulling an image c8d: Unpack by hand when pulling an image May 30, 2023
Copy link
Copy Markdown
Member

@laurazard laurazard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@rumpl rumpl force-pushed the fix-snapshotter-change branch from f793648 to 19e2c9e Compare May 30, 2023 10:21
@rumpl rumpl changed the title c8d: Unpack by hand when pulling an image c8d: Fix re-pull of an image when the snapshotter is changed May 30, 2023
Copy link
Copy Markdown
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but left two questions 😅

}
}

desc, err = containerdimages.Config(ctx, cs, desc, matcher)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Silly question; does this step need the unpacked image to do its job? (Mostly wondering if the unpack can be done later)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Config doesn't need the image to be unpacked, I can move the code but the descriptor returned by Config is immediately used to get the RootFS so I thought it might be nicer to bundle related things closer

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha; my take was "try to do lightweight checks first, before we do heavy-lifting".

I must admit that I haven't checked in-depth on that, but perhaps you know from the top of your head.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Config is in fact a heavy-lifter :)

I did however removed the call to resolveDescriptor and are calling resolveImage directly, that way we don't have to get the image twice

return err
}
platformImg := containerd.NewImageWithPlatform(i.client, img, matcher)
unpacked, err := platformImg.IsUnpacked(ctx, i.snapshotter)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At a quick glance, PrepareShapshot() is only called on container create, correct? So in this case we must always (at least currently) use the snapshotter that's used as a default, and never take existing containers (which were potentially created with "previous default") into account?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this is only used when a container is created. The containers from other snapshotters are filtered out and not seen IIRC

@thaJeztah
Copy link
Copy Markdown
Member

Failure is unrelated, and tracked in #44128 (still quite flaky that one)

=== FAIL: amd64.integration-cli TestDockerCLIRunSuite/TestRunAttachDetachFromInvalidFlag (0.41s)
    docker_cli_run_unix_test.go:232: read |0: file already closed
    --- FAIL: TestDockerCLIRunSuite/TestRunAttachDetachFromInvalidFlag (0.41s)

@rumpl rumpl added this to the 25.0.0 milestone May 30, 2023
Switching snapshotter implementations would result in an error when
preparing a snapshot, check that the image is indeed unpacked for the
current snapshot before trying to prepare a snapshot.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
@rumpl rumpl force-pushed the fix-snapshotter-change branch from 19e2c9e to ed32f5e Compare May 30, 2023 12:45
Copy link
Copy Markdown
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/images Image Distribution containerd-integration Issues and PRs related to containerd integration kind/bugfix PR's that fix bugs process/cherry-picked

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants