Skip to content

core: improve image publish/export format compatibility.#5365

Merged
sipsma merged 1 commit intodagger:mainfrom
sipsma:better-export-types
Jun 24, 2023
Merged

core: improve image publish/export format compatibility.#5365
sipsma merged 1 commit intodagger:mainfrom
sipsma:better-export-types

Conversation

@sipsma
Copy link
Copy Markdown
Contributor

@sipsma sipsma commented Jun 24, 2023

I went down a bit of a rabbit hole after looking into this problem related to running dagger tests w/ magicache.

That rabbit hole is an extension of previous rabbit holes when similar issues were hit w/ Airbyte and when trying to do an upgrade the other week.

The important takeaways from the most recent investigation:

  1. vnd.docker.image.rootfs.diff.tar.zstd has never been a thing ever outside of buildkit, afaict it's the only place that string even exists? I now think the fact that buildkit ever exports using those might actually just be a bug rather than a feature. The only zstd media type that really exists in the ecosystem is the oci one.
  2. Docker has supported pulling images that use OCI media types from registries since 2018, so given that plus the fact that every registry has supported them for at least as long, I think it's safe to always publish images using them.
  3. Docker doesn't seem to care what the media type of layers are when docker loading them; it just checks the archive headers directly to figure out whether it's a support compression type.

So given that the conclusions were:

  1. When publishing an image, just always use oci mediatypes
  2. When exporting an image to a tar, always use oci mediatypes, but if the image is single-platform use the docker exporter type (which has both oci and docker compatible manifests)

I updated that logic and also added an integ test that verifies the expected docker load compatibility behavior going back to 20.x releases (not sure we need to care about older). This compat is important as it's explicitly relied on in our docs.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
@sipsma sipsma requested review from jlongtine and vito June 24, 2023 02:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants