fix(mutate): preserve config blob and layers for non-Docker OCI artifacts#2286
Conversation
…acts compute() unconditionally parses the config as a Docker image config and derives layers from RootFS.DiffIDs. For non-Docker OCI artifacts (Helm charts, WASM modules), the config has no DiffIDs, causing Layers() to return empty (remote.Write skips blob uploads) and RawConfigFile() to corrupt the config digest via re-marshaling. Detect non-image configs by checking the config media type. When it is not DockerConfigJSON or OCIConfigJSON, delegate to computeArtifact which preserves the original config blob and derives layers from the base image and manifest descriptors. Fixes google#2251 Signed-off-by: Dayna Blackwell <dayna@blackwell-systems.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2286 +/- ##
==========================================
+ Coverage 56.83% 56.89% +0.05%
==========================================
Files 165 165
Lines 11319 11334 +15
==========================================
+ Hits 6433 6448 +15
Misses 4121 4121
Partials 765 765 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Confirmed with the image in the original bug that this prevents corrupting the image on upload and download. I can approve this change after comments and linter errors are addressed. |
Subserial
left a comment
There was a problem hiding this comment.
Sorry for noting this just now, but this code is very poor quality and duplicates several code paths that already exist. I've left comments, but from now on please review your generated changes to reduce this introduction of code debt.
| // For non-image OCI artifacts, RootFS.DiffIDs is empty so partial.DiffIDs | ||
| // returns nothing. Fall back to the base image's layers plus any added layers. | ||
| if i.manifest != nil && !isImageConfig(i.manifest.Config.MediaType) { | ||
| layers, err := i.base.Layers() | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| for _, add := range i.adds { | ||
| if add.Layer != nil { | ||
| layers = append(layers, add.Layer) | ||
| } | ||
| } | ||
| return layers, nil | ||
| } | ||
|
|
There was a problem hiding this comment.
Same comment here. This logic already exists and is duplicated. Please reduce the footprint of this change.
There was a problem hiding this comment.
To fulfill this, you can remove this block and change the above if statement to:
if err := i.compute(); errors.Is(err, stream.ErrNotComputed) || (i.manifest != nil && !isImageConfig(i.manifest.Config.MediaType) {
Per review feedback: - Inline artifact-specific behavior as conditional blocks within compute() instead of a separate computeArtifact() function - Share manifest layer loop, annotations, mediaType, and subject assignment between both paths - Trim isImageConfig comment - Layers() artifact fallback stays inline (no new abstractions) Net -70 lines.
|
Thanks for the detailed feedback. I've refactored to inline the artifact logic into |
Subserial
left a comment
There was a problem hiding this comment.
Some of the comments seem to refer to images vs artifacts, which should be updated to images vs non-image artifacts.
Also, please do not remove arbitrary whitespace and comments from untouched code.
| // For non-image OCI artifacts, RootFS.DiffIDs is empty so partial.DiffIDs | ||
| // returns nothing. Fall back to the base image's layers plus any added layers. | ||
| if i.manifest != nil && !isImageConfig(i.manifest.Config.MediaType) { | ||
| layers, err := i.base.Layers() | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| for _, add := range i.adds { | ||
| if add.Layer != nil { | ||
| layers = append(layers, add.Layer) | ||
| } | ||
| } | ||
| return layers, nil | ||
| } | ||
|
|
There was a problem hiding this comment.
To fulfill this, you can remove this block and change the above if statement to:
if err := i.compute(); errors.Is(err, stream.ErrNotComputed) || (i.manifest != nil && !isImageConfig(i.manifest.Config.MediaType) {
Address review feedback: - Remove comment at line 124 - Merge the !isImageConfig block into the ErrNotComputed condition
|
Thanks @Subserial and @blackwell-systems for the fix! |
…acts (google#2286) * fix(mutate): preserve config blob and layers for non-Docker OCI artifacts compute() unconditionally parses the config as a Docker image config and derives layers from RootFS.DiffIDs. For non-Docker OCI artifacts (Helm charts, WASM modules), the config has no DiffIDs, causing Layers() to return empty (remote.Write skips blob uploads) and RawConfigFile() to corrupt the config digest via re-marshaling. Detect non-image configs by checking the config media type. When it is not DockerConfigJSON or OCIConfigJSON, delegate to computeArtifact which preserves the original config blob and derives layers from the base image and manifest descriptors. Fixes google#2251 Signed-off-by: Dayna Blackwell <dayna@blackwell-systems.com> * test: replace no-op _ = layers with assertion per review * fix: rename unused originalManifest parameter to _ (revive lint) * fix: remove unused originalManifest parameter from computeArtifact per review * refactor: inline artifact logic into compute(), remove computeArtifact Per review feedback: - Inline artifact-specific behavior as conditional blocks within compute() instead of a separate computeArtifact() function - Share manifest layer loop, annotations, mediaType, and subject assignment between both paths - Trim isImageConfig comment - Layers() artifact fallback stays inline (no new abstractions) Net -70 lines. * Combine Layers() artifact check into compute error branch Address review feedback: - Remove comment at line 124 - Merge the !isImageConfig block into the ErrNotComputed condition --------- Signed-off-by: Dayna Blackwell <dayna@blackwell-systems.com>
Summary
compute()inimage.gounconditionally parses the config as a Docker image config and derives layers fromRootFS.DiffIDs. For non-Docker OCI artifacts (Helm charts, WASM modules, arbitrary blobs), the config has noRootFS.DiffIDs, causing:Layers()returns empty.remote.WriteusesLayers()to determine which blobs to upload. On a cross-repository push, layer blobs are never uploaded, and the registry rejects the manifest withMANIFEST_BLOB_UNKNOWN.The config blob digest is corrupted.
RawConfigFile()re-marshals the config through the DockerConfigFilestruct, producing a different digest than the original. The manifest references a blob that was never uploaded.Repro:
crane copy registry-1.docker.io/bitnamicharts/nginx:13.2.14 <dest>(Helm chart withapplication/vnd.cncf.helm.config.v1+jsonconfig).Fix
Detect non-image configs by checking the config media type against the two known image config types (
DockerConfigJSON,OCIConfigJSON). When the config is neither:computeArtifact(): new method that preserves the original config blob (no re-marshal, no digest corruption) and builds the layer list from manifest descriptors and added layers.Layers(): falls back to base image layers + added layers instead of deriving from emptyDiffIDs.RawConfigFile(): returns the original raw bytes from the base image instead of re-marshaling.Standard Docker/OCI image workflows are unchanged (all existing tests pass).
Testing
Added
TestNonImageArtifactPreservesConfigwhich creates an artifact withapplication/vnd.cncf.helm.config.v1+jsonconfig type, mutates annotations, and verifies:Layers()does not errorFull
./pkg/...test suite passes across all packages.Fixes #2251