OCPNODE-3945: Adapt ociartifact to new interfaces of libartifact#9691
Conversation
📝 WalkthroughWalkthroughRefactors OCI artifact handling by introducing a new datastore package, updating store interfaces and usage of libartifact, and migrating seccomp integration to the datastore types. Adds a new PullData flow with blob verification and size limits. Updates mocks and tests accordingly. Separately bumps dependencies, notably switching CRIU imports from v7 to v8. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
actor Caller as Caller
participant DS as Datastore.Store
participant OA as OCIArtifact.Store
participant CI as containers/image
participant Reg as Registry
Caller->>DS: PullData(ref, PullOptions)
DS->>DS: sanitizeOptions()
DS->>OA: Pull(ref, CopyOptions)
OA->>Reg: Fetch manifest/layers
OA-->>DS: manifest digest
DS->>CI: Parse & create ImageReference
DS->>CI: NewImageSource(ref, SystemContext)
activate CI
loop for each layer
DS->>CI: GetBlob(layer)
CI-->>DS: Blob stream, size
DS->>DS: Enforce size limit
DS->>DS: ReadAll + verify digest
end
deactivate CI
DS-->>Caller: []ArtifactData or error
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
🧪 Generate unit tests (beta)
Important Action Needed: IP Allowlist UpdateIf your organization protects your Git platform with IP whitelisting, please add the new CodeRabbit IP address to your allowlist:
Failure to add the new IP will result in interrupted reviews. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
9ef5781 to
2d43e62
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #9691 +/- ##
==========================================
- Coverage 67.47% 67.45% -0.02%
==========================================
Files 210 210
Lines 29043 29046 +3
==========================================
- Hits 19597 19594 -3
- Misses 7763 7771 +8
+ Partials 1683 1681 -2 🚀 New features to boost your workflow:
|
|
I'll resume once podman-container-tools/container-libs#408 is fixed. |
|
@bitoku: This pull request references OCPNODE-3945 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/jira refresh |
|
@bitoku: This pull request references OCPNODE-3945 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/retest @cri-o/cri-o-maintainers PTAL The change became big. may be better to look per commit |
|
@cri-o/cri-o-maintainers PTAL |
Signed-off-by: Ayato Tokubi <atokubi@redhat.com> # Conflicts: # go.mod # go.sum # vendor/github.com/klauspost/compress/flate/huffman_bit_writer.go # vendor/modules.txt
…ct` and remove redundant `buildArtifact` logic. Signed-off-by: Ayato Tokubi <atokubi@redhat.com>
… handling Signed-off-by: Ayato Tokubi <atokubi@redhat.com>
Signed-off-by: Ayato Tokubi <atokubi@redhat.com>
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Fix all issues with AI agents
In `@internal/ociartifact/datastore/store.go`:
- Around line 171-174: The size check currently allows one extra byte because it
compares against int64(maxArtifactSize)+1; update the conditional in the code
that uses variables size and maxArtifactSize (the block that returns
fmt.Errorf("exceeded maximum allowed size...")) to compare directly against
int64(maxArtifactSize) (i.e., use size > int64(maxArtifactSize)) so blobs larger
than maxArtifactSize are rejected as intended; keep the special-case check for
size == -1 unchanged.
In `@internal/ociartifact/libartifact_store.go`:
- Around line 11-15: The mock implementation contains an orphaned
SystemContext() method that is not part of the LibartifactStore interface;
remove the SystemContext() method from the ociartifact mock so the mock only
implements the interface methods Remove, List, Pull, and Inspect, ensuring the
mock signature matches LibartifactStore exactly.
In `@internal/ociartifact/store.go`:
- Around line 54-60: The code calls ref.DockerReference().String() without
checking for a nil DockerReference, which can panic for non-Docker references;
update the function to call ref.DockerReference() into a local variable (e.g.,
dr := ref.DockerReference()), check if dr == nil and handle that case (return a
clear error or fall back to ref.String() if appropriate), then pass dr.String()
(or the fallback) into libart.NewArtifactReference; reference the symbols
DockerReference(), String(), libart.NewArtifactReference, and the strRef
variable when making the change.
🧹 Nitpick comments (6)
internal/ociartifact/datastore/impl.go (1)
16-17: Make the interface comment state the intent (DI/mocking)Proposed comment update
-// Impl is the interface for the implementation. +// Impl exists to allow dependency injection/mocking of image operations in tests.As per coding guidelines: Add comments explaining 'why' not 'what' in Go code.
internal/ociartifact/datastore/suite_test.go (1)
12-16: Make the TestRun comment explain why it existsProposed comment update
-// TestRun runs the created specs. +// TestRun wires Ginkgo into go test so the datastore suite runs under `go test`.As per coding guidelines: Add comments explaining 'why' not 'what' in Go code.
internal/ociartifact/datastore/store_test.go (1)
44-83: Consider adding happy-path tests forPullData.The current tests cover error scenarios well (ParseNormalizedNamed and DockerNewReference failures). Consider adding tests for the successful pull path and other failure scenarios (e.g., artifact size exceeding
MaxSize, empty artifact data) to improve coverage.internal/ociartifact/artifact.go (1)
32-51: Consider acceptingcontext.Contextfor proper log propagation.
NewArtifactusescontext.Background()for logging (line 42), which loses any tracing/logging context the caller might have. Per the coding guidelines,context.Contextshould be propagated through function calls.♻️ Suggested change to accept context
-func NewArtifact(art *libartifact.Artifact) *Artifact { +func NewArtifact(ctx context.Context, art *libartifact.Artifact) *Artifact { artifact := &Artifact{ Artifact: art, namedRef: unknownRef{}, } if art.Name != "" { namedRef, err := reference.ParseNormalizedNamed(art.Name) if err != nil { - log.Warnf(context.Background(), "Failed to parse artifact name %s with the error %s", art.Name, err) + log.Warnf(ctx, "Failed to parse artifact name %q: %v", art.Name, err) namedRef = unknownRef{} } artifact.namedRef = namedRef } return artifact }As per coding guidelines: "Propagate context.Context through function calls in Go code".
internal/ociartifact/store.go (1)
102-111: Consider wrapping the error fromRemovefor consistent error handling.Other methods in this file wrap errors with
fmt.Errorfand%w, butRemovereturns the error directly on line 108. This inconsistency may lose context in error chains.♻️ Suggested fix for consistency
func (s *Store) Remove(ctx context.Context, nameOrDigest string) error { artRef, err := libart.NewArtifactStorageReference(nameOrDigest) if err != nil { return fmt.Errorf("invalid nameOrDigest: %w", err) } - _, err = s.libartifactStore.Remove(ctx, artRef) - - return err + if _, err = s.libartifactStore.Remove(ctx, artRef); err != nil { + return fmt.Errorf("remove artifact: %w", err) + } + + return nil }As per coding guidelines: "Use
fmt.Errorfwith%wfor error wrapping in Go code".internal/ociartifact/datastore/store.go (1)
228-231: Consider documenting the minimum digest prefix length.The
len(strRef) >= 3check enforces a minimum prefix length for digest matching but lacks documentation explaining why 3 was chosen. Adding a brief comment would improve maintainability.📝 Suggested comment
- // if strRef is a just digest or short digest + // if strRef is a digest or short digest prefix (minimum 3 chars to avoid overly ambiguous matches) if idx := slices.IndexFunc(artifacts, func(a *ociartifact.Artifact) bool { return strings.HasPrefix(a.Digest().Encoded(), strRef) }); len(strRef) >= 3 && idx != -1 {As per coding guidelines: "Add comments explaining 'why' not 'what' in Go code".
|
|
||
| if size != -1 && size > int64(maxArtifactSize)+1 { | ||
| return nil, fmt.Errorf("exceeded maximum allowed size of %d bytes for a single layer", maxArtifactSize) | ||
| } |
There was a problem hiding this comment.
Off-by-one in size comparison allows one extra byte.
The check size > int64(maxArtifactSize)+1 allows blobs up to maxArtifactSize + 1 bytes, not maxArtifactSize. This should likely be size > int64(maxArtifactSize) for consistency with the error message and intent.
🛡️ Proposed fix
- if size != -1 && size > int64(maxArtifactSize)+1 {
+ if size != -1 && size > int64(maxArtifactSize) {
return nil, fmt.Errorf("exceeded maximum allowed size of %d bytes for a single layer", maxArtifactSize)
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if size != -1 && size > int64(maxArtifactSize)+1 { | |
| return nil, fmt.Errorf("exceeded maximum allowed size of %d bytes for a single layer", maxArtifactSize) | |
| } | |
| if size != -1 && size > int64(maxArtifactSize) { | |
| return nil, fmt.Errorf("exceeded maximum allowed size of %d bytes for a single layer", maxArtifactSize) | |
| } |
🤖 Prompt for AI Agents
In `@internal/ociartifact/datastore/store.go` around lines 171 - 174, The size
check currently allows one extra byte because it compares against
int64(maxArtifactSize)+1; update the conditional in the code that uses variables
size and maxArtifactSize (the block that returns fmt.Errorf("exceeded maximum
allowed size...")) to compare directly against int64(maxArtifactSize) (i.e., use
size > int64(maxArtifactSize)) so blobs larger than maxArtifactSize are rejected
as intended; keep the special-case check for size == -1 unchanged.
| type LibartifactStore interface { | ||
| // Remove an artifact from the local artifact store. | ||
| Remove(ctx context.Context, name string) (*digest.Digest, error) | ||
|
|
||
| // List artifacts in the local store. | ||
| Remove(ctx context.Context, asr libartifact.ArtifactStoreReference) (*digest.Digest, error) | ||
| List(ctx context.Context) (libartifact.ArtifactList, error) | ||
|
|
||
| // Pull an artifact from an image registry to a local store. | ||
| Pull(ctx context.Context, name string, opts libimage.CopyOptions) (digest.Digest, error) | ||
|
|
||
| // SystemContext returns the internal system context | ||
| SystemContext() *types.SystemContext | ||
| } | ||
|
|
||
| type RealLibartifactStore struct { | ||
| *libartStore.ArtifactStore | ||
| } | ||
|
|
||
| func (r RealLibartifactStore) SystemContext() *types.SystemContext { | ||
| return r.ArtifactStore.SystemContext | ||
| Pull(ctx context.Context, ref libartifact.ArtifactReference, opts libimage.CopyOptions) (digest.Digest, error) | ||
| Inspect(ctx context.Context, asr libartifact.ArtifactStoreReference) (*libartifact.Artifact, error) |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Search for old method signatures that should have been updated
echo "=== Searching for old Remove signature ==="
rg -n "Remove\(ctx context\.Context, name string" -g '*.go'
echo "=== Searching for old Pull signature ==="
rg -n "Pull\(ctx context\.Context, name string" -g '*.go'
echo "=== Searching for old SystemContext method ==="
rg -n "SystemContext\(\) \*types\.SystemContext" -g '*.go'
echo "=== Finding all LibartifactStore implementations ==="
rg -n "func.*Remove\(ctx context\.Context, asr" -g '*.go' -A 1
echo "=== Finding Pull implementations ==="
rg -n "func.*Pull\(ctx context\.Context, ref" -g '*.go' -A 1
echo "=== Finding Inspect implementations ==="
rg -n "func.*Inspect\(ctx context\.Context, asr" -g '*.go' -A 1Repository: cri-o/cri-o
Length of output: 2794
🏁 Script executed:
#!/bin/bash
# Check the LibartifactStore interface definition
echo "=== LibartifactStore interface definition ==="
cat -n internal/ociartifact/libartifact_store.go | head -20
echo ""
echo "=== Mock implementation around SystemContext ==="
sed -n '260,275p' test/mocks/ociartifact/ociartifact.goRepository: cri-o/cri-o
Length of output: 1330
Remove stale SystemContext() method from mock implementation
The mock in test/mocks/ociartifact/ociartifact.go still has a SystemContext() method (line 269) that is not defined in the LibartifactStore interface. All other method implementations (Remove, List, Pull, Inspect) are correctly updated; this orphaned mock method should be removed.
🤖 Prompt for AI Agents
In `@internal/ociartifact/libartifact_store.go` around lines 11 - 15, The mock
implementation contains an orphaned SystemContext() method that is not part of
the LibartifactStore interface; remove the SystemContext() method from the
ociartifact mock so the mock only implements the interface methods Remove, List,
Pull, and Inspect, ensuring the mock signature matches LibartifactStore exactly.
| ) (manifestDigest *digest.Digest, err error) { | ||
| strRef := s.impl.DockerReferenceString(ref) | ||
| strRef := ref.DockerReference().String() | ||
|
|
||
| dgst, err := s.Pull(ctx, strRef, *opts) | ||
| artRef, err := libart.NewArtifactReference(strRef) | ||
| if err != nil { | ||
| return nil, fmt.Errorf("invalid reference: %w", err) | ||
| } |
There was a problem hiding this comment.
Potential nil pointer dereference if DockerReference() returns nil.
ref.DockerReference() can return nil for non-Docker references. Calling .String() on a nil reference would panic.
🛡️ Proposed fix with nil check
func (s *Store) Pull(
ctx context.Context,
ref types.ImageReference,
opts *libimage.CopyOptions,
) (manifestDigest *digest.Digest, err error) {
- strRef := ref.DockerReference().String()
+ dockerRef := ref.DockerReference()
+ if dockerRef == nil {
+ return nil, fmt.Errorf("reference is not a Docker reference")
+ }
+ strRef := dockerRef.String()
artRef, err := libart.NewArtifactReference(strRef)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| ) (manifestDigest *digest.Digest, err error) { | |
| strRef := s.impl.DockerReferenceString(ref) | |
| strRef := ref.DockerReference().String() | |
| dgst, err := s.Pull(ctx, strRef, *opts) | |
| artRef, err := libart.NewArtifactReference(strRef) | |
| if err != nil { | |
| return nil, fmt.Errorf("invalid reference: %w", err) | |
| } | |
| ) (manifestDigest *digest.Digest, err error) { | |
| dockerRef := ref.DockerReference() | |
| if dockerRef == nil { | |
| return nil, fmt.Errorf("reference is not a Docker reference") | |
| } | |
| strRef := dockerRef.String() | |
| artRef, err := libart.NewArtifactReference(strRef) | |
| if err != nil { | |
| return nil, fmt.Errorf("invalid reference: %w", err) | |
| } |
🤖 Prompt for AI Agents
In `@internal/ociartifact/store.go` around lines 54 - 60, The code calls
ref.DockerReference().String() without checking for a nil DockerReference, which
can panic for non-Docker references; update the function to call
ref.DockerReference() into a local variable (e.g., dr := ref.DockerReference()),
check if dr == nil and handle that case (return a clear error or fall back to
ref.String() if appropriate), then pass dr.String() (or the fallback) into
libart.NewArtifactReference; reference the symbols DockerReference(), String(),
libart.NewArtifactReference, and the strRef variable when making the change.
|
/retest |
|
@cri-o/cri-o-maintainers PTAL |
1 similar comment
|
@cri-o/cri-o-maintainers PTAL |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: bitoku, saschagrunert The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/cherry-pick release-1.35 |
|
@bitoku: #9691 failed to apply on top of branch "release-1.35": DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
I knew it |
What type of PR is this?
/kind cleanup
What this PR does / why we need it:
This PR fixes cri-o code to adapt it to new interfaces of libartifact store.
podman-container-tools/container-libs@539435a
Which issue(s) this PR fixes:
Special notes for your reviewer:
Does this PR introduce a user-facing change?
Summary by CodeRabbit