Skip to content

Add support for additional read-only artifact stores#9702

Merged
openshift-merge-bot[bot] merged 2 commits into
cri-o:mainfrom
pauloappbr:feat/artifact-additional-stores
Mar 18, 2026
Merged

Add support for additional read-only artifact stores#9702
openshift-merge-bot[bot] merged 2 commits into
cri-o:mainfrom
pauloappbr:feat/artifact-additional-stores

Conversation

@pauloappbr

@pauloappbr pauloappbr commented Jan 8, 2026

Copy link
Copy Markdown
Contributor

What type of PR is this?

/kind feature

What this PR does / why we need it:

Adds support for additional read-only OCI artifact stores via the additional_artifact_stores configuration option and --additional-artifact-stores CLI flag. This allows pre-populated artifacts (e.g. large AI/ML models on NFS) to be used without re-downloading them into the main store.

Artifacts in additional stores take priority over the main store. Pulls are skipped when an artifact already exists in any store. Tag re-pointing is not supported for read-only stores.

Which issue(s) this PR fixes:

Fixes #9570
Related to #9583

Does this PR introduce a user-facing change?

Added support for configuring additional read-only artifact stores via the `additional_artifact_stores` configuration option.

@pauloappbr pauloappbr requested a review from mrunalp as a code owner January 8, 2026 16:40
@openshift-ci openshift-ci Bot requested review from klihub and littlejawa January 8, 2026 16:40
@openshift-ci openshift-ci Bot added kind/feature Categorizes issue or PR as related to a new feature. dco-signoff: yes Indicates the PR's author has DCO signed all their commits. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jan 8, 2026
@openshift-ci

openshift-ci Bot commented Jan 8, 2026

Copy link
Copy Markdown
Contributor

Hi @pauloappbr. Thanks for your PR.

I'm waiting for a cri-o member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@coderabbitai

coderabbitai Bot commented Jan 8, 2026

Copy link
Copy Markdown

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds support for configurable read-only auxiliary OCI artifact stores. NewStore accepts additionalPaths; Store tracks and queries additionalStores for Status/List/Pull (RO-first, dedup by digest). Removal remains limited to the main writable store. Config gains ReadOnlyArtifactStores and call sites updated.

Changes

Cohort / File(s) Summary
OCI artifact store core
internal/ociartifact/store.go, internal/ociartifact/artifact.go
NewStore signature accepts additionalPaths []string; Store stores additionalPaths, additionalStores, and validAdditionalPaths. Pull/Status/List consult additional (RO) stores first; List dedupes by digest preserving RO priority; Remove only deletes from main store. Artifact gained rootPath field and RootPath() accessor.
Call sites / integrations
server/server.go, internal/storage/image.go, internal/ociartifact/datastore/store.go
Updated NewStore invocations to pass additionalPaths (from config or nil); one code path now calls NewStore(..., nil, ...). Datastore adapter uses NewStore(rootPath, nil, systemContext).
Runtime configuration
pkg/config/config.go
Added ReadOnlyArtifactStores []string to RuntimeConfig (toml:"read_only_artifact_stores,omitempty"). Added testing/injection setters SetCgroupManager and SetCheckpointRestore.
Tests / fixtures
internal/factory/container/device_test.go
Test expectations updated: rspec.LinuxDevice entries now include FileMode *os.FileMode set to 0o660.

Sequence Diagram(s)

sequenceDiagram
  participant Server as CRI-O Server
  participant RO as Read-Only Store(s)
  participant Main as Writable Store
  participant Registry as Remote Registry

  Note over Server,RO: Request to ensure/pull artifact by reference
  Server->>RO: Query Status/List for artifact (in configured order)
  alt found in RO
    RO-->>Server: Return artifact (digest/path)
  else not found in any RO
    Server->>Main: Query Status/List
    alt found in Main
      Main-->>Server: Return artifact (digest/path)
    else not found
      Server->>Registry: Pull artifact (copy.Image)
      Registry-->>Main: Write artifact into main writable store
      Main-->>Server: Return artifact (digest/path)
    end
  end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Suggested reviewers

  • mrunalp
  • littlejawa

Poem

🐰 I hopped through paths both new and old,

found quiet stores, read-only and bold.
I checked the shelves, then skipped the rest,
pulled what was missing, left the best.
A rabbit's hop — artifacts blessed!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Linked Issues check ✅ Passed All primary coding objectives from issue #9570 are met: configuration field added [#9570], multi-store initialization implemented [#9570], artifact resolution across stores added [#9570], read-only enforcement confirmed [#9570], and lookup order implemented [#9570].
Out of Scope Changes check ✅ Passed All changes relate directly to the linked issue objectives: artifact store configuration and multi-store support. Test updates address required interface changes. Device test updates are necessary compilation fixes.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and accurately summarizes the primary change: adding support for read-only artifact stores, which is the main objective of this multi-file feature.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
📝 Coding Plan
  • Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (6)
pkg/config/config.go (1)

425-427: Consider adding path validation for AdditionalArtifactStores.

The new AdditionalArtifactStores field lacks validation in the RuntimeConfig.Validate() method. Similar configuration fields like HooksDir (lines 1364-1385) include checks to ensure paths are valid directories. Consider adding validation to:

  • Check if paths are absolute
  • Verify paths exist or can be created
  • Log warnings for invalid paths rather than failing startup (to match the behavior in internal/ociartifact/store.go which ignores invalid stores)

[scratchpad_end] -->

♻️ Example validation pattern

You could add validation in the RuntimeConfig.Validate() method similar to the HooksDir pattern:

// Validate additional artifact stores
for _, storePath := range c.AdditionalArtifactStores {
	if !filepath.IsAbs(storePath) {
		logrus.Warnf("Additional artifact store path is not absolute: %s", storePath)
		continue
	}
	if err := os.MkdirAll(storePath, 0o755); err != nil {
		logrus.Warnf("Cannot create additional artifact store directory %s: %v", storePath, err)
	}
}

Note: Since internal/ociartifact/store.go already silently ignores invalid stores, this validation would be informational and help users catch configuration errors early.

internal/storage/image.go (1)

904-904: Pass AdditionalArtifactStores to the artifact fallback path for consistency.

The OCI artifact store created in the fallback path (line 904) uses nil for additional stores, while the main image pull path in server/server.go:461 explicitly passes config.AdditionalArtifactStores. To ensure consistent artifact resolution across all code paths, consider threading the additional artifact stores through ImageCopyOptions to pullImageImplementation. This would allow the fallback artifact retrieval to check additional read-only stores, matching behavior with the primary pull path.

internal/ociartifact/store.go (4)

57-69: Consider logging errors when ignoring invalid additional stores.

While silently ignoring invalid stores prevents CRI-O startup failures, users may not realize their configured additional stores are being skipped. Adding a warning log would improve observability.

📋 Proposed improvement
 	for _, path := range additionalPaths {
 		// Assume the pattern is to have an "artifacts" directory within the configured path,
 		// similar to how additionalimagestores works in containers/storage.
 		addPath := filepath.Join(path, "artifacts")
 
 		addStore, err := libartStore.NewArtifactStore(addPath, systemContext)
 		if err != nil {
-			// Just ignore invalid stores to prevent breaking CRI-O startup
+			// Just ignore invalid stores to prevent breaking CRI-O startup
+			log.Warnf(context.Background(), "Ignoring invalid additional artifact store %q: %v", path, err)
 			continue
 		}
 		additionalStores = append(additionalStores, RealLibartifactStore{addStore})
 		validAdditionalPaths = append(validAdditionalPaths, addPath)
 	}

As per coding guidelines, use logrus with structured fields for logging.


110-110: Use structured logging fields instead of string formatting.

The logging statement uses string formatting, but coding guidelines recommend using logrus with structured fields.

📋 Proposed improvement
-	log.Infof(ctx, "Checking/Pulling OCI artifact from ref: %s", ref)
+	log.WithFields(log.Fields{
+		"ref": ref,
+	}).Infof(ctx, "Checking/Pulling OCI artifact")

As per coding guidelines, use logrus with structured fields for logging.


166-172: Consider logging errors when listing additional stores.

Errors from additional stores are silently ignored, which could hide issues like permission problems or corrupted stores. Adding warning logs would improve observability without breaking functionality.

📋 Proposed improvement
 	for _, addStore := range s.additionalStores {
 		addArts, err := addStore.List(ctx)
 		if err != nil {
+			log.Warnf(ctx, "Failed to list artifacts from additional store: %v", err)
 			continue
 		}
 		arts = append(arts, addArts...)
 	}

As per coding guidelines, use logrus with structured fields for logging.


473-488: Consider logging errors during path resolution.

The resolveRootPath function silently ignores errors from List() calls (lines 475, 481). While this might be intentional for robustness with read-only stores, logging these errors would help diagnose configuration or permission issues.

📋 Proposed improvement
 func (s *Store) resolveRootPath(ctx context.Context, nameOrDigest string) (string, error) {
-	arts, _ := s.LibartifactStore.List(ctx)
+	arts, err := s.LibartifactStore.List(ctx)
+	if err != nil {
+		log.Debugf(ctx, "Failed to list artifacts from main store during path resolution: %v", err)
+	}
 	if s.containsArtifact(arts, nameOrDigest) {
 		return s.rootPath, nil
 	}
 
 	for i, store := range s.additionalStores {
-		arts, _ := store.List(ctx)
+		arts, err := store.List(ctx)
+		if err != nil {
+			log.Debugf(ctx, "Failed to list artifacts from additional store during path resolution: %v", err)
+			continue
+		}
 		if s.containsArtifact(arts, nameOrDigest) {
 			return s.additionalPaths[i], nil
 		}
 	}
 
 	return "", fmt.Errorf("artifact %s not found in any store", nameOrDigest)
 }

As per coding guidelines, use logrus with structured fields for logging.

📜 Review details

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5669e52 and b07cdc4.

📒 Files selected for processing (7)
  • internal/config/seccomp/seccompociartifact/seccompociartifact.go
  • internal/oci/container.go
  • internal/ociartifact/store.go
  • internal/ociartifact/store_test.go
  • internal/storage/image.go
  • pkg/config/config.go
  • server/server.go
🧰 Additional context used
📓 Path-based instructions (2)
**/*.go

📄 CodeRabbit inference engine (AGENTS.md)

**/*.go: Use interface-based design and dependency injection patterns in Go code
Propagate context.Context through function calls in Go code
Use fmt.Errorf with %w for error wrapping in Go code
Use logrus with structured fields for logging in Go code
Add comments explaining 'why' not 'what' in Go code
Use platform-specific file naming: *_{linux,freebsd}.go for platform-dependent code

Files:

  • internal/storage/image.go
  • internal/oci/container.go
  • server/server.go
  • internal/config/seccomp/seccompociartifact/seccompociartifact.go
  • pkg/config/config.go
  • internal/ociartifact/store.go
  • internal/ociartifact/store_test.go
**/*_test.go

📄 CodeRabbit inference engine (AGENTS.md)

Use *_test.go naming convention for unit test files

Files:

  • internal/ociartifact/store_test.go
🧠 Learnings (3)
📓 Common learnings
Learnt from: CR
Repo: cri-o/cri-o PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-03T18:27:19.593Z
Learning: When adding/changing features, update related repositories: cri-o.io website and packaging repositories
📚 Learning: 2025-12-17T13:38:34.646Z
Learnt from: bitoku
Repo: cri-o/cri-o PR: 9667
File: server/container_create.go:1233-1236
Timestamp: 2025-12-17T13:38:34.646Z
Learning: In the cri-o/cri-o repository, protobuf-generated Get* methods for k8s.io/cri-api types are nil-safe: if the receiver is nil, GetX() returns the zero value instead of panicking. Do not add explicit nil checks before chaining calls on such getters. Apply this guidance to all Go code that uses these generated getters across the codebase.

Applied to files:

  • internal/storage/image.go
  • internal/oci/container.go
  • server/server.go
  • internal/config/seccomp/seccompociartifact/seccompociartifact.go
  • pkg/config/config.go
  • internal/ociartifact/store.go
  • internal/ociartifact/store_test.go
📚 Learning: 2025-12-18T13:28:24.244Z
Learnt from: bitoku
Repo: cri-o/cri-o PR: 9676
File: internal/lib/stats/cgroup_stats_unsupported.go:1-7
Timestamp: 2025-12-18T13:28:24.244Z
Learning: In the cri-o/cri-o repository, for platform-specific types guarded by Go build tags (for example //go:build !linux), implement empty structs for unsupported platforms to permit compilation and clearly indicate the feature is not available rather than mirroring the Linux struct with unpopulated fields. Apply this pattern to all relevant platform-specific files across the codebase (i.e., any file under build-taged sections that should compile on all targets but lacks full implementation for some platforms).

Applied to files:

  • internal/storage/image.go
  • internal/oci/container.go
  • server/server.go
  • internal/config/seccomp/seccompociartifact/seccompociartifact.go
  • pkg/config/config.go
  • internal/ociartifact/store.go
  • internal/ociartifact/store_test.go
🧬 Code graph analysis (6)
internal/storage/image.go (1)
internal/ociartifact/store.go (1)
  • NewStore (46-78)
server/server.go (1)
internal/ociartifact/store.go (2)
  • NewStore (46-78)
  • Store (36-43)
internal/config/seccomp/seccompociartifact/seccompociartifact.go (2)
internal/ociartifact/store.go (1)
  • NewStore (46-78)
internal/config/seccomp/seccompociartifact/impl.go (1)
  • Impl (10-12)
pkg/config/config.go (2)
internal/config/cgmgr/cgmgr_linux.go (2)
  • SetCgroupManager (112-133)
  • CgroupManager (41-98)
internal/config/cgmgr/cgmgr_unsupported.go (2)
  • SetCgroupManager (40-42)
  • CgroupManager (9-27)
internal/ociartifact/store.go (5)
internal/ociartifact/libartifact_store.go (2)
  • LibartifactStore (13-25)
  • RealLibartifactStore (27-29)
internal/ociartifact/impl.go (1)
  • Impl (16-28)
vendor/go.podman.io/common/pkg/libartifact/store/store.go (1)
  • NewArtifactStore (50-84)
internal/ociartifact/artifact.go (2)
  • Artifact (13-18)
  • ArtifactData (21-23)
vendor/go.podman.io/common/pkg/libartifact/artifact.go (1)
  • Artifact (13-18)
internal/ociartifact/store_test.go (4)
test/mocks/ociartifact/ociartifact.go (2)
  • MockLibartifactStore (213-217)
  • NewMockLibartifactStore (225-229)
internal/ociartifact/store.go (3)
  • Store (36-43)
  • NewStore (46-78)
  • FakeLibartifactStore (518-520)
internal/ociartifact/store_test_inject.go (1)
  • FakeLibartifactStore (21-23)
internal/ociartifact/libartifact_store.go (1)
  • LibartifactStore (13-25)
🔇 Additional comments (10)
pkg/config/config.go (1)

2233-2249: LGTM - Testing helpers follow Go conventions.

These setter methods provide appropriate dependency injection points for testing. The "Used for testing only" comments clearly indicate their purpose. This pattern is common in Go codebases for enabling test isolation.

server/server.go (1)

461-461: LGTM - Correctly wires additional artifact stores configuration.

The artifact store initialization properly passes config.AdditionalArtifactStores to enable support for multiple read-only artifact stores. This change aligns with the updated NewStore signature and implements the feature as intended.

internal/oci/container.go (1)

1003-1016: LGTM - Testing helpers are well-structured.

These test-only setter methods provide appropriate mechanisms for test setup:

  • SetState allows direct state injection
  • SetStateAndSpoofPid additionally sets a deterministic PID (12345) for testing scenarios

The hardcoded PID value is clearly for testing and aligns with the existing Spoofed() pattern in the container struct.

internal/config/seccomp/seccompociartifact/seccompociartifact.go (2)

24-24: LGTM - NewStore call updated correctly.

The artifact store initialization has been updated to match the new three-argument signature, passing nil for additional stores. This is appropriate for the seccomp artifact context.


94-98: LGTM - Test injection method follows best practices.

The SetImpl method provides a clean dependency injection point for testing, allowing tests to mock the artifact store implementation. This follows common Go testing patterns.

internal/ociartifact/store.go (2)

262-265: Verify fallback behavior is intentional.

When resolveRootPath fails, this function falls back to s.rootPath, whereas BlobMountPaths (lines 420-423) returns an error. This inconsistency suggests different design intentions, but it should be verified that the fallback behavior here is correct.

If the artifact doesn't exist in any store, using s.rootPath will likely cause a subsequent error when trying to access the artifact data. Consider whether it's better to return the error immediately for clearer error messages.


504-526: LGTM: Clean testing infrastructure.

The testing helpers follow good dependency injection patterns and are clearly documented. The approach of exposing SetImpl and SetFakeStore methods allows for flexible test setup while keeping the production code clean.

internal/ociartifact/store_test.go (3)

50-77: LGTM: Test properly adapted to new Store structure.

The test correctly:

  • Creates Store with nil for additionalPaths (testing the base case)
  • Injects mocks using SetImpl and SetFakeStore
  • Sets up appropriate expectations with AnyTimes() for methods called multiple times
  • Verifies error messages contain expected substrings

The test maintains good coverage while adapting to the new multi-store architecture.


79-109: LGTM: Consistent test pattern.

This test follows the same well-structured pattern as the first test, maintaining consistency in the test suite. The error path coverage for DockerNewReference is valuable.


112-188: LGTM: Comprehensive test coverage for Status function.

The Status tests provide good coverage of different scenarios:

  • Unqualified names with empty store (should return ErrNotFound)
  • Unqualified names with artifacts present (should return validation error)
  • Fully-qualified names (should succeed)

The test setup is consistent with other tests in the file and properly injects dependencies.

@openshift-ci openshift-ci Bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Jan 8, 2026

@bitoku bitoku left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR!

We're in the middle of migration from our implementation to libartifact in c/container-libs.
I think we should wait for a while until it's done to avoid rework.

Comment thread pkg/config/config.go Outdated
@pauloappbr pauloappbr marked this pull request as draft January 10, 2026 17:01
@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 10, 2026
@saschagrunert

Copy link
Copy Markdown
Member

Ref: #9691

@codecov

codecov Bot commented Jan 13, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 81.08108% with 21 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.59%. Comparing base (ad5f448) to head (a21c685).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9702      +/-   ##
==========================================
+ Coverage   64.68%   67.59%   +2.90%     
==========================================
  Files         212      212              
  Lines       29235    29327      +92     
==========================================
+ Hits        18911    19823     +912     
+ Misses       8635     7812     -823     
- Partials     1689     1692       +3     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions

Copy link
Copy Markdown

A friendly reminder that this PR had no activity for 30 days.

@github-actions github-actions Bot added lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Feb 13, 2026
@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 16, 2026
@pauloappbr pauloappbr force-pushed the feat/artifact-additional-stores branch from b07cdc4 to 4f3cb91 Compare February 23, 2026 15:03
@openshift-merge-robot openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 23, 2026
@pauloappbr pauloappbr marked this pull request as ready for review February 23, 2026 15:12
@openshift-ci openshift-ci Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 23, 2026
@openshift-ci openshift-ci Bot requested a review from QiWang19 February 23, 2026 15:12

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
internal/factory/container/device_test.go (1)

299-303: Consider using octal notation for file mode clarity.

os.FileMode(432) is correct but opaque. Using os.FileMode(0o660) makes the intent (owner+group read-write) immediately obvious to reviewers.

🔧 Suggested change
-						FileMode: func() *os.FileMode { m := os.FileMode(432); return &m }(),
+						FileMode: func() *os.FileMode { m := os.FileMode(0o660); return &m }(),

Apply similarly at lines 310, 390, and 397.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@internal/factory/container/device_test.go` around lines 299 - 303, Replace
opaque decimal filemode literals with octal notation for clarity: change
instances like FileMode: func() *os.FileMode { m := os.FileMode(432); return &m
}() to FileMode: func() *os.FileMode { m := os.FileMode(0o660); return &m }(),
and apply the same replacement for the other FileMode occurrences in the test
(the FileMode struct fields shown in the device test cases) so each uses the
equivalent 0o... octal literal instead of a decimal number.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@internal/ociartifact/store.go`:
- Around line 94-106: Store.List currently concatenates results from
s.libartifactStore and s.additionalStores causing duplicate Artifact entries;
modify Store.List to deduplicate artifacts by a uniqueness key (e.g.,
Artifact.Digest()) when merging. Implement a seen map keyed by the digest (or
the Artifact's unique identifier) and only append artifacts whose key is not yet
in the map; keep using s.libartifactStore and s.additionalStores and ensure
callers like getByNameOrDigest continue to work unchanged. Ensure nil-safe
handling if Digest() can return empty and preserve original ordering by adding
main store items first followed by first-seen items from additional stores.
- Around line 40-54: The loop over additionalPaths in
internal/ociartifact/store.go currently swallows errors from
libart.NewArtifactStore; update the error branch inside the for loop (the block
handling addStore, err := libart.NewArtifactStore(addPath, systemContext)) to
emit a warning using logrus with structured fields (include the attempted path
and the error) — e.g., use logrus.WithFields(map[string]interface{}{"path":
addPath, "err": err}).Warn("skipping additional artifact store") — and then
continue to preserve existing behavior; do not change the control flow or how
additionalStores and validAdditionalPaths are appended. Ensure you import logrus
if not already present. If you prefer propagating context, consider evolving
NewArtifactStore to accept context.Context in a follow-up, but for this change
use logrus structured warning in the error branch.
- Around line 153-171: In resolveRootPath, wrap the error returned by
libart.NewArtifactStorageReference using fmt.Errorf with %w to preserve the
original error; specifically change the error return when
NewArtifactStorageReference(nameOrDigest) fails to return fmt.Errorf("invalid
nameOrDigest: %w", err) so it matches the wrapping style used in Status and
Remove.

---

Nitpick comments:
In `@internal/factory/container/device_test.go`:
- Around line 299-303: Replace opaque decimal filemode literals with octal
notation for clarity: change instances like FileMode: func() *os.FileMode { m :=
os.FileMode(432); return &m }() to FileMode: func() *os.FileMode { m :=
os.FileMode(0o660); return &m }(), and apply the same replacement for the other
FileMode occurrences in the test (the FileMode struct fields shown in the device
test cases) so each uses the equivalent 0o... octal literal instead of a decimal
number.

ℹ️ Review info

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between b07cdc4 and 4f3cb91.

📒 Files selected for processing (6)
  • internal/factory/container/device_test.go
  • internal/ociartifact/datastore/store.go
  • internal/ociartifact/store.go
  • internal/storage/image.go
  • pkg/config/config.go
  • server/server.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • internal/storage/image.go

Comment thread internal/ociartifact/store.go
Comment thread internal/ociartifact/store.go
Comment thread internal/ociartifact/store.go Outdated
@bitoku

bitoku commented Feb 23, 2026

Copy link
Copy Markdown
Contributor

/ok-to-test

@saschagrunert saschagrunert force-pushed the feat/artifact-additional-stores branch 2 times, most recently from 6cc110b to e7d9dce Compare March 17, 2026 15:23
@saschagrunert

Copy link
Copy Markdown
Member

I pushed a commit on top of this branch to address the review comments. PTAL again @cri-o/cri-o-maintainers

/cherry-pick release-1.35

@openshift-cherrypick-robot

Copy link
Copy Markdown

@saschagrunert: once the present PR merges, I will cherry-pick it on top of release-1.35 in a new PR and assign it to you.

Details

In response to this:

I pushed a commit on top of this branch to address the review comments. PTAL again @cri-o/cri-o-maintainers

/cherry-pick release-1.35

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@saschagrunert saschagrunert left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/hold

@openshift-ci openshift-ci Bot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. lgtm Indicates that a PR is ready to be merged. labels Mar 17, 2026
@openshift-ci

openshift-ci Bot commented Mar 17, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: pauloappbr, saschagrunert

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Comment thread internal/ociartifact/store.go Outdated
Comment on lines +114 to +125
// Skip pulling if the artifact already exists in any store (including
// read-only additional stores). This avoids duplicating artifacts that
// are already available and prevents unnecessary network traffic.
// To force a re-pull (e.g., after a tag re-point), the artifact must
// be removed first.
if art, err := s.Status(ctx, strRef); err == nil {
log.Infof(ctx, "Artifact %s already exists locally in %s, skipping pull", strRef, art.RootPath())
dgst := art.Digest()

return &dgst, nil
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will change the current behavior even without additional store.

registry.example.com/models/llm:v2 which resolves to digest sha256:aaa won't be updated once it's in the main store even if someone pushes a new artifact to registry.example.com/models/llm:v2, same tag with different digest sha256:bbb.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. The skip now only applies to artifacts found in read-only additional stores. Artifacts in the main store are always re-pulled, so tag re-pointing works as expected.

@openshift-ci openshift-ci Bot removed the lgtm Indicates that a PR is ready to be merged. label Mar 18, 2026
Comment thread internal/ociartifact/datastore/store.go Outdated
Size: uint64(a.TotalSizeBytes()),
RepoTags: repoTags,
RepoDigests: []string{a.CanonicalName()},
Pinned: true,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's expected that artifacts are garbage collected, isn't it? There should be a check in GC to check whether the artifact/image is used.

Comment thread docs/crio.conf.5.md Outdated
Comment thread internal/ociartifact/store.go Outdated
@saschagrunert saschagrunert force-pushed the feat/artifact-additional-stores branch from e7d9dce to 1de2bd2 Compare March 18, 2026 12:11
@saschagrunert saschagrunert removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 18, 2026
@saschagrunert

Copy link
Copy Markdown
Member

Integration tests fail all over the place because critest requires go 1.26 now. We may wanna push #9774 for that.

pauloappbr and others added 2 commits March 18, 2026 17:11
This commit implements the logic allowing users to configure additional
read-only paths for OCI artifacts. This addresses use cases where artifacts
(such as large AI models) are pre-populated on shared filesystems or
read-only media, preventing redundant downloads to the graph root.

Changes included:

1. Core Logic (internal/ociartifact):
   - Updated `Store` struct and `NewStore` to accept `additionalPaths`.
   - Implemented `resolveRootPath` helper to look up artifacts in additional
     stores before falling back to the main writeable store.
   - Updated `PullData`, `List`, and `BlobMountPaths` to respect these paths.

2. Configuration (pkg/config):
   - Added `AdditionalArtifactStores` slice to `RuntimeConfig`.

3. Wiring (server):
   - Injected the configuration value into the artifact store initialization.

4. Build & Test Fixes:
   - Updated `NewStore` signatures in `internal/storage/image.go` and
     `seccompociartifact.go`.
   - Added missing test helpers (`SetState`, `SetImpl`, `SetCgroupManager`)
     in `container.go`, `config.go`, and `seccompociartifact.go` to ensure
     full project compilation and test passing.

Fixes: cri-o#9570
Related: cri-o#9583

Signed-off-by: Paulo Henrique <paulo.hco47@gmail.com>
- Only skip pulling for artifacts found in additional (read-only) stores,
  not the main store, so tag re-pointing still works as expected
- Deduplicate List results by reference only (additional stores win)
- Move implementation detail from doc comment into function body
- Mark additional_artifact_stores as experimental, subject to change

Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
@saschagrunert saschagrunert force-pushed the feat/artifact-additional-stores branch from 1de2bd2 to a21c685 Compare March 18, 2026 16:11
@saschagrunert

Copy link
Copy Markdown
Member

Rebased

@bitoku

bitoku commented Mar 18, 2026

Copy link
Copy Markdown
Contributor

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Mar 18, 2026
@saschagrunert saschagrunert changed the title feat: add support for additional read-only artifact stores Add support for additional read-only artifact stores Mar 18, 2026
@openshift-merge-bot openshift-merge-bot Bot merged commit 4d23485 into cri-o:main Mar 18, 2026
70 of 74 checks passed
@openshift-cherrypick-robot

Copy link
Copy Markdown

@saschagrunert: new pull request created: #9829

Details

In response to this:

I pushed a commit on top of this branch to address the review comments. PTAL again @cri-o/cri-o-maintainers

/cherry-pick release-1.35

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci

openshift-ci Bot commented Mar 18, 2026

Copy link
Copy Markdown
Contributor

@pauloappbr: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/ci-e2e-conmonrs a21c685 link unknown /test ci-e2e-conmonrs

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. dco-signoff: yes Indicates the PR's author has DCO signed all their commits. kind/feature Categorizes issue or PR as related to a new feature. lgtm Indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note Denotes a PR that will be considered when it comes time to generate release notes.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Default artifact location

5 participants