feat(oci): enhance repository provider with user agent, temp dir, and caching support#963
Merged
fabianburth merged 15 commits intoOct 7, 2025
Conversation
762a948 to
d440648
Compare
#### What this PR does / why we need it Introduces options for configuring the `CachingComponentVersionRepositoryProvider`: - **UserAgent**: Ability to set a custom user agent header. - **TempDir**: Define a temporary directory for repository operations. #### Key changes - Added an `Options` struct and corresponding `WithTempDir` and `WithUserAgent` helpers. - Updated repository provider constructor to accept these options. - Enhanced caching capabilities with an internal `storeCache`. - Adjusted `NewFromCTFRepoV1` to return both the repository and store, simplifying repository reuse. - Validation checks for unsupported archive operations on read-write access. #### Implications This enhances flexibility by supporting configurable repository behavior, improving cache management, and enforcing operation safety on restricted formats. Signed-off-by: Fabian Burth <fabian.burth@sap.com>
#### Why this change was made Replaced the `ctf` module dependency with a forked version hosted on GitHub: - Ensures compatibility with recent updates or fixes in the forked version. - Removes reliance on the older `ocm.software` hosted variant. #### Key changes - Introduced `replace` directive in `go.mod` to point to the forked dependency. - Updated `go.sum` to reflect the new checksums for the replaced module. - Removed outdated `ctf` module references. #### Implications - Resolves potential issues associated with the older dependency version. - Aligns the project with the updated forked repository to improve maintainability. Signed-off-by: Fabian Burth <fabian.burth@sap.com>
…chive handling --- #### Why this change was made - Updated `ctf.OpenCTFByFileExtension` to discard unused return values for clarity and correctness. - Fixed potential misuse in the credential cache retrieval logic by safely accessing the `store`. --- #### Key changes - Switched to explicitly ignoring the unused `format` value from `ctf.OpenCTFByFileExtension` for better readability. - Removed unnecessary variable destructuring in `credential_cache.go` to prevent silent errors. --- #### Implications - Enhances code maintainability by removing misleading or redundant constructs. - Reduces the chance of introducing unexpected behavior in future updates. Signed-off-by: Fabian Burth <fabian.burth@sap.com>
--- #### Why this change was made - Inserted a blank line in `go.mod` to follow standard formatting conventions. #### Key changes - Added a blank line between the `replace` directive and `require` block for improved readability. #### Implications - Ensures adherence to Go module formatting best practices. - Enhances the readability and maintainability of `go.mod`. Signed-off-by: Fabian Burth <fabian.burth@sap.com>
--- #### Why this change was made Refactored the repository provider and caching mechanisms for improved readability, maintainability, and safety: - Simplified store caching with `loadOrStore` to reduce boilerplate code and ensure thread safety. - Modularized `NewFromCTFRepoV1` by delegating store creation to `NewStoreFromCTFRepoV1`, enhancing composability. --- #### Key changes - Replaced `storeCache.get` and `storeCache.add` methods with `loadOrStore` for atomic cache operations. - Decoupled store creation logic into `NewStoreFromCTFRepoV1` for easier reuse. - Updated concurrency handling in `Resolve` to use mutual exclusion for safety. - Adjusted dependencies in `go.mod` and `go.sum` to use `ctf v0.3.0`. --- #### Implications - Reduces redundancy and potential race conditions in cache handling. - Makes repository creation logic modular and easier to maintain. - Updates dependency for compatibility with recent changes. Signed-off-by: Fabian Burth <fabian.burth@sap.com>
--- #### Why this change was made - Removed the redundant third return value (`store`) in `NewFromCTFRepoV1` calls within repository tests. - Adjusted test logic to align with the updated function signature. --- #### Key changes - Refactored `repository_test.go` to exclude the unused `store` variable. - Updated assertions to maintain proper error handling without the removed return value. --- #### Implications - Enhances code clarity and reduces unnecessary complexity in test cases. - Aligns with recent modularization and simplifications in repository creation logic. Signed-off-by: Fabian Burth <fabian.burth@sap.com>
--- #### Why this change was made - Improved concurrency safety in test cases for `ComponentVersion` operations. - Introduced a thread-safe `storeCache` to manage repository stores. - Replaced `tags` locking mechanism in `store.go` for stricter mutual exclusion. --- #### Key changes - Added a new `storeCache` implementation to `provider` for atomic store handling, ensuring consistent behavior across multiple threads. - Refactored tests to simulate concurrent access scenarios with isolated versions to prevent blob write races. - Enhanced `Tags` method in `store.go` to use exclusive locking for race prevention. - Removed redundant `storeCache` implementation from `credential_cache.go`. --- #### Implications - Improves test reliability in simulated concurrent repository operations. - Reduces the risk of data corruption and races in repository store caching and blob management. - Simplifies and centralizes store caching logic for better maintainability. Signed-off-by: Fabian Burth <fabian.burth@sap.com>
Signed-off-by: Fabian Burth <fabian.burth@sap.com>
Signed-off-by: Fabian Burth <fabian.burth@sap.com>
b51005a to
ee6d166
Compare
Signed-off-by: Jakob Möller <jakob.moeller@sap.com>
ikhandamirov
previously approved these changes
Oct 7, 2025
Skarlso
reviewed
Oct 7, 2025
Signed-off-by: Fabian Burth <fabian.burth@sap.com>
ikhandamirov
previously approved these changes
Oct 7, 2025
jakobmoellerdev
left a comment
Member
There was a problem hiding this comment.
pretty much only nits on docs due to the complexity of the change. change itself is lgtm
Signed-off-by: Fabian Burth <fabian.burth@sap.com>
Signed-off-by: Fabian Burth <fabian.burth@sap.com>
jakobmoellerdev
approved these changes
Oct 7, 2025
Skarlso
approved these changes
Oct 7, 2025
matthiasbruns
pushed a commit
that referenced
this pull request
Oct 7, 2025
<!-- markdownlint-disable MD041 --> #### What this PR does / why we need it This PR fixes is supposed to fix the race conditions occuring when multiple ctf instances operate on the same ctf archive. The actual fix is provided in #964 and #963. Here, a the archive instance and its lock are shared between multiple ctf repositories operating on the archive. #### Which issue(s) this PR fixes <!-- Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`. --> Contributes to open-component-model/ocm-project#564 --------- Signed-off-by: Fabian Burth <fabian.burth@sap.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it
Core features:
storeCache, enabling repository caching based on path to enable sharing a lock.This (hopefully) fixes the current bugs related to race conditions.
AddComponentVersionandListComponentVersion.Summary:
WithUserAgent,WithTempDir) for more flexible configuration of repository providers.UserAgentvalue (DefaultCreator).NewFromCTFRepoV1to return both repository and store, allowing to cache the store.OpenCTFByFileExtensionfor better modularity by separating format detection intoDiscoverCTFFormatFromPath.Implications
Which issue(s) this PR fixes
Contributes to open-component-model/ocm-project#564