fix: Add a maximum resource size to the deployer controller#2022
Conversation
Signed-off-by: Frederic Wilhelm <frederic.wilhelm@sap.com>
Signed-off-by: Frederic Wilhelm <frederic.wilhelm@sap.com>
Signed-off-by: Frederic Wilhelm <frederic.wilhelm@sap.com>
Signed-off-by: Frederic Wilhelm <frederic.wilhelm@sap.com>
Signed-off-by: Frederic Wilhelm <frederic.wilhelm@sap.com>
Signed-off-by: Frederic Wilhelm <frederic.wilhelm@sap.com>
Signed-off-by: Frederic Wilhelm <frederic.wilhelm@sap.com>
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThis PR adds a configurable maximum downloadable resource size to the deployer: a Helm value and CLI flag, validation and parsing of that quantity, wiring into the deployer Reconciler, and enforcement via a new limitedReadCloser with unit tests. Changes
Sequence DiagramsequenceDiagram
participant Config as Helm/CLI Config
participant Main as Controller Main
participant Reconciler as Deployer Reconciler
participant OCM as OCM Blob Provider
participant LimitedReader as limitedReadCloser
participant Cache as Cache/Storage
Config->>Main: provide deployer-download-max-resource-size
Main->>Reconciler: initialize with MaxResourceSizeBytes
Reconciler->>OCM: DownloadResourceWithOCM()
OCM-->>Reconciler: blob (ReadOnlyBlob) [+ size metadata if available]
Reconciler->>Reconciler: check blob.SizeAware for immediate reject
Reconciler->>LimitedReader: wrap blob reader with limit
LimitedReader->>Reconciler: stream bytes (stops or errors if exceeded)
alt within limit
Reconciler->>Cache: cache validated resource
else exceeds limit
Reconciler->>Reconciler: mark not-ready / return error
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…omponent-model into add-verify-and-resource-limit
…omponent-model into add-verify-and-resource-limit
Signed-off-by: Frederic Wilhelm <frederic.wilhelm@sap.com>
Signed-off-by: Frederic Wilhelm <frederic.wilhelm@sap.com>
…omponent-model into add-verify-and-resource-limit
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
There was a problem hiding this comment.
🧹 Nitpick comments (1)
kubernetes/controller/internal/controller/deployer/blob_size.go (1)
1-61: Add unit tests for theblob_size.gomodule.The
limitedReadCloserandapplyResourceSizeLimitfunctions have multiple edge cases that should be covered:
- Blob exactly at the size limit (should succeed)
- Blob exceeding the limit by one byte (should fail with clear error message)
MaxResourceSizeMiB = 0(should bypass size limiting entirely)SizeAwareblob pre-check when declared size exceeds limit (should reject without reading data)- Unknown blob size (
SizeUnknown) falling through to runtime limit enforcement🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@kubernetes/controller/internal/controller/deployer/blob_size.go` around lines 1 - 61, Add unit tests covering limitedReadCloser.Read and Reconciler.applyResourceSizeLimit: create tests for (1) blob exactly at the limit using a reader of size exactly maxBytes and assert reads succeed and EOF, (2) blob exceeding by one byte and assert the read returns the "resource exceeds maximum allowed size" error from limitedReadCloser, (3) Reconciler.MaxResourceSizeMiB == 0 where applyResourceSizeLimit returns the original readerBlob unchanged, (4) a SizeAware mock blob whose Size() returns a value > maxBytes and assert applyResourceSizeLimit returns an immediate error without reading, and (5) a SizeAware mock that returns blob.SizeUnknown and a reader larger than the limit to assert the runtime limitedReadCloser triggers the overflow error; use the Reconciler.applyResourceSizeLimit function and limitedReadCloser types to locate code paths and verify error messages and that wrapped readers are closed/return expected bytes.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@kubernetes/controller/internal/controller/deployer/blob_size.go`:
- Around line 1-61: Add unit tests covering limitedReadCloser.Read and
Reconciler.applyResourceSizeLimit: create tests for (1) blob exactly at the
limit using a reader of size exactly maxBytes and assert reads succeed and EOF,
(2) blob exceeding by one byte and assert the read returns the "resource exceeds
maximum allowed size" error from limitedReadCloser, (3)
Reconciler.MaxResourceSizeMiB == 0 where applyResourceSizeLimit returns the
original readerBlob unchanged, (4) a SizeAware mock blob whose Size() returns a
value > maxBytes and assert applyResourceSizeLimit returns an immediate error
without reading, and (5) a SizeAware mock that returns blob.SizeUnknown and a
reader larger than the limit to assert the runtime limitedReadCloser triggers
the overflow error; use the Reconciler.applyResourceSizeLimit function and
limitedReadCloser types to locate code paths and verify error messages and that
wrapped readers are closed/return expected bytes.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 9591bd68-3da0-4f5d-b686-773d26036ce6
📒 Files selected for processing (8)
kubernetes/controller/chart/README.mdkubernetes/controller/chart/templates/manager/manager.yamlkubernetes/controller/chart/values.schema.jsonkubernetes/controller/chart/values.yamlkubernetes/controller/cmd/main.gokubernetes/controller/internal/controller/deployer/blob_size.gokubernetes/controller/internal/controller/deployer/deployer_controller.gokubernetes/controller/internal/controller/deployer/suite_test.go
Signed-off-by: Frederic Wilhelm <frederic.wilhelm@sap.com>
Signed-off-by: Frederic Wilhelm <frederic.wilhelm@sap.com>
…omponent-model into add-verify-and-resource-limit
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
There was a problem hiding this comment.
🧹 Nitpick comments (1)
kubernetes/controller/internal/controller/deployer/blob_size_test.go (1)
115-194: Good coverage of applyResourceSizeLimit control flow. Consider adding a negative value test.The tests thoroughly cover:
- Disabled limit (0)
- Size-aware blobs: under limit, over limit, unknown size
- Non-size-aware blobs
One gap: there's no test for negative
MaxResourceSizeMiBvalues. Adding such a test would either document the expected behavior or highlight the need for validation.🧪 Suggested test for negative limit
func TestApplyResourceSizeLimit_NegativeLimit(t *testing.T) { r := reconcilerWithLimit(-1) data := []byte("small") b := &mockBlob{data: data} rc := nopCloser{bytes.NewReader(data)} limited, err := r.applyResourceSizeLimit(b, rc) if err != nil { t.Fatalf("unexpected error: %v", err) } // Document current behavior: negative limit creates io.LimitedReader with N < 0, // which returns EOF immediately on first read. buf := make([]byte, 10) n, readErr := limited.Read(buf) if n != 0 || readErr != io.EOF { t.Fatalf("expected immediate EOF with negative limit, got n=%d, err=%v", n, readErr) } }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@kubernetes/controller/internal/controller/deployer/blob_size_test.go` around lines 115 - 194, Add a unit test that covers negative MaxResourceSizeMiB behavior: create a Reconciler via reconcilerWithLimit(-1), build a mockBlob and nopCloser, call applyResourceSizeLimit with those, and assert the observed behavior (e.g., that the returned reader returns immediate EOF on first read or document/expect the current behavior). Name the test TestApplyResourceSizeLimit_NegativeLimit and include assertions that validate both the returned reader type (if any) and the read behavior or returned error so the negative-limit behavior is explicitly specified.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@kubernetes/controller/internal/controller/deployer/blob_size_test.go`:
- Around line 115-194: Add a unit test that covers negative MaxResourceSizeMiB
behavior: create a Reconciler via reconcilerWithLimit(-1), build a mockBlob and
nopCloser, call applyResourceSizeLimit with those, and assert the observed
behavior (e.g., that the returned reader returns immediate EOF on first read or
document/expect the current behavior). Name the test
TestApplyResourceSizeLimit_NegativeLimit and include assertions that validate
both the returned reader type (if any) and the read behavior or returned error
so the negative-limit behavior is explicitly specified.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 37b73d1d-2c24-49dd-91f0-3f4d152fa92f
📒 Files selected for processing (2)
kubernetes/controller/internal/controller/deployer/blob_size_test.gokubernetes/controller/internal/controller/deployer/deployer_controller.go
Signed-off-by: Frederic Wilhelm <frederic.wilhelm@sap.com>
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
1 similar comment
✅ Actions performedReview triggered.
|
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
kubernetes/controller/internal/controller/deployer/deployer_controller.go (1)
334-340:⚠️ Potential issue | 🔴 CriticalDon't cache resources that have no verified cache key.
The TODO already calls out that
resource.Status.Resource.Digest.Valuemay be missing for skipped/unverified resolution, but this still feeds that value directly intoDownloadCache.Load. When the digest is empty, every such resource shares the""entry, so one unverified manifest can be replayed for a different deployer on the next reconcile. Please bypass the cache unless you have a stable verified key here, or switch to the resolver-provided cache key before merging.Possible guard
key := resource.Status.Resource.Digest.Value - - objs, err := r.DownloadCache.Load(key, func() ([]*unstructured.Unstructured, error) { - return r.DownloadResourceWithOCM(ctx, deployer, resource) - }) + var objs []*unstructured.Unstructured + if key == "" { + objs, err = r.DownloadResourceWithOCM(ctx, deployer, resource) + } else { + objs, err = r.DownloadCache.Load(key, func() ([]*unstructured.Unstructured, error) { + return r.DownloadResourceWithOCM(ctx, deployer, resource) + }) + }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@kubernetes/controller/internal/controller/deployer/deployer_controller.go` around lines 334 - 340, The current code uses resource.Status.Resource.Digest.Value as the cache key and calls r.DownloadCache.Load(...) even when that digest is empty, causing different unverified resources to collide on the "" key; change the logic in the reconcile path so that before calling r.DownloadCache.Load you verify the digest is present and non-empty (resource.Status.Resource.Digest.Value != "") and only then use it as the cache key; if the digest is empty, bypass r.DownloadCache.Load and call r.DownloadResourceWithOCM(ctx, deployer, resource) directly (or alternatively use the resolver-provided stable cache key when available) so unverified/skipped-resolution resources are not cached under a shared empty key.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@kubernetes/controller/internal/controller/deployer/blob.go`:
- Around line 45-63: In getLimitedReader, avoid opening the blob stream before
the size pre-check: move the blob.SizeAware check (type assertion to
blob.SizeAware and calling Size()) ahead of calling resourceBlob.ReadCloser() so
you can reject oversized blobs without creating a ReadCloser; keep the existing
maxResourceSizeMiB==0 path but call resourceBlob.ReadCloser() only after all
pre-checks pass, and ensure you still handle and return any error from
resourceBlob.ReadCloser().
---
Outside diff comments:
In `@kubernetes/controller/internal/controller/deployer/deployer_controller.go`:
- Around line 334-340: The current code uses
resource.Status.Resource.Digest.Value as the cache key and calls
r.DownloadCache.Load(...) even when that digest is empty, causing different
unverified resources to collide on the "" key; change the logic in the reconcile
path so that before calling r.DownloadCache.Load you verify the digest is
present and non-empty (resource.Status.Resource.Digest.Value != "") and only
then use it as the cache key; if the digest is empty, bypass
r.DownloadCache.Load and call r.DownloadResourceWithOCM(ctx, deployer, resource)
directly (or alternatively use the resolver-provided stable cache key when
available) so unverified/skipped-resolution resources are not cached under a
shared empty key.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 0875cbbd-6c1a-46eb-bb37-98e73afce75a
📒 Files selected for processing (3)
kubernetes/controller/internal/controller/deployer/blob.gokubernetes/controller/internal/controller/deployer/blob_test.gokubernetes/controller/internal/controller/deployer/deployer_controller.go
Signed-off-by: Frederic Wilhelm <frederic.wilhelm@sap.com>
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
There was a problem hiding this comment.
🧹 Nitpick comments (2)
kubernetes/controller/internal/controller/deployer/deployer_controller.go (2)
535-547: Consider improving clarity and defensive checks in size limit enforcement.A few observations on this logic:
Integer overflow (minor):
r.MaxResourceSizeMiB * 1024 * 1024could theoretically overflow for extremely largeint64values. In practice, reasonable values (1-10000 MiB) won't overflow, but a defensive approach could use constants or check bounds.Variable naming:
limitedReaderis named before limiting is applied (line 521), which could be slightly confusing. Consider renaming toreaderinitially andlimitedReaderonly after wrapping.♻️ Optional: Clearer variable naming
- limitedReader, err := resourceBlob.ReadCloser() + reader, err := resourceBlob.ReadCloser() if err != nil { status.MarkNotReady(r.EventRecorder, deployer, deliveryv1alpha1.GetOCMResourceFailedReason, err.Error()) return nil, fmt.Errorf("getting reader for resource blob: %w", err) } + limitedReader := io.ReadCloser(reader) defer func() { err = errors.Join(err, limitedReader.Close()) }() // Enforce resource size limit... if r.MaxResourceSizeMiB > 0 { maxBytes := r.MaxResourceSizeMiB * 1024 * 1024 if sizeAware, ok := resourceBlob.(blob.SizeAware); ok { if size := sizeAware.Size(); size != blob.SizeUnknown && size > maxBytes { err := fmt.Errorf("resource size %d bytes exceeds maximum allowed size of %d MiB", size, r.MaxResourceSizeMiB) status.MarkNotReady(r.EventRecorder, deployer, deliveryv1alpha1.GetOCMResourceFailedReason, err.Error()) return nil, fmt.Errorf("failed to get limited reader: %w", err) } } - limitedReader = &limitedReadCloser{Closer: limitedReader, limited: &io.LimitedReader{R: limitedReader, N: maxBytes}} + limitedReader = &limitedReadCloser{Closer: reader, limited: &io.LimitedReader{R: reader, N: maxBytes}} }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@kubernetes/controller/internal/controller/deployer/deployer_controller.go` around lines 535 - 547, The size-enforcement block in deployer_controller.go can be made more defensive and clearer: in the MaxResourceSizeMiB logic (seen around r.MaxResourceSizeMiB usage and the limitedReadCloser wrapping), add a bounds check to detect and guard against overflow when computing maxBytes (e.g., validate r.MaxResourceSizeMiB is within a safe range or clamp it before multiplying), and rename the pre-existing generic limitedReader variable to reader (or srcReader) and only create/assign limitedReader when you actually wrap it with io.LimitedReader; ensure you use the new names consistently in the Size-aware check (sizeAware.Size()) and when constructing &limitedReadCloser so the code is clearer and avoids accidental misuse.
539-543: Misleading error message wrapper.The wrapping error message "failed to get limited reader" doesn't accurately describe the actual failure (resource size exceeding the limit). Consider a more accurate message.
♻️ Suggested fix for clearer error message
- return nil, fmt.Errorf("failed to get limited reader: %w", err) + return nil, errOr if you prefer to add context:
- return nil, fmt.Errorf("failed to get limited reader: %w", err) + return nil, fmt.Errorf("resource size check failed: %w", err)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@kubernetes/controller/internal/controller/deployer/deployer_controller.go` around lines 539 - 543, Replace the misleading wrapper error "failed to get limited reader" with a message that reflects the actual failure (resource size exceeds limit) in the error returned from the block around status.MarkNotReady and the return in deployer_controller.go; e.g., return a wrapped error like "resource size exceeds maximum allowed limit: %w" or include the original err when returning from the function that contains status.MarkNotReady, so callers see the accurate condition reported by err (related symbols: status.MarkNotReady, deployer, deliveryv1alpha1.GetOCMResourceFailedReason).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@kubernetes/controller/internal/controller/deployer/deployer_controller.go`:
- Around line 535-547: The size-enforcement block in deployer_controller.go can
be made more defensive and clearer: in the MaxResourceSizeMiB logic (seen around
r.MaxResourceSizeMiB usage and the limitedReadCloser wrapping), add a bounds
check to detect and guard against overflow when computing maxBytes (e.g.,
validate r.MaxResourceSizeMiB is within a safe range or clamp it before
multiplying), and rename the pre-existing generic limitedReader variable to
reader (or srcReader) and only create/assign limitedReader when you actually
wrap it with io.LimitedReader; ensure you use the new names consistently in the
Size-aware check (sizeAware.Size()) and when constructing &limitedReadCloser so
the code is clearer and avoids accidental misuse.
- Around line 539-543: Replace the misleading wrapper error "failed to get
limited reader" with a message that reflects the actual failure (resource size
exceeds limit) in the error returned from the block around status.MarkNotReady
and the return in deployer_controller.go; e.g., return a wrapped error like
"resource size exceeds maximum allowed limit: %w" or include the original err
when returning from the function that contains status.MarkNotReady, so callers
see the accurate condition reported by err (related symbols:
status.MarkNotReady, deployer, deliveryv1alpha1.GetOCMResourceFailedReason).
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: c76e9c3c-f0df-4558-8bc2-822ad18d6349
📒 Files selected for processing (3)
kubernetes/controller/internal/controller/deployer/deployer_controller.gokubernetes/controller/internal/controller/deployer/limited_reader.gokubernetes/controller/internal/controller/deployer/limited_reader_test.go
Signed-off-by: Frederic Wilhelm <frederic.wilhelm@sap.com>
jakobmoellerdev
left a comment
There was a problem hiding this comment.
just a nit, apart from that okay
Signed-off-by: Frederic Wilhelm <frederic.wilhelm@sap.com>
…omponent-model into add-verify-and-resource-limit
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
Signed-off-by: Frederic Wilhelm <frederic.wilhelm@sap.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@kubernetes/controller/internal/controller/deployer/deployer_controller.go`:
- Around line 335-336: Update the TODO comment that currently reads "This is
unoptimal because we cannot ensure that the resource status will hold a digest
(e.g. for unverified component versoion/skipped verification)." — fix the typos
by replacing "unoptimal" with "suboptimal" and "versoion" with "version" so the
comment reads "This is suboptimal because we cannot ensure that the resource
status will hold a digest (e.g. for unverified component version/skipped
verification)." Locate and edit the TODO comment string in
deployer_controller.go (the TODO block shown in the diff) and commit the
corrected comment.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 623fd97b-3aeb-4211-8c77-9153339e9954
📒 Files selected for processing (8)
kubernetes/controller/chart/README.mdkubernetes/controller/chart/values.schema.jsonkubernetes/controller/chart/values.yamlkubernetes/controller/cmd/main.gokubernetes/controller/internal/controller/deployer/deployer_controller.gokubernetes/controller/internal/controller/deployer/limited_reader.gokubernetes/controller/internal/controller/deployer/limited_reader_test.gokubernetes/controller/internal/controller/deployer/suite_test.go
✅ Files skipped from review due to trivial changes (3)
- kubernetes/controller/chart/values.schema.json
- kubernetes/controller/chart/README.md
- kubernetes/controller/internal/controller/deployer/limited_reader_test.go
🚧 Files skipped from review as they are similar to previous changes (4)
- kubernetes/controller/internal/controller/deployer/suite_test.go
- kubernetes/controller/chart/values.yaml
- kubernetes/controller/internal/controller/deployer/limited_reader.go
- kubernetes/controller/cmd/main.go
What this PR does / why we need it
This PR introduces as maximum resource size for the download cache in the deployer controller. The default maximum resource size is 2MiB which should be enough even for big manifest files. The maximum resource size is configurable as CLI argument for the controller manager and can be adjusted in the Helm chart as well.
Which issue(s) this PR fixes
Fixes open-component-model/ocm-project#963