Skip to content

feat!: drop keys with nil values when not required by scheme spec#1889

Merged
matthiasbruns merged 18 commits into
open-component-model:mainfrom
matthiasbruns:feat/null_drop_transform
Mar 6, 2026
Merged

feat!: drop keys with nil values when not required by scheme spec#1889
matthiasbruns merged 18 commits into
open-component-model:mainfrom
matthiasbruns:feat/null_drop_transform

Conversation

@matthiasbruns

@matthiasbruns matthiasbruns commented Mar 3, 2026

Copy link
Copy Markdown
Contributor

What this PR does / why we need it

This PR solves the issue that happens when you want to pass nil values for optional pointers through the transformation steps.

If in a spec a type is annotated by omitempty and resolved to nil

one would suspect that nil is a valid value. The problem is, that our schemagen does not generate a rule that allows null to be passed in such cases.
Since we have no way to omitting keys based on the fact that values exist or not, we have to circumvent that in the transformer graph.

This PR does that.

Here is an example of what we are trying to solve with this change:

convertToOCITransform := transformv1alpha1.GenericTransformation{
	TransformationMeta: meta.TransformationMeta{
		Type: ociv1alpha1.ConvertHelmToOCIV1alpha1,
		ID:   convertResourceID,
	},
	Spec: &runtime.Unstructured{Data: map[string]any{
		"resource":  fmt.Sprintf("${%s.output.resource}", getResourceID),
		"chartFile": fmt.Sprintf("${%s.output.chartFile}", getResourceID),
		"provFile":  fmt.Sprintf("${%[1]s.output.?provFile}", getResourceID),
	}},
}

provFile might be null, if not provided by the helm chart. We can only run checks on the values with CEL here. If we pass nil to the provFile json field, the schemavalidation will fail.

#1846

Which issue(s) this PR fixes

Contributes: open-component-model/ocm-project#883

Testing

How to test the changes

Fully tested in https://github.com/open-component-model/open-component-model/pull/1846/changes#diff-85d55581ce565d132e8a8b027169c162a024c7e25f6df4c5cc2ddbbfb1294d2fR356

Verification
  • I have tested the changes locally by running ocm

Summary by CodeRabbit

  • New Features

    • Schema-driven resolution: resolvers can accept optional schemas (including spec sub-schemas) to detect optional fields and guide processing.
  • Bug Fixes

    • Optional-field removal: optional fields that resolve to nil are removed from resources (no null retained).
    • Runtime processing no longer treats optional nil values as errors.
  • Tests

    • Expanded coverage for nil handling, nested/referenced schemas, array items, and targeted deletions.

On-behalf-of: SAP <matthias.bruns@sap.com>
Signed-off-by: Matthias Bruns <git@matthiasbruns.com>
@github-actions github-actions Bot added kind/feature new feature, enhancement, improvement, extension size/m Medium labels Mar 3, 2026
@matthiasbruns matthiasbruns marked this pull request as ready for review March 3, 2026 11:37
@matthiasbruns matthiasbruns requested a review from a team as a code owner March 3, 2026 11:37
matthiasbruns and others added 2 commits March 3, 2026 12:37
On-behalf-of: SAP <matthias.bruns@sap.com>
Signed-off-by: Matthias Bruns <git@matthiasbruns.com>
@github-actions github-actions Bot added the size/l Large label Mar 3, 2026
On-behalf-of: SAP <matthias.bruns@sap.com>
Signed-off-by: Matthias Bruns <git@matthiasbruns.com>
@matthiasbruns matthiasbruns force-pushed the feat/null_drop_transform branch from c93c79b to edc67c0 Compare March 3, 2026 13:13
Signed-off-by: Matthias Bruns <git@matthiasbruns.com>
@matthiasbruns matthiasbruns force-pushed the feat/null_drop_transform branch from e238ca4 to 517e6ef Compare March 3, 2026 13:18
On-behalf-of: SAP <matthias.bruns@sap.com>
Signed-off-by: Matthias Bruns <git@matthiasbruns.com>
On-behalf-of: SAP <matthias.bruns@sap.com>
Signed-off-by: Matthias Bruns <git@matthiasbruns.com>

@Skarlso Skarlso left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of questions and remarks.

Comment thread bindings/go/transform/graph/runtime/resolver/resolver.go
Comment thread bindings/go/transform/graph/runtime/resolver/resolver.go Outdated
Comment thread bindings/go/transform/graph/runtime/resolver/resolver.go Outdated
Comment thread bindings/go/transform/graph/runtime/resolver/resolver.go Outdated
Comment thread bindings/go/transform/graph/runtime/resolver/resolver.go
@matthiasbruns matthiasbruns changed the title feat: drop keys with nil values when not required by scheme spec feat!: drop keys with nil values when not required by scheme spec Mar 3, 2026
@matthiasbruns matthiasbruns added the !BREAKING-CHANGE! Breaking change in API or ocm-cli or spec label Mar 3, 2026
matthiasbruns and others added 4 commits March 4, 2026 08:17
On-behalf-of: SAP <matthias.bruns@sap.com>
Signed-off-by: Matthias Bruns <git@matthiasbruns.com>
On-behalf-of: SAP <matthias.bruns@sap.com>
Signed-off-by: Matthias Bruns <git@matthiasbruns.com>
…ctions

On-behalf-of: SAP <matthias.bruns@sap.com>
Signed-off-by: Matthias Bruns <git@matthiasbruns.com>
@coderabbitai

coderabbitai Bot commented Mar 4, 2026

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • ✅ Review completed - (🔄 Check again to review again)
📝 Walkthrough

Walkthrough

Resolver now accepts an optional jsonschema.Schema (spec-level) and uses it to detect optional fields; nil values for optional fields are deleted from resources instead of being written. Deletions are tracked on ResolutionResult/ResolutionSummary, tests updated, and runtime now passes the spec sub-schema to the resolver.

Changes

Cohort / File(s) Summary
Resolver implementation
bindings/go/transform/graph/runtime/resolver/resolver.go
Add schema *jsonschema.Schema to Resolver; new helpers (isOptionalField, schemaProperty, isRequired, schemaItems); implement deleteValueAtPath; mark ResolutionResult.Deleted and increment ResolutionSummary.DeletedExpressions; change NewResolver signature to accept schema.
Resolver tests
bindings/go/transform/graph/runtime/resolver/resolver_test.go
Update NewResolver calls to include schema (mostly nil), add extensive tests for nil vs spec-driven behavior (nested $refs, arrays/items, optional vs required), add TestDeleteValueAtPath and TestIsOptionalField_NestedRefs.
Runtime integration
bindings/go/transform/graph/runtime/runtime.go
Extract spec sub-schema via specSubSchema(transformation.Schema) and pass it into NewResolver(..., spec); adjust constructor call sites to new signature.
Builder test tweak
bindings/go/transform/graph/builder/builder_test.go
Normalize YAML object: field formatting in a test and change one assertion from expecting an error to expecting no runtime processing error.
Modules / manifests
go.mod, manifest*
Add dependency github.com/santhosh-tekuri/jsonschema/v6 and update manifests/imports to support schema-driven logic.

Sequence Diagram(s)

sequenceDiagram
  participant Runtime as Runtime
  participant Resolver as Resolver
  participant Schema as jsonschema.Schema
  participant Resource as Resource (map[string]interface{})
  Runtime->>Resolver: NewResolver(resource, data, specSubSchema)
  Resolver->>Schema: Inspect path/schema (schemaProperty / isOptionalField / schemaItems)
  Resolver->>Resolver: Resolve expression -> value
  alt Field is optional and resolved value == nil
    Resolver->>Resource: deleteValueAtPath(path)
    Resolver-->>Runtime: ResolutionResult{Deleted:true}
    Resolver->>Resolver: increment DeletedExpressions
  else Field is required or value non-nil
    Resolver->>Resource: write resolved value
    Resolver-->>Runtime: ResolutionResult{Deleted:false}
  end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

"I hopped through schemas, sniffed optional trees,
Nibbled nils soft so nothing would freeze.
When fields were optional I munched them away,
Leaving maps tidy for a bright new day.
— 🐇"

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 31.25% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: dropping nil-valued keys when not required by schema spec, which aligns with the core objective of removing optional fields with null values during transformation.
Linked Issues check ✅ Passed The PR addresses schema validation failures from nil values in optional fields during helm chart transformations by implementing schema-driven deletion logic, supporting the broader helm chart transformer implementation in issue #883.
Out of Scope Changes check ✅ Passed All changes focus on resolver schema integration and nil value deletion for optional fields, which is directly necessary for the transformer to handle optional fields correctly during helm chart conversions.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
bindings/go/transform/graph/builder/builder_test.go (1)

160-177: Consider asserting the resulting object shape, not just NoError

For this case, it would be valuable to also verify that version is actually absent in the processed output, so the test checks the semantic goal (drop optional nil field), not only execution success.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@bindings/go/transform/graph/builder/builder_test.go` around lines 160 - 177,
The test currently only asserts staticAnalysisErr and runtimeProcessingErr are
NoError; add an assertion that the processed output for the second
transformation does not contain a version field (i.e. the optional cel reference
${add1.spec.object.?version} produced no value). After the processing step that
yields the result, locate the resolved object for "add2" (or the structure
populated from transformationSpec) and assert that add2.spec.object.version is
absent/nil (use the existing test helpers like require.Nil/require.NotContains
or equivalent) so the test verifies the semantic goal of dropping the optional
nil field.
bindings/go/transform/graph/runtime/resolver/resolver_test.go (1)

634-844: Please add one indexed-path optional deletion test

The new optional-field coverage is solid, but it would help to add a case like items[0].optionalField resolving to nil to lock behavior for array-nested optional fields.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@bindings/go/transform/graph/runtime/resolver/resolver_test.go` around lines
634 - 844, Add a new t.Run inside TestResolveField_NilOptionalPointer_SpecSchema
that exercises an indexed-path optional deletion: create a resource with an
array property (e.g., "items": []interface{}{
map[string]interface{}{"optionalField": "${optExpr}", "other": "keep"} }), set
data map with "optExpr": nil, call r := NewResolver(resource, data, specSchema)
and result := r.resolveField(variable.FieldDescriptor{Path:
fieldpath.MustParse("items[0].optionalField"), Expressions:
[]variable.Expression{{Value: "optExpr"}}, StandaloneExpression: true}), then
require.NoError(result.Error), require.True(result.Resolved),
require.True(result.Deleted) and assert the first item no longer contains
"optionalField" but still contains "other"; use the same specSchema and
jsonschema refs as the surrounding tests and mirror assertions style used in
other t.Run cases.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@bindings/go/transform/graph/builder/builder_test.go`:
- Around line 160-177: The test currently only asserts staticAnalysisErr and
runtimeProcessingErr are NoError; add an assertion that the processed output for
the second transformation does not contain a version field (i.e. the optional
cel reference ${add1.spec.object.?version} produced no value). After the
processing step that yields the result, locate the resolved object for "add2"
(or the structure populated from transformationSpec) and assert that
add2.spec.object.version is absent/nil (use the existing test helpers like
require.Nil/require.NotContains or equivalent) so the test verifies the semantic
goal of dropping the optional nil field.

In `@bindings/go/transform/graph/runtime/resolver/resolver_test.go`:
- Around line 634-844: Add a new t.Run inside
TestResolveField_NilOptionalPointer_SpecSchema that exercises an indexed-path
optional deletion: create a resource with an array property (e.g., "items":
[]interface{}{ map[string]interface{}{"optionalField": "${optExpr}", "other":
"keep"} }), set data map with "optExpr": nil, call r := NewResolver(resource,
data, specSchema) and result := r.resolveField(variable.FieldDescriptor{Path:
fieldpath.MustParse("items[0].optionalField"), Expressions:
[]variable.Expression{{Value: "optExpr"}}, StandaloneExpression: true}), then
require.NoError(result.Error), require.True(result.Resolved),
require.True(result.Deleted) and assert the first item no longer contains
"optionalField" but still contains "other"; use the same specSchema and
jsonschema refs as the surrounding tests and mirror assertions style used in
other t.Run cases.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 583b021f-d34e-4da1-81d2-e375a8ce3d6e

📥 Commits

Reviewing files that changed from the base of the PR and between c52ee88 and 5dacb3e.

📒 Files selected for processing (4)
  • bindings/go/transform/graph/builder/builder_test.go
  • bindings/go/transform/graph/runtime/resolver/resolver.go
  • bindings/go/transform/graph/runtime/resolver/resolver_test.go
  • bindings/go/transform/graph/runtime/runtime.go

Comment thread bindings/go/transform/graph/runtime/resolver/resolver.go
matthiasbruns and others added 3 commits March 4, 2026 14:11
@matthiasbruns matthiasbruns force-pushed the feat/null_drop_transform branch from c21318f to b58f60e Compare March 4, 2026 14:49

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
bindings/go/transform/graph/runtime/resolver/resolver_test.go (1)

891-915: Add an array-item $ref regression case to harden isOptionalField.

The array test currently covers Items2020 with inline object schema, but not Items2020 pointing to a $ref schema. Adding that case will lock behavior for common generated-schema patterns.

Proposed test addition
 func TestIsOptionalField_NestedRefs(t *testing.T) {
@@
 	rArr := NewResolver(nil, nil, arrayTopSchema)

 	assert.False(t, rArr.isOptionalField(fieldpath.MustParse("servers[0].host")),
 		"host is required inside array item (Items2020)")
 	assert.True(t, rArr.isOptionalField(fieldpath.MustParse("servers[0].timeout")),
 		"timeout is optional inside array item (Items2020)")
 	assert.True(t, rArr.isOptionalField(fieldpath.MustParse("servers[2].timeout")),
 		"timeout is optional regardless of array index")
+
+	// array items via $ref
+	arrayTopSchemaWithRefItems := &jsonschema.Schema{
+		Properties: map[string]*jsonschema.Schema{
+			"servers": {Items2020: &jsonschema.Schema{Ref: itemSchema}},
+		},
+		Required: []string{"servers"},
+	}
+	rArrRef := NewResolver(nil, nil, arrayTopSchemaWithRefItems)
+	assert.False(t, rArrRef.isOptionalField(fieldpath.MustParse("servers[0].host")),
+		"host is required inside array item ($ref)")
+	assert.True(t, rArrRef.isOptionalField(fieldpath.MustParse("servers[0].timeout")),
+		"timeout is optional inside array item ($ref)")
 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@bindings/go/transform/graph/runtime/resolver/resolver_test.go` around lines
891 - 915, Add a regression test that mirrors the existing array Items2020 case
but where the array item schema is a $ref instead of an inline object: create a
referenced schema (e.g., "Server" schema with Required ["host"]) and have
arrayTopSchema.Properties["servers"].Items2020 point to a schema with Ref set to
that reference; then construct the resolver with NewResolver(nil, nil,
arrayTopSchema) and assert via
rArr.isOptionalField(fieldpath.MustParse("servers[0].host")) and
rArr.isOptionalField(fieldpath.MustParse("servers[0].timeout")) (and an index
like "servers[2].timeout") to ensure required/optional behavior matches the
inline Items2020 case.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@bindings/go/transform/graph/runtime/resolver/resolver.go`:
- Around line 333-339: Array item branch in Resolve traversal sets current =
itemSchema (from schemaItems(current)) without dereferencing a possible Ref,
unlike the property branch that checks propSchema.Ref; update the array-item
handling in the resolver (the segment.Index branch that uses schemaItems and
itemSchema) to detect and follow itemSchema.Ref (similar to how propSchema.Ref
is handled) before assigning/continuing traversal so array items are
consistently dereferenced.

---

Nitpick comments:
In `@bindings/go/transform/graph/runtime/resolver/resolver_test.go`:
- Around line 891-915: Add a regression test that mirrors the existing array
Items2020 case but where the array item schema is a $ref instead of an inline
object: create a referenced schema (e.g., "Server" schema with Required
["host"]) and have arrayTopSchema.Properties["servers"].Items2020 point to a
schema with Ref set to that reference; then construct the resolver with
NewResolver(nil, nil, arrayTopSchema) and assert via
rArr.isOptionalField(fieldpath.MustParse("servers[0].host")) and
rArr.isOptionalField(fieldpath.MustParse("servers[0].timeout")) (and an index
like "servers[2].timeout") to ensure required/optional behavior matches the
inline Items2020 case.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 340223bb-7a32-430a-9367-11e64055e76a

📥 Commits

Reviewing files that changed from the base of the PR and between 5dacb3e and c21318f.

📒 Files selected for processing (2)
  • bindings/go/transform/graph/runtime/resolver/resolver.go
  • bindings/go/transform/graph/runtime/resolver/resolver_test.go

Comment thread bindings/go/transform/graph/runtime/resolver/resolver.go

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
bindings/go/transform/graph/runtime/resolver/resolver_test.go (1)

891-915: Consider adding a test case for array items with Ref.

The current array tests use Items2020 pointing directly to a schema with properties. Consider adding a test case where Items2020 itself contains a Ref (e.g., {Items2020: &jsonschema.Schema{Ref: itemSchema}}) to ensure consistent traversal behavior with the property-ref pattern tested elsewhere.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@bindings/go/transform/graph/runtime/resolver/resolver_test.go` around lines
891 - 915, Add a new test variant that mirrors the existing array item checks
but sets Items2020 to a schema containing a Ref instead of inline properties:
create itemSchema (same as existing), then create arrayTopSchema where "servers"
has Items2020: &jsonschema.Schema{Ref: itemSchema} and instantiate resolver via
NewResolver(nil, nil, arrayTopSchema); assert using
rArr.isOptionalField(fieldpath.MustParse("servers[0].host")) and
"servers[0].timeout" (and a different index like "servers[2].timeout") to verify
required/optional behavior matches the inline-Items2020 case. Ensure you
reference the same symbols (itemSchema, arrayTopSchema, Items2020, Ref,
NewResolver, isOptionalField, fieldpath.MustParse) so traversal with a Ref is
covered.
bindings/go/transform/graph/runtime/resolver/resolver.go (1)

396-403: Final segment handling assumes map key without validation.

The code at line 402 uses segment.Name unconditionally, but doesn't verify that the final segment is indeed a map key (not an array index). While current usage via resolveField ensures this is always true (since isOptionalField returns false for array indices), the function signature doesn't enforce this constraint.

Consider adding a guard for robustness:

Defensive check
 if i == len(path)-1 {
-    // final segment: always a map key deletion
+    // final segment: only map key deletion is supported
+    if segment.Index != nil {
+        return fmt.Errorf("cannot delete array element at path segment: %v", segment)
+    }
     currentMap, ok := current.(map[string]interface{})
     if !ok {
         return fmt.Errorf("expected map at path segment: %v", segment)
     }
     delete(currentMap, segment.Name)
     return nil
 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@bindings/go/transform/graph/runtime/resolver/resolver.go` around lines 396 -
403, The final-segment deletion assumes the segment is a map key and uses
segment.Name unconditionally; add a defensive check before treating it as a map
key (e.g., verify the segment represents a key rather than an array index —
using whatever discriminator the Segment type exposes, such as a boolean like
IsIndex or by checking an empty Name) and return a clear error if it's not
valid; update the block in the resolver function where currentMap is asserted
(the same code path referenced by resolveField and isOptionalField) to validate
the segment kind before calling delete(currentMap, segment.Name).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@bindings/go/transform/graph/runtime/resolver/resolver_test.go`:
- Around line 891-915: Add a new test variant that mirrors the existing array
item checks but sets Items2020 to a schema containing a Ref instead of inline
properties: create itemSchema (same as existing), then create arrayTopSchema
where "servers" has Items2020: &jsonschema.Schema{Ref: itemSchema} and
instantiate resolver via NewResolver(nil, nil, arrayTopSchema); assert using
rArr.isOptionalField(fieldpath.MustParse("servers[0].host")) and
"servers[0].timeout" (and a different index like "servers[2].timeout") to verify
required/optional behavior matches the inline-Items2020 case. Ensure you
reference the same symbols (itemSchema, arrayTopSchema, Items2020, Ref,
NewResolver, isOptionalField, fieldpath.MustParse) so traversal with a Ref is
covered.

In `@bindings/go/transform/graph/runtime/resolver/resolver.go`:
- Around line 396-403: The final-segment deletion assumes the segment is a map
key and uses segment.Name unconditionally; add a defensive check before treating
it as a map key (e.g., verify the segment represents a key rather than an array
index — using whatever discriminator the Segment type exposes, such as a boolean
like IsIndex or by checking an empty Name) and return a clear error if it's not
valid; update the block in the resolver function where currentMap is asserted
(the same code path referenced by resolveField and isOptionalField) to validate
the segment kind before calling delete(currentMap, segment.Name).

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: ce33f583-cded-45b7-ab50-ccfa1180e121

📥 Commits

Reviewing files that changed from the base of the PR and between c21318f and b58f60e.

📒 Files selected for processing (2)
  • bindings/go/transform/graph/runtime/resolver/resolver.go
  • bindings/go/transform/graph/runtime/resolver/resolver_test.go

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@bindings/go/transform/graph/runtime/resolver/resolver.go`:
- Around line 380-390: schemaItems currently ignores tuple-style Items (slice
[]*jsonschema.Schema), causing nil returns during indexed array traversal in the
resolver; update schemaItems to detect when s.Items is a []*jsonschema.Schema
and return the first element (to match the wrapper Schema.Items() behavior) or
the indexed element when available during traversal, so optional-field detection
at the resolver's indexed array handling (around where schemaItems is used at
the indexed traversal) no longer gets nil; ensure you check for non-empty slice
before returning.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: f785ff12-1a1b-43da-bfe3-3c35f83bc659

📥 Commits

Reviewing files that changed from the base of the PR and between b58f60e and 930257e.

📒 Files selected for processing (2)
  • bindings/go/transform/graph/runtime/resolver/resolver.go
  • bindings/go/transform/graph/runtime/resolver/resolver_test.go

Comment thread bindings/go/transform/graph/runtime/resolver/resolver.go Outdated
@matthiasbruns matthiasbruns force-pushed the feat/null_drop_transform branch from 930257e to c01b1f6 Compare March 4, 2026 19:33
On-behalf-of: SAP <matthias.bruns@sap.com>
Signed-off-by: Matthias Bruns <git@matthiasbruns.com>
@matthiasbruns matthiasbruns force-pushed the feat/null_drop_transform branch from c01b1f6 to 62a2254 Compare March 4, 2026 19:34

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
bindings/go/transform/graph/runtime/resolver/resolver.go (1)

323-360: Add defensive dereferencing for root-level schema refs in isOptionalField.

The current implementation doesn't handle cases where the passed schema is itself a $ref wrapper (particularly in Ref chains). While specSubSchema dereferences one level, if the resulting target itself has a Ref, traversal begins without dereferencing it, causing schemaProperty to fail to find properties. Add the derefSchema helper function and apply it at the root and all intermediate levels as proposed to handle Ref chains defensively.

♻️ Proposed hardening
+func derefSchema(s *jsonschema.Schema) *jsonschema.Schema {
+	for s != nil && s.Ref != nil {
+		s = s.Ref
+	}
+	return s
+}
+
 func (r *Resolver) isOptionalField(path fieldpath.Path) bool {
 	if r.schema == nil || len(path) == 0 {
 		return false
 	}
-	current := r.schema
+	current := derefSchema(r.schema)
 	// Walk the segments and either check a property or the ref schema until we get to the last segment, where we check if it's required or not.
 	for i, segment := range path {
+		current = derefSchema(current)
 		if current == nil {
 			return false
 		}
 		if segment.Index != nil {
 			itemSchema := current.Items2020
 			if itemSchema == nil {
 				return false
 			}
-			if itemSchema.Ref != nil {
-				current = itemSchema.Ref
-			} else {
-				current = itemSchema
-			}
+			current = derefSchema(itemSchema)
 			continue
 		}
 		propSchema := schemaProperty(current, segment.Name)
 		if propSchema == nil {
 			return false
 		}
@@ 
-		if propSchema.Ref != nil {
-			// if we have a ref, we need to resolve it before continuing to the next segment
-			current = propSchema.Ref
-		} else {
-			current = propSchema
-		}
+		current = derefSchema(propSchema)
 	}
 	return false
 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@bindings/go/transform/graph/runtime/resolver/resolver.go` around lines 323 -
360, The isOptionalField implementation must defensively dereference $ref
wrapper schemas to avoid failing property lookups when the root or intermediate
target is a Ref; add and use a helper derefSchema (or reuse specSubSchema
behavior) to replace current = propSchema.Ref / itemSchema.Ref / r.schema with
derefSchema(current) at the start and whenever you follow a Ref (including the
root r.schema and after resolving itemSchema.Ref or propSchema.Ref) so
schemaProperty always receives the actual target schema rather than a Ref
wrapper; update isOptionalField to call derefSchema on r.schema initially and
after each Ref resolution before continuing traversal.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@bindings/go/transform/graph/runtime/resolver/resolver.go`:
- Around line 323-360: The isOptionalField implementation must defensively
dereference $ref wrapper schemas to avoid failing property lookups when the root
or intermediate target is a Ref; add and use a helper derefSchema (or reuse
specSubSchema behavior) to replace current = propSchema.Ref / itemSchema.Ref /
r.schema with derefSchema(current) at the start and whenever you follow a Ref
(including the root r.schema and after resolving itemSchema.Ref or
propSchema.Ref) so schemaProperty always receives the actual target schema
rather than a Ref wrapper; update isOptionalField to call derefSchema on
r.schema initially and after each Ref resolution before continuing traversal.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: c8b2d120-4575-472b-a7ba-16d5fd7dc57d

📥 Commits

Reviewing files that changed from the base of the PR and between 930257e and 62a2254.

📒 Files selected for processing (2)
  • bindings/go/transform/graph/runtime/resolver/resolver.go
  • bindings/go/transform/graph/runtime/resolver/resolver_test.go

jakobmoellerdev
jakobmoellerdev previously approved these changes Mar 5, 2026

@jakobmoellerdev jakobmoellerdev left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well done

fabianburth
fabianburth previously approved these changes Mar 6, 2026
Comment thread bindings/go/transform/graph/runtime/resolver/resolver.go Outdated
Co-authored-by: Fabian Burth <fabian.burth@sap.com>
Signed-off-by: Matthias Bruns <github@matthiasbruns.com>
@matthiasbruns matthiasbruns enabled auto-merge (squash) March 6, 2026 15:37
@matthiasbruns matthiasbruns merged commit ed40e61 into open-component-model:main Mar 6, 2026
16 checks passed
@matthiasbruns matthiasbruns linked an issue Mar 10, 2026 that may be closed by this pull request
8 tasks
frewilhelm pushed a commit to frewilhelm/open-component-model that referenced this pull request Mar 12, 2026
…en-component-model#1889)

#### What this PR does / why we need it
This PR solves the issue that happens when you want to pass nil values
for optional pointers through the transformation steps.

If in a spec a type is annotated by `omitempty` and resolved to `nil`

one would suspect that `nil` is a valid value. The problem is, that our
schemagen does not generate a rule that allows `null` to be passed in
such cases.
Since we have no way to omitting keys based on the fact that values
exist or not, we have to circumvent that in the transformer graph.

This PR does that.

Here is an example of what we are trying to solve with this change:


```go
convertToOCITransform := transformv1alpha1.GenericTransformation{
	TransformationMeta: meta.TransformationMeta{
		Type: ociv1alpha1.ConvertHelmToOCIV1alpha1,
		ID:   convertResourceID,
	},
	Spec: &runtime.Unstructured{Data: map[string]any{
		"resource":  fmt.Sprintf("${%s.output.resource}", getResourceID),
		"chartFile": fmt.Sprintf("${%s.output.chartFile}", getResourceID),
		"provFile":  fmt.Sprintf("${%[1]s.output.?provFile}", getResourceID),
	}},
}
```

`provFile` might be null, if not provided by the helm chart. We can only
run checks on the values with CEL here. If we pass `nil` to the
`provFile` json field, the schemavalidation will fail.

open-component-model#1846

#### Which issue(s) this PR fixes
Contributes:
open-component-model/ocm-project#883

#### Testing

##### How to test the changes

Fully tested in
https://github.com/open-component-model/open-component-model/pull/1846/changes#diff-85d55581ce565d132e8a8b027169c162a024c7e25f6df4c5cc2ddbbfb1294d2fR356

##### Verification

- [x] I have tested the changes locally by running `ocm`
  

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Schema-driven resolution: resolvers can accept optional schemas
(including spec sub-schemas) to detect optional fields and guide
processing.

* **Bug Fixes**
* Optional-field removal: optional fields that resolve to nil are
removed from resources (no null retained).
  * Runtime processing no longer treats optional nil values as errors.

* **Tests**
* Expanded coverage for nil handling, nested/referenced schemas, array
items, and targeted deletions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Matthias Bruns <git@matthiasbruns.com>
Signed-off-by: Matthias Bruns <github@matthiasbruns.com>
Co-authored-by: Fabian Burth <fabian.burth@sap.com>
morri-son pushed a commit to morri-son/open-component-model that referenced this pull request Mar 18, 2026
…en-component-model#1889)

#### What this PR does / why we need it
This PR solves the issue that happens when you want to pass nil values
for optional pointers through the transformation steps.

If in a spec a type is annotated by `omitempty` and resolved to `nil`

one would suspect that `nil` is a valid value. The problem is, that our
schemagen does not generate a rule that allows `null` to be passed in
such cases.
Since we have no way to omitting keys based on the fact that values
exist or not, we have to circumvent that in the transformer graph.

This PR does that.

Here is an example of what we are trying to solve with this change:

```go
convertToOCITransform := transformv1alpha1.GenericTransformation{
	TransformationMeta: meta.TransformationMeta{
		Type: ociv1alpha1.ConvertHelmToOCIV1alpha1,
		ID:   convertResourceID,
	},
	Spec: &runtime.Unstructured{Data: map[string]any{
		"resource":  fmt.Sprintf("${%s.output.resource}", getResourceID),
		"chartFile": fmt.Sprintf("${%s.output.chartFile}", getResourceID),
		"provFile":  fmt.Sprintf("${%[1]s.output.?provFile}", getResourceID),
	}},
}
```

`provFile` might be null, if not provided by the helm chart. We can only
run checks on the values with CEL here. If we pass `nil` to the
`provFile` json field, the schemavalidation will fail.

open-component-model#1846

#### Which issue(s) this PR fixes
Contributes:
open-component-model/ocm-project#883

#### Testing

##### How to test the changes

Fully tested in
https://github.com/open-component-model/open-component-model/pull/1846/changes#diff-85d55581ce565d132e8a8b027169c162a024c7e25f6df4c5cc2ddbbfb1294d2fR356

##### Verification

- [x] I have tested the changes locally by running `ocm`

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Schema-driven resolution: resolvers can accept optional schemas
(including spec sub-schemas) to detect optional fields and guide
processing.

* **Bug Fixes**
* Optional-field removal: optional fields that resolve to nil are
removed from resources (no null retained).
  * Runtime processing no longer treats optional nil values as errors.

* **Tests**
* Expanded coverage for nil handling, nested/referenced schemas, array
items, and targeted deletions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Matthias Bruns <git@matthiasbruns.com>
Signed-off-by: Matthias Bruns <github@matthiasbruns.com>
Co-authored-by: Fabian Burth <fabian.burth@sap.com>
Signed-off-by: Gerald Morrison (SAP) <gerald.morrison@sap.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

!BREAKING-CHANGE! Breaking change in API or ocm-cli or spec kind/feature new feature, enhancement, improvement, extension size/l Large size/m Medium

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement helm chart download transformer

4 participants