Skip to content

Composite Tools Supports DefaultResults for Skippable Steps#3006

Merged
jerm-dro merged 17 commits intomainfrom
jerm/composite-tools-default-results
Dec 12, 2025
Merged

Composite Tools Supports DefaultResults for Skippable Steps#3006
jerm-dro merged 17 commits intomainfrom
jerm/composite-tools-default-results

Conversation

@jerm-dro
Copy link
Copy Markdown
Contributor

@jerm-dro jerm-dro commented Dec 11, 2025

Summary

Implements defaultResults for composite tool workflow steps, allowing skipped steps (due to conditions evaluating to false) or failed steps (with onError.action: continue) to provide fallback output values for downstream step templates.

Closes #2989

Large PR Justification

A significant chunk of this PR is tests and documentation. If reviewers would like, I could split the validation logic into a separate PR or I could remove it entirely, because validation is a nice-to-have.

Changes

Schema & Configuration

  • Added defaultResults field to WorkflowStep CRD type (map[string]runtime.RawExtension)
  • Added corresponding fields in raw config, internal config, and composer types
  • Updated all converters (CRD→config, YAML→config, config→composer)

Validation

  • Implemented shared validation in workflow_validation.go
  • Validates that defaultResults[field] is specified when:
    • Step may be skipped (has condition or onError.action: continue)
    • Downstream steps reference that specific output field
  • Validation is reused by both VirtualMCPServer and VirtualMCPCompositeToolDefinition webhooks

Template Reference Extraction

  • Created pkg/templates/references.go with AST-based template parsing
  • Uses text/template/parse package for reliable reference extraction
  • Extracts .steps.<stepID>.output.<field> references for validation

Runtime

  • Updated RecordStepSkipped to accept and store defaultResults as step output
  • Updated handleToolStepFailure to set defaultResults when continuing on error

Documentation

  • Added "Default Results" section to virtualmcpcompositetooldefinition-guide.md
  • Added quick reference to composite-tools-quick-reference.md
  • Added "Pattern 5: Default Results for Skippable Steps" to advanced-workflow-patterns.md

Example Usage

steps:
  - id: optional_enrichment
    type: tool
    tool: enrichment.api
    condition: "{{.params.enable_enrichment}}"
    arguments:
      data: "{{.params.input}}"
    defaultResults:
      text: "no enrichment performed"

  - id: process_result
    type: tool
    tool: processor.handle
    dependsOn: [optional_enrichment]
    arguments:
      # Works whether optional_enrichment ran or was skipped
      enriched_data: "{{.steps.optional_enrichment.output.text}}"

Testing

  • Unit tests: Template reference extraction, validation logic
  • Integration tests:
    • TestVMCPServer_DefaultResults_ConditionalSkip - verifies defaults used when condition=false
    • TestVMCPServer_DefaultResults_ContinueOnError - verifies defaults used when step fails with continue
  • E2E test: virtualmcp_composite_defaultresults_test.go - operator integration with conditional step

Notes

During implementation, discovered that nested access into backend tool results (e.g., {{.steps.step1.output.message}}) doesn't work as documented - backend tool calls store text content under a "text" key. See #2994 for more details.


Signed-off-by: Jeremy Drouillard <jeremy@stacklok.com>
Signed-off-by: Jeremy Drouillard <jeremy@stacklok.com>
Signed-off-by: Jeremy Drouillard <jeremy@stacklok.com>
Signed-off-by: Jeremy Drouillard <jeremy@stacklok.com>
Signed-off-by: Jeremy Drouillard <jeremy@stacklok.com>
@jerm-dro jerm-dro force-pushed the jerm/composite-tools-default-results branch from e4be512 to d87b79c Compare December 11, 2025 18:09
@github-actions github-actions bot added the size/XL Extra large PR: 1000+ lines changed label Dec 11, 2025
Copy link
Copy Markdown
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Large PR Detected

This PR exceeds 1000 lines of changes and requires justification before it can be reviewed.

How to unblock this PR:

Add a section to your PR description with the following format:

## Large PR Justification

[Explain why this PR must be large, such as:]
- Generated code that cannot be split
- Large refactoring that must be atomic
- Multiple related changes that would break if separated
- Migration or data transformation

Alternative:

Consider splitting this PR into smaller, focused changes (< 1000 lines each) for easier review and reduced risk.

See our Contributing Guidelines for more details.


This review will be automatically dismissed once you add the justification section.

@codecov
Copy link
Copy Markdown

codecov bot commented Dec 11, 2025

Codecov Report

❌ Patch coverage is 79.48718% with 40 lines in your changes missing coverage. Please review.
✅ Project coverage is 56.50%. Comparing base (02f494d) to head (afe6def).
⚠️ Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
...d/thv-operator/api/v1alpha1/workflow_validation.go 75.23% 15 Missing and 11 partials ⚠️
cmd/thv-operator/pkg/vmcpconfig/converter.go 0.00% 7 Missing and 1 partial ⚠️
pkg/templates/references.go 93.10% 3 Missing and 1 partial ⚠️
pkg/vmcp/composer/workflow_engine.go 33.33% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3006      +/-   ##
==========================================
+ Coverage   56.39%   56.50%   +0.11%     
==========================================
  Files         333      335       +2     
  Lines       33002    33183     +181     
==========================================
+ Hits        18612    18751     +139     
- Misses      12809    12838      +29     
- Partials     1581     1594      +13     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: Jeremy Drouillard <jeremy@stacklok.com>
Signed-off-by: Jeremy Drouillard <jeremy@stacklok.com>
Signed-off-by: Jeremy Drouillard <jeremy@stacklok.com>
@github-actions github-actions bot added size/XL Extra large PR: 1000+ lines changed and removed size/XL Extra large PR: 1000+ lines changed labels Dec 11, 2025
Signed-off-by: Jeremy Drouillard <jeremy@stacklok.com>
Signed-off-by: Jeremy Drouillard <jeremy@stacklok.com>
@github-actions github-actions bot added size/XL Extra large PR: 1000+ lines changed and removed size/XL Extra large PR: 1000+ lines changed labels Dec 11, 2025
Signed-off-by: Jeremy Drouillard <jeremy@stacklok.com>
@github-actions github-actions bot added size/XL Extra large PR: 1000+ lines changed and removed size/XL Extra large PR: 1000+ lines changed labels Dec 11, 2025
Signed-off-by: Jeremy Drouillard <jeremy@stacklok.com>
Signed-off-by: Jeremy Drouillard <jeremy@stacklok.com>
@github-actions github-actions bot added size/XL Extra large PR: 1000+ lines changed and removed size/XL Extra large PR: 1000+ lines changed labels Dec 11, 2025
@jerm-dro jerm-dro changed the title Jerm/composite tools default results Composite Tools Supports DefaultResults for Skippable Steps Dec 11, 2025
@jerm-dro jerm-dro marked this pull request as ready for review December 11, 2025 21:37
@jerm-dro jerm-dro requested a review from tgrunnagle December 11, 2025 21:37
@jerm-dro jerm-dro requested review from JAORMX, dmjb and jhrozek December 11, 2025 21:37
Copy link
Copy Markdown
Contributor

@jhrozek jhrozek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks really solid after a first quick read..but the PR is quite big, will have to go over it again tomorrow. Some nits and questions inline.

really nice work!

jhrozek
jhrozek previously approved these changes Dec 12, 2025
Copy link
Copy Markdown
Contributor

@jhrozek jhrozek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's clean the docs in a follow up but otherwise this is clean work, nice!

@github-actions github-actions bot added size/XL Extra large PR: 1000+ lines changed and removed size/XL Extra large PR: 1000+ lines changed labels Dec 12, 2025
Signed-off-by: Jeremy Drouillard <jeremy@stacklok.com>
@github-actions github-actions bot added size/XL Extra large PR: 1000+ lines changed and removed size/XL Extra large PR: 1000+ lines changed labels Dec 12, 2025
Signed-off-by: Jeremy Drouillard <jeremy@stacklok.com>
@github-actions github-actions bot added size/XL Extra large PR: 1000+ lines changed and removed size/XL Extra large PR: 1000+ lines changed labels Dec 12, 2025
@jerm-dro jerm-dro merged commit 00474d4 into main Dec 12, 2025
38 of 39 checks passed
@jerm-dro jerm-dro deleted the jerm/composite-tools-default-results branch December 12, 2025 16:26
@jerm-dro jerm-dro mentioned this pull request Dec 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/XL Extra large PR: 1000+ lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Composite Tools Supports Default Return Values

2 participants