Skip to content

Add workflow harness integration tests #326

@marccampbell

Description

@marccampbell

Problem

Workflow support now has enough behavior that unit tests and full E2E tests leave a gap. Unit tests prove parsing or isolated helpers, and E2E proves the whole stack with real providers, but neither gives us a fast deterministic way to verify scripted workflow transitions end to end.

We need an integration test layer that runs the real workflow engine while mocking only the outside world.

Proposal

Add a workflow harness integration suite that:

  • loads prescripted workflow YAML fixtures from testdata, for example testdata/workflows/*.yaml
  • uses the real pipeline parser, DB, stage matching, transition code, runOnEnter behavior, and dedupe/claiming logic
  • creates fake agent rows with workspace/workflow/issue context
  • feeds scripted agent messages through the same server path used by bridge messages
  • uses fake provider exec and fake issue tracker clients for external effects
  • asserts final stage, injected messages, command execution, issue tracker actions, terminal behavior, and no duplicate transitions

The key boundary: do not mock the workflow engine. Mock only provider exec and external APIs.

Suggested Fixtures

github-issue-precommit.yaml

Entry stage injects work instructions. The fake agent replies [READY_TO_COMMIT]. The pre-commit stage runs a command, injects the Clawpatch review instructions, then the fake agent replies [DONE]. Assert the workflow reaches the review stage and records the expected command.

run-fails-stop.yaml

A run action exits non-zero with continue_on_error omitted/false. Assert the failure is visible and later on_enter actions do not run.

run-fails-continue.yaml

A run action exits non-zero with continue_on_error: true. Assert a warning is inserted and the following inject still runs.

linear-move-issue.yaml

A message transition emits move_issue with rendered issue context such as {{.Issue.Identifier}}. Assert the fake Linear client receives the expected move request.

terminal-stage.yaml

A terminal transition marks the agent deleted and requests VM termination once. Assert no duplicate termination is requested if the trigger/message is replayed.

Why This Matters

This would catch bugs that are easy to miss today:

  • workflow syntax parses but transitions do not fire
  • legacy factory assumptions leak into workflow execution
  • duplicate stage transitions create duplicate actions or agents
  • run actions block or reorder subsequent injects incorrectly
  • issue tracker actions render the wrong issue context
  • terminal stages fail to clean up or clean up more than once

This suite should be fast enough to run on every PR, while the real provider E2E tests continue to validate Daytona/Replicated/exe.dev and external webhook behavior.

Metadata

Metadata

Assignees

No one assigned

    Labels

    doneThe issue is complete

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions