Skip to content

Add E2E smoke test workflows to CI #1254

@coleam00

Description

@coleam00

Summary

Add real E2E smoke tests that execute Archon workflows in GitHub Actions CI, verifying that the full workflow engine works end-to-end with both Claude and Codex providers.

Motivation

Current CI only runs unit/integration tests with mocks and a Docker health check. There's no verification that actual workflow execution works — provider selection, structured output, tool use, skills, MCP, script nodes, etc. A regression in any of these would only be caught manually.

Scope

New GitHub Actions workflow (.github/workflows/e2e-smoke.yml)

  • Triggers on push to main/dev only (no PR trigger to avoid API cost abuse)
  • 4 parallel jobs: deterministic, claude, codex, mixed-providers
  • Installs Claude CLI, Codex CLI, uv for Python scripts

E2E workflow files

  • e2e-deterministic.yaml — bash, script (bun), script (uv), conditions, trigger rules (0 API calls)
  • e2e-claude-smoke.yaml — enhanced with command node, model overrides
  • e2e-codex-smoke.yaml — with explicit model config
  • e2e-mixed-providers.yaml — cross-provider output refs
  • e2e-all-nodes.yaml — fix broken script node
  • e2e-skills-mcp.yaml — skills injection, MCP server, effort, systemPrompt

Supporting files

  • Test skill, MCP config, test command, Python script

Cost

  • ~$0.01-0.02 per run using haiku (Claude) and gpt-5.1-codex-mini (Codex)

Requirements

  • ANTHROPIC_API_KEY and OPENAI_API_KEY GitHub secrets must be configured

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions