Skip to content

Workflow validator scans prompt: body strings for $nodeId.output refs — false positives in fenced documentation examples (archon-workflow-builder fails to load) #1413

@xtksystems

Description

@xtksystems

Summary

packages/workflows/src/loader.ts validates $nodeId.output references across node.when, node.prompt, and loop.prompt strings (lines 148–166). The regex /\\$([a-zA-Z_][a-zA-Z0-9_-]*)\\.output/g matches anywhere in the prompt body, including inside fenced code blocks that are documentation examples for the LLM. As a result, archon-workflow-builder.yaml (a bundled default) fails to load on the latest dev branch.

Repro

On dev HEAD (verified at SHA 91226735):

```bash
git clone https://github.com/coleam00/Archon.git && cd Archon
bun install
bun run cli workflow list
```

Output includes:

```json
{"level":40,...,"filename":"archon-workflow-builder.yaml","structureError":"Node 'generate-yaml' references unknown node '$other-node.output'","msg":"dag_structure_invalid"}
```

And:

```bash
bun run cli workflow run archon-workflow-builder "Author hello-world. Single bash node that echoes hi."
```

```
Error: Workflow 'archon-workflow-builder' failed to load: Node 'generate-yaml' references unknown node '$other-node.output'
```

Root cause

.archon/workflows/defaults/archon-workflow-builder.yaml line 138, inside the generate-yaml prompt body, contains a fenced YAML example showing how to author a script node:

```yaml
script: |
const raw = String.raw`$other-node.output`;
const data = JSON.parse(raw);
```

This is documentation text for the LLM — meant to render literally so the model can emulate the syntax. The string `$other-node.output` is not intended to be substituted at runtime.

But the validator's regex matches it, sees no `other-node` in this workflow's node IDs, and rejects the whole file.

Expected

`archon-workflow-builder` loads. `bun run cli workflow run archon-workflow-builder "..."` succeeds.

Suggested fix

Have the validator skip fenced code blocks when scanning `prompt:` body strings. Minimal patch sketch (in `packages/workflows/src/loader.ts` near line 152):

```ts
if ('prompt' in node && typeof node.prompt === 'string') {
// Strip fenced code blocks before scanning — they're literal LLM-facing examples,
// not real $nodeId.output references.
const stripped = node.prompt.replace(/```[\s\S]*?```/g, '');
sources.push(stripped);
}
```

Equivalent treatment for `loop.prompt`. Indented code blocks (4-space) could be handled if needed but fenced blocks cover the existing default's case.

Alternative: add a workflow-author opt-out (e.g. a marker like ``), but that's a worse API since every author of an LLM-facing example would need to know about it.

Impact

  • All downstream forks at this pin SHA cannot use `archon-workflow-builder` to author new workflows. Mandatory "workflows are authored only via the builder" rules (e.g. Cortex's Boundary Rule Add docker-compose configuration with Streamlit and environment setup #32) become unreachable.
  • Users discover this only at runtime — `bun run cli workflow list` warns, but the failure mode for `workflow run archon-workflow-builder` is opaque (just "failed to load").
  • The same false positive will affect any user workflow whose `prompt:` body uses fenced-code documentation that mentions `$.output` syntax.

Environment

  • `coleam00/Archon` `dev` at SHA `91226735` ("v0.3.9-rc-13" informally — v0.3.9 + 13 post-tag commits, validated 2026-04-25)
  • Bun 1.3.13
  • macOS 14 (Darwin 25.4.0)
  • `bun run validate` otherwise passes (the broken default doesn't break the build, only runtime use of the builder).

Discovered while bringing up a private Archon fork's bootstrapped harness layer; happy to PR the validator fix if helpful.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High priority - Address soon, next in queuearea: workflowsWorkflow enginebugSomething is brokeneffort/lowSingle file or function, one responsibility, isolated change

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions