Skip to content

fix(workflows): persist structuredOutput on NodeOutput so $node.output.field works for Pi#1637

Closed
truffle-dev wants to merge 1 commit into
coleam00:mainfrom
truffle-dev:fix/workflows-persist-structured-output-on-node-output-1571
Closed

fix(workflows): persist structuredOutput on NodeOutput so $node.output.field works for Pi#1637
truffle-dev wants to merge 1 commit into
coleam00:mainfrom
truffle-dev:fix/workflows-persist-structured-output-on-node-output-1571

Conversation

@truffle-dev

Copy link
Copy Markdown
Contributor

Scope-fixed re-open of #1636 (closed by @Wirasm with "more changes than the pr body claims"). The branch was rebased off downstream dev which carried 179 unmerged-to-main commits; force-pushed onto upstream/main. Diff now matches the body: 5 files, +283/-9, single commit. bun test src/condition-evaluator.test.ts src/dag-executor.test.ts passes 263/0 locally.


Summary

  • Problem: When a provider parses fence-wrapped or preamble-prefixed JSON onto the result chunk (Pi/Minimax via tryParseStructuredOutput), the DAG executor captures it into a local variable but never persists it onto NodeOutput. Downstream $node.output.field substitution and when: conditions then try JSON.parse(output) on the original prose-prefixed text, throw, and resolve to empty.
  • Why it matters: every Pi/Minimax workflow with a downstream node that reads $classifier.output.type (or any other field) silently routes to the wrong branch or drops the value.
  • What changed: structuredOutput?: unknown added to the completed/running and failed NodeOutput branches; producer call sites persist it; consumers (substituteNodeOutputRefs, condition-evaluator.resolveOutputRef) prefer it over JSON.parse(output) and fall back to the existing path when absent.
  • What did not change: bare $node.output (unfielded) still reads output text. Cross-resume rehydration from event_data is out of scope (resumed runs that re-execute downstream nodes fall through to JSON.parse). Providers that already overwrite output with a JSON.stringify of structuredOutput (Claude/Codex with output_format) are unaffected because their output is already JSON.

UX Journey

Before

Pi/Minimax DAG with downstream $.field reads

  classify (provider=pi, prompt asks for JSON)
    ├─ raw chunk:        "Here is the classification:\n{\"type\":\"BUG\"}"
    ├─ tryParseStructuredOutput → { type: "BUG" }
    ├─ stored:           NodeOutput.output = raw text, structuredOutput = DROPPED
    │
  triage (depends_on: classify, when: $classify.output.type == 'BUG')
    └─ resolveOutputRef:
         JSON.parse("Here is the classification:\n{...}")  ── throws
         returns ''
         condition: '' == 'BUG' → false → SKIPPED  ❌

After

Pi/Minimax DAG with downstream $.field reads

  classify (provider=pi, prompt asks for JSON)
    ├─ raw chunk:        "Here is the classification:\n{\"type\":\"BUG\"}"
    ├─ tryParseStructuredOutput → { type: "BUG" }
    ├─ stored:           NodeOutput.output = raw text,
                         [NodeOutput.structuredOutput = { type: "BUG" }]
    │
  triage (depends_on: classify, when: $classify.output.type == 'BUG')
    └─ resolveOutputRef:
         [prefers NodeOutput.structuredOutput.type]  → "BUG"
         condition: 'BUG' == 'BUG' → true → RUNS  ✓

Architecture Diagram

Before

   ┌───────────────────────────┐
   │  providers (pi/minimax)   │
   │  emits result chunk with  │
   │  structuredOutput field   │
   └───────────┬───────────────┘
               │
               ▼
   ┌───────────────────────────┐
   │     dag-executor.ts       │
   │   captures msg.struct-    │
   │   uredOutput in local     │
   │   var (line ~913)         │
   │                           │
   │   on success return:      │
   │   { state, output,        │
   │     sessionId, costUsd }  │── structuredOutput dropped
   └───────────┬───────────────┘
               │
               ▼
   ┌───────────────────────────┐    ┌───────────────────────────┐
   │ substituteNodeOutputRefs  │    │   condition-evaluator     │
   │   JSON.parse(output)      │    │   JSON.parse(output)      │
   │   throws on prose preamble│    │   throws on prose preamble│
   └───────────────────────────┘    └───────────────────────────┘

After

   ┌───────────────────────────┐
   │  providers (pi/minimax)   │
   │  emits result chunk with  │
   │  structuredOutput field   │
   └───────────┬───────────────┘
               │
               ▼
   ┌───────────────────────────┐
   │     dag-executor.ts       │
   │   captures msg.struct-    │
   │   uredOutput in local     │
   │   var (line ~913)         │
   │                           │
   │   on success return:      │
   │   { state, output,        │
   │     sessionId, costUsd,   │
   │  [~ structuredOutput ] }  │── persisted
   └───────────┬───────────────┘
               │
               ▼
   ┌───────────────────────────┐    ┌───────────────────────────┐
   │ substituteNodeOutputRefs  │    │   condition-evaluator     │
   │ [~ prefer structuredOut]  │    │ [~ prefer structuredOut]  │
   │   falls back to           │    │   falls back to           │
   │   JSON.parse(output)      │    │   JSON.parse(output)      │
   └───────────────────────────┘    └───────────────────────────┘

Connection inventory:

From To Status Notes
dag-executor (single-shot success return) NodeOutput.structuredOutput new spread when defined
dag-executor (loop terminal-iteration return) NodeOutput.structuredOutput new new lastIterationStructuredOutput local
substituteNodeOutputRefs NodeOutput.structuredOutput new preferred over JSON.parse
condition-evaluator.resolveOutputRef NodeOutput.structuredOutput new preferred over JSON.parse
nodeOutputSchema (completed/running, failed) z.unknown().optional() new field back-compat

Label Snapshot

  • Risk: risk: low
  • Size: size: S
  • Scope: workflows
  • Module: workflows:executor, workflows:condition-evaluator, workflows:schemas

Change Metadata

  • Change type: bug
  • Primary scope: workflows

Linked Issue

Validation Evidence (required)

bun run --filter @archon/workflows test       # all green (workflows package)
bun --filter '*' test                          # all packages green
bun run --filter @archon/workflows type-check  # 0 errors
bun run type-check                             # full repo, 0 errors
bun run format:check                           # All matched files use Prettier code style!
bun x eslint .                                 # 0 errors
bun run check:bundled                          # up to date
bun run check:bundled-skill                    # up to date

New tests added:

  • condition-evaluator.test.ts: 12 new tests covering structuredOutput preference, Claude/Codex JSON.parse fallback regression, numeric/boolean/object/array field coercion, null/top-level-array/primitive fall-through, missing-field semantics, and bare $node.output (no field) still uses output text.
  • dag-executor.test.ts: 13 new tests for substituteNodeOutputRefs covering the same matrix plus shell-escaping safety on structuredOutput fields.

Security Impact (required)

  • New permissions/capabilities? No
  • New external network calls? No
  • Secrets/tokens handling changed? No
  • File system access scope changed? No

Compatibility / Migration

  • Backward compatible? Yes. structuredOutput is optional on the schema; producers populate it only when the provider emits one; consumers fall through to the existing JSON.parse(output) path when it is absent. Workflows that depend on the JSON.parse path keep working unchanged.
  • Config/env changes? No
  • Database migration needed? No. NodeOutput is stored inline in workflow event data; older rows without structuredOutput deserialize as undefined and hit the JSON.parse fallback.
  • Resume note: on a paused-then-resumed workflow run, downstream nodes that re-execute against an already-completed upstream will rehydrate NodeOutput from event_data. If the original run was on this version, the field is preserved; if older, it is absent and the JSON.parse fallback runs. Cross-version resume of Pi workflows that previously routed wrong will route correctly once the upstream re-runs on the new version.

Human Verification (required)

  • Verified scenarios:
    • Pi-shape: prose output + populated structuredOutput → dot-access reads structuredOutput.
    • Claude/Codex output_format-shape: empty/JSON output, no structuredOutput → JSON.parse fallback still works.
    • Loop terminal iteration: structuredOutput from the final iteration is what survives onto NodeOutput.
    • Bare $node.output (no field) ignores structuredOutput and reads output text.
    • Missing field on structuredOutput → empty string (does not silently retry JSON.parse).
    • null / top-level-array / primitive structuredOutput → falls through to JSON.parse (matches existing edge-case semantics).
    • Shell-escaping path (escapedForBash=true) wraps structuredOutput strings/objects in single quotes; numbers/booleans stay unquoted.
  • Edge cases checked: empty output text combined with populated structuredOutput (Pi-only-structured case) still resolves the field correctly because the empty-output early-return was moved inward.
  • What was not verified by hand: end-to-end Pi workflow against the live provider (the change is verifiable via unit tests against the consumer call sites and the producer return shape).

Side Effects / Blast Radius (required)

  • Affected subsystems/workflows: any DAG workflow with a $node.output.field substitution or a when: expression that uses dot notation.
  • Potential unintended effects: a workflow author who currently relies on JSON.parse(output) returning a different shape than what the provider's parsed structuredOutput contains (e.g. provider parses one JSON object out of multiple in the prose) would see a behavior change. In practice the providers that emit structuredOutput are emitting the canonical structured payload that the prompt asked for, so the change is the intended fix, not a surprise.
  • Guardrails: existing log lines (condition_json_parse_failed, dag_node_output_ref_json_parse_failed) still fire on the fallback path; no new error paths introduced.

Rollback Plan (required)

  • Fast rollback: revert the single commit on this PR. Schema field is optional and writing it on new rows does not break readers that don't know about it.
  • Feature flags: none. The change is mechanical and small enough that flagging it would add more risk than it removes.
  • Observable failure symptoms (if regression slipped through): $node.output.field would resolve to JSON-stringified content where it previously resolved to a plain string (would surface as condition mismatches or wrongly-formatted prompts on Claude/Codex output_format workflows).

Risks and Mitigations

  • Risk: a workflow that historically saw an empty string for a $node.output.field on a Pi/Minimax node (because the JSON.parse failed) might now see the resolved value and take a previously-unreachable branch. This is the intended behavior change for Parsed structured output not persisted to NodeOutput — condition evaluator and $node.output.field re-parse raw text, breaking Pi/Minimax #1571 but could surprise authors who built around the bug.
    • Mitigation: workflow authors who want the old empty-string behavior can keep relying on missing-field semantics (the new code returns '' when the structuredOutput object doesn't contain the requested key). The fix only changes the case where the provider actually parsed the structured payload the prompt requested.

…t.field works for Pi

When a provider parses fence-wrapped or preamble-prefixed JSON onto the
result chunk (Pi/Minimax via tryParseStructuredOutput), the executor
captured it locally but never persisted it onto NodeOutput. Downstream
consumers (substituteNodeOutputRefs, condition-evaluator) then
JSON.parse(output)'d the original prose-prefixed text, which threw, and
$node.output.field resolved to empty.

This persists structuredOutput on NodeOutput (single-shot and
loop-terminal-iteration success paths) and teaches both consumers to
prefer the parsed object over re-parsing prose. Falls back to
JSON.parse(output) when structuredOutput is absent so Claude/Codex
output_format-encoded NodeOutput rows (and older rows written before
this field existed) keep working.

Cross-resume rehydration of structuredOutput from event_data is
out of scope here; resumed runs that re-execute downstream nodes
will fall through to the JSON.parse path, which matches existing
behavior.

Closes coleam00#1571
@coderabbitai

coderabbitai Bot commented May 11, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 83ee59bb-6e6c-44ac-ab1f-b88495005ef1

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@Wirasm

Wirasm commented May 12, 2026

Copy link
Copy Markdown
Collaborator

This is targeting main, please reopen targeting dev

@truffle-dev

Copy link
Copy Markdown
Contributor Author

Re-targeted onto dev as #1654. Branch rebased clean onto upstream/dev, no code change. fcb33ed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants