Skip to content

fix(agents): replace prose terminal classifiers#93228

Merged
steipete merged 5 commits into
mainfrom
fix/structured-terminal-tool-fallbacks
Jun 15, 2026
Merged

fix(agents): replace prose terminal classifiers#93228
steipete merged 5 commits into
mainfrom
fix/structured-terminal-tool-fallbacks

Conversation

@steipete

Copy link
Copy Markdown
Contributor

Summary

  • remove language-specific assistant-prose classification and use explicit harness/protocol terminal states
  • preserve trusted post-middleware terminal summaries for structured replay-safe tools, with fail-closed mutation and replay evidence
  • keep exhausted and non-fallbackable failures non-winning and failure-terminal across lifecycle, session, and model bookkeeping
  • align OpenClaw and Codex tool semantics; supersedes fix: surface safe terminal tool fallbacks #90872 while preserving @fuller-stack-dev's original intent and commit credit

Release note: prevents silent agent turns without parsing natural-language progress text and avoids replaying side effects during model fallback.

Verification

  • focused matrix: 698 tests across unit-fast, auto-reply, agents, E2E, and Codex-extension shards
  • pnpm tsgo:core
  • pnpm check:test-types
  • pnpm build
  • clean Codex branch autoreview
  • Azure Linux check:changed: run_2a378f6aa965 (proof)

Source Study

  • OpenCode: structured tool/runtime completion behavior
  • Codex: app-server plan items, command actions, dynamic tools, collaboration items, and shell safety
  • Cline: explicit task/tool lifecycle handling

@openclaw-barnacle openclaw-barnacle Bot added docs Improvements or additions to documentation agents Agent runtime and tooling extensions: codex size: XL maintainer Maintainer-authored PR labels Jun 15, 2026
@steipete

steipete commented Jun 15, 2026

Copy link
Copy Markdown
Contributor Author

Land-ready verification for 2c83b4e543b14aaea45e3a38e4214793d29acfc5:

  • Focused terminal-tool matrix: 239 tests passed across unit, agent, and E2E shards.
  • Final affected-path rerun: 148 tests passed, including structured replay classification and before-tool-call E2E coverage.
  • pnpm tsgo:core — passed.
  • pnpm check:test-types — passed.
  • pnpm build — passed.
  • Touched-file oxfmt, oxlint, and git diff --check — passed.
  • Codex autoreview — clean; no accepted/actionable findings.
  • Full GitHub CI — passed: https://github.com/openclaw/openclaw/actions/runs/27537729181
  • Azure Linux Crabbox check:changed — passed: run_084cb5e4c6aa, lease cbx_2758a90baaed.

No known proof gaps.

@clawsweeper

clawsweeper Bot commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs changes before merge. Reviewed June 15, 2026, 5:51 AM ET / 09:51 UTC.

Summary
This PR replaces prose-based agent and Codex terminal classification with structured terminal/replay metadata, fallback-exhaustion handling, SDK/runtime helper exports, docs, and regression tests.

Reproducibility: yes. Source inspection shows current main lacks the structured terminal/replay path, and the PR head still has a concrete Codex dynamic-tool ordering path where concurrent server requests can finish out of order without ordinals.

Review metrics: 1 noteworthy metric.

  • Runtime contract additions: 1 optional agent-core event field, 4 agent-harness-runtime exports. These additive surfaces affect plugin and harness compatibility, so maintainers should notice them before merge.

Stored data model
Persistent data-model change detected: serialized state: src/agents/command/session-store.test.ts, serialized state: src/agents/embedded-agent-runner/run.empty-error-retry.test.ts, serialized state: src/auto-reply/reply/session-usage.ts, serialized state: src/auto-reply/reply/session.test.ts, unknown-data-model-change: src/agents/command/session-store.test.ts, unknown-data-model-change: src/auto-reply/reply/session-usage.ts, and 6 more. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🦐 gold shrimp
Proof: 🐚 platinum hermit ✨ media proof bonus
Patch quality: 🦐 gold shrimp
Result: needs maintainer review before merge.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P2] Pass allocateToolOutcomeOrdinal through Codex dynamic tool construction and add an out-of-order terminal-presentation regression test.
  • Have maintainers explicitly accept or internalize the new experimental harness SDK exports.

Risk before merge

  • [P1] Codex dynamic terminal presentations can be selected by completion order rather than model tool-call order until allocateToolOutcomeOrdinal is passed through dynamic-tool construction.
  • [P1] The new openclaw/plugin-sdk/agent-harness-runtime exports broaden an experimental plugin/harness API surface; maintainers should explicitly accept that contract or keep it internal before merge.
  • [P2] Fallback, replay, and session-state changes decide whether channel-visible tool results are retried, preserved, or treated as terminal, so green unit/CI proof does not remove all integration risk.

Maintainer options:

  1. Fix Codex terminal-summary ordering (recommended)
    Pass allocateToolOutcomeOrdinal through Codex dynamic tool construction and cover two out-of-order terminal-presentation tools before merge.
  2. Accept the harness SDK additions
    Maintainers can explicitly approve the new experimental agent-harness-runtime exports as the intended plugin contract for trusted harnesses.
  3. Keep helpers internal instead
    If the SDK surface is not intended to become public, move the new replay and terminal-presentation helpers behind an internal core/Codex seam before landing.
Copy recommended automerge instruction
@clawsweeper automerge

Special instructions:
Pass `allocateToolOutcomeOrdinal` through Codex dynamic tool construction so terminal summaries use model-call order, and add a focused Codex dynamic-tool test where two terminal-presentation tools finish out of order.

Next step before merge

  • [P2] A narrow automated repair can address the Codex ordinal blocker, while maintainers still need to make the SDK API decision before merge.

Security
Cleared: No workflow, lockfile, dependency, package-script, or secret-handling change was found; the terminal-summary paths use redaction-oriented helpers and I found no concrete security defect.

Review findings

  • [P2] Pass Codex tool outcome ordinals — extensions/codex/src/app-server/dynamic-tool-build.ts:293
Review details

Best possible solution:

Pass model-order tool outcome ordinals through the Codex dynamic-tool path, add a focused out-of-order terminal-presentation test, then land only after maintainers accept the experimental harness SDK surface.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection shows current main lacks the structured terminal/replay path, and the PR head still has a concrete Codex dynamic-tool ordering path where concurrent server requests can finish out of order without ordinals.

Is this the best way to solve the issue?

No, not yet. The structured protocol-state direction is the right layer, but the Codex dynamic-tool path needs ordinal propagation and maintainer acceptance of the SDK surface before this is the best merge shape.

Full review comments:

  • [P2] Pass Codex tool outcome ordinals — extensions/codex/src/app-server/dynamic-tool-build.ts:293
    Codex app-server requests are dispatched concurrently, but this new onToolOutcome path does not also forward allocateToolOutcomeOrdinal into createOpenClawCodingTools. Terminal presentations then fall back to completion order, so a slower earlier dynamic tool can overwrite or appear after a later one and the incomplete-turn summary can cite the wrong tool.
    Confidence: 0.9

Overall correctness: patch is incorrect
Overall confidence: 0.88

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 773ffd87a1e1.

Label changes

Label changes:

  • add P1: The PR targets silent agent turns and unsafe model-fallback side-effect replay, which can break real agent/channel workflows.
  • add merge-risk: 🚨 compatibility: The diff expands experimental Plugin SDK/harness exports and changes fallback behavior that existing harness users may depend on.
  • add merge-risk: 🚨 message-delivery: The diff changes terminal replay and tool-result presentation around channel-visible output, where a wrong classification can suppress or misrepresent messages.
  • add merge-risk: 🚨 session-state: The diff changes exhausted-fallback session usage and runtime model preservation, so mistakes could stale or mis-associate session state.
  • add rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🐚 platinum hermit and patch quality is 🦐 gold shrimp.
  • add status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (linked_artifact): The PR body/comment provide exact after-patch verification plus a linked Crabbox run, and GitHub reports the Real behavior proof check as passing for this head.
  • remove rating: 🐚 platinum hermit: Current PR rating is rating: 🦐 gold shrimp, so this older rating label is no longer current.
  • remove status: 👀 ready for maintainer look: Current PR status label is status: ⏳ waiting on author.

Label justifications:

  • P1: The PR targets silent agent turns and unsafe model-fallback side-effect replay, which can break real agent/channel workflows.
  • merge-risk: 🚨 compatibility: The diff expands experimental Plugin SDK/harness exports and changes fallback behavior that existing harness users may depend on.
  • merge-risk: 🚨 message-delivery: The diff changes terminal replay and tool-result presentation around channel-visible output, where a wrong classification can suppress or misrepresent messages.
  • merge-risk: 🚨 session-state: The diff changes exhausted-fallback session usage and runtime model preservation, so mistakes could stale or mis-associate session state.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🐚 platinum hermit and patch quality is 🦐 gold shrimp.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (linked_artifact): The PR body/comment provide exact after-patch verification plus a linked Crabbox run, and GitHub reports the Real behavior proof check as passing for this head.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body/comment provide exact after-patch verification plus a linked Crabbox run, and GitHub reports the Real behavior proof check as passing for this head.
Evidence reviewed

Acceptance criteria:

  • [P1] node scripts/run-vitest.mjs extensions/codex/src/app-server/dynamic-tool-build.test.ts extensions/codex/src/app-server/dynamic-tools.test.ts -- --reporter=dot.
  • [P1] node scripts/run-vitest.mjs src/agents/agent-tools.before-tool-call.integration.e2e.test.ts -- --reporter=dot.
  • [P1] pnpm tsgo:core.

What I checked:

Likely related people:

  • steipete: Merged the recent empty post-tool final-turn fix that this PR builds on and owns the current maintainer-labeled branch beyond being the proposer. (role: recent merger and adjacent owner; confidence: high; commits: 77a682c5def2; files: src/agents/embedded-agent-runner/run.ts, extensions/codex/src/app-server/dynamic-tools.ts, src/plugin-sdk/agent-harness-runtime.ts)
  • fuller-stack-dev: Authored the related closed terminal-fallback PR and the merged narrower post-tool empty-turn fix that this PR supersedes and extends. (role: source feature contributor; confidence: high; commits: 77a682c5def2, 78083a3a98ba; files: src/agents/embedded-agent-runner/result-fallback-classifier.ts, src/agents/embedded-agent-runner/run.ts, extensions/codex/src/app-server/dynamic-tools.ts)
  • zenglingbiao: Recently changed before_tool_call wrapper idempotency on the hook metadata path this PR extends for adjusted params, blocked calls, and replay trust. (role: recent adjacent contributor; confidence: medium; commits: 07dfdd4bd038; files: src/agents/agent-tools.before-tool-call.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: baeb8d3eca

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

recordToolPrepStage: (name) => {
toolBuildStages.mark(name);
},
onToolOutcome: params.onToolOutcome,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Pass Codex tool outcome ordinals

In Codex app-server turns that issue more than one dynamic tool call concurrently (the client dispatches server requests with void this.handleServerRequest(...)), this newly enabled onToolOutcome path records terminal presentations without the allocateToolOutcomeOrdinal that the normal embedded path passes. observeToolOutcome then falls back to completion order, so a slower earlier tool can overwrite/appear after a later tool and the incomplete-turn terminal summary can cite the wrong tool. Please pass params.allocateToolOutcomeOrdinal through here.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed and landed in #93287 (a09f6b1). Tool outcome ordinals now come from raw model call order and are reused across native and dynamic lifecycle events; E2E coverage exercises delayed native command and ID-less web-search completion after a later dynamic summary.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Jun 15, 2026
@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. P1 High-priority user-facing bug, regression, or broken workflow. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Jun 15, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2c83b4e543

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

return;
}
this.terminalPresentationClearedItemIds.add(item.id);
this.params.onToolOutcome?.({

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve ordering when clearing terminal presentations

For Codex turns where a native tool item only appears in the final turn snapshot and an OpenClaw dynamic tool later recorded a terminal presentation, this unordered onToolOutcome clear runs after the dynamic outcome and has no toolCallOrdinal; observeToolOutcome then treats it as the newest observation and erases the later trusted terminal summary, so fallback-exhausted incomplete turns show the generic error instead of the tool-authored summary. Fresh evidence beyond the existing dynamic-tool ordinal comment is this newly added native snapshot clearing path.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed and landed in #93287 (a09f6b1). Final snapshot selection now ignores stale prior-turn items, and completion-only native plus timeout-late-success paths have regression coverage.

@steipete steipete merged commit 0314819 into main Jun 15, 2026
210 of 220 checks passed
@steipete steipete deleted the fix/structured-terminal-tool-fallbacks branch June 15, 2026 09:53
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jun 16, 2026
* fix(agents): replace prose terminal classifiers

Co-authored-by: fuller-stack-dev <263060202+fuller-stack-dev@users.noreply.github.com>

* fix(agents): preserve terminal failure lifecycles

* fix(agents): order parallel terminal summaries

* fix(agents): preserve structured post-tool silence

* fix(agents): preserve structured replay provenance

---------

Co-authored-by: fuller-stack-dev <263060202+fuller-stack-dev@users.noreply.github.com>
karmafeast pushed a commit to karmaterminal/openclaw that referenced this pull request Jun 16, 2026
Commit 0314819 (openclaw#93228) refactored the Codex dynamic-tool bridge to
consume before_tool_call adjusted params into executedArgs and then fed
that merged view to BOTH the native agentToolResultMiddleware runner and
the legacy codex app-server extension runner.

The native middleware runner is meant to observe OpenClaw's executed args
(merged adjustments), matching the after_tool_call hook view — that part
is correct and covered by dynamic-tools.test.ts. But the legacy
codex app-server extension runner has a different historical contract: it
observes the tool's prepared call args, before before_tool_call rewrites
them (openclaw-owned-tool-runtime-contract.test.ts). With the merged view
the contract extension's args assertion threw, the runner swallowed the
error, and the extension's compacted result was silently discarded — so
after_tool_call and the bridge return observed the raw result.

Restore the legacy extension runner's prepared-args view; the native
middleware runner keeps executedArgs. executedArgs (merged adjustments)
remains reserved for telemetry, messaging, replay-safety, and
after_tool_call.

Note: this regression reproduces on pristine upstream/main HEAD
(32d1ccd), not only on the merge — confirmed by running the contract
test in an upstream worktree.

Verified: full vitest.extension-codex-app-server-tools shard, 307/307.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
karmafeast pushed a commit to karmaterminal/openclaw that referenced this pull request Jun 16, 2026
Commit 0314819 (openclaw#93228) refactored the Codex dynamic-tool bridge to
consume before_tool_call adjusted params into executedArgs and then fed
that merged view to BOTH the native agentToolResultMiddleware runner and
the legacy codex app-server extension runner.

The native middleware runner is meant to observe OpenClaw's executed args
(merged adjustments), matching the after_tool_call hook view — that part
is correct and covered by dynamic-tools.test.ts. But the legacy
codex app-server extension runner has a different historical contract: it
observes the tool's prepared call args, before before_tool_call rewrites
them (openclaw-owned-tool-runtime-contract.test.ts). With the merged view
the contract extension's args assertion threw, the runner swallowed the
error, and the extension's compacted result was silently discarded — so
after_tool_call and the bridge return observed the raw result.

Restore the legacy extension runner's prepared-args view; the native
middleware runner keeps executedArgs. executedArgs (merged adjustments)
remains reserved for telemetry, messaging, replay-safety, and
after_tool_call.

Note: this regression reproduces on pristine upstream/main HEAD
(32d1ccd), not only on the merge — confirmed by running the contract
test in an upstream worktree.

Verified: full vitest.extension-codex-app-server-tools shard, 307/307.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
crh-code pushed a commit to crh-code/openclaw that referenced this pull request Jun 18, 2026
* fix(agents): replace prose terminal classifiers

Co-authored-by: fuller-stack-dev <263060202+fuller-stack-dev@users.noreply.github.com>

* fix(agents): preserve terminal failure lifecycles

* fix(agents): order parallel terminal summaries

* fix(agents): preserve structured post-tool silence

* fix(agents): preserve structured replay provenance

---------

Co-authored-by: fuller-stack-dev <263060202+fuller-stack-dev@users.noreply.github.com>
badgerbees pushed a commit to badgerbees/openclaw that referenced this pull request Jul 8, 2026
* fix(agents): replace prose terminal classifiers

Co-authored-by: fuller-stack-dev <263060202+fuller-stack-dev@users.noreply.github.com>

* fix(agents): preserve terminal failure lifecycles

* fix(agents): order parallel terminal summaries

* fix(agents): preserve structured post-tool silence

* fix(agents): preserve structured replay provenance

---------

Co-authored-by: fuller-stack-dev <263060202+fuller-stack-dev@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling docs Improvements or additions to documentation extensions: codex maintainer Maintainer-authored PR merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. P1 High-priority user-facing bug, regression, or broken workflow. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. size: XL status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant