Skip to content

agents: strengthen GPT-5.4 execution bias and close the one-action-then-narrative loophole#65257

Closed
100yenadmin wants to merge 10 commits intoopenclaw:mainfrom
electricsheephq:fix/gpt54-agentic-behavior
Closed

agents: strengthen GPT-5.4 execution bias and close the one-action-then-narrative loophole#65257
100yenadmin wants to merge 10 commits intoopenclaw:mainfrom
electricsheephq:fix/gpt54-agentic-behavior

Conversation

@100yenadmin
Copy link
Copy Markdown
Contributor

@100yenadmin 100yenadmin commented Apr 12, 2026

Summary

Four targeted fixes that close the behavioral gap between GPT-5.4 and Claude agents in OpenClaw. Based on root-cause analysis of the actual code paths in extensions/openai/prompt-overlay.ts, src/agents/pi-embedded-runner/run.ts, and src/agents/pi-embedded-runner/run/incomplete-turn.ts.

Part of #64227. See the umbrella for how this fits with #65219 (runtime activation) and #65224 (test proof).

What changed (3 commits)

Commit 1: Prompt + detection layer

extensions/openai/prompt-overlay.ts

  • OPENAI_GPT5_EXECUTION_BIAS rewritten from passive ("Start the real work") to imperative ("Use a real tool call FIRST. Commentary-only turns are incomplete. Do not stop after one step to ask permission."). Removed "Before the final answer, quickly verify" which GPT-5.4 used as a pause point.
  • New OPENAI_GPT5_TOOL_CALL_STYLE section override: "Call tools directly without narrating. If multiple tool calls are needed, call them in sequence without stopping." Includes all three approval safety lines from the default section (never exec /approve, allow-once is single-command, preserve full command for approval).
  • resolveOpenAISystemPromptContribution now returns execution_bias + tool_call_style + interaction_style overrides.

src/agents/pi-embedded-runner/run/incomplete-turn.ts

  • New isSingleActionThenNarrativePattern(): detects turns where exactly 1 non-plan tool call + planning prose ("I'll", "let me") = retry eligible.
  • Changed the exemption at line 355: hasNonPlanToolActivity && !isSingleActionThenNarrativePattern so the one-action loophole is closed.

Commit 2: Auto-continuation loop

Config surface (src/config/types.agent-defaults.ts + zod schemas + labels):

  • agents.defaults.embeddedPi.continuationMode: "auto" / "prompt" / "off" (default: "auto" for strict-agentic GPT-5, "prompt" for everything else)
  • agents.defaults.embeddedPi.continuationBudget: 1-20 (default: 5)

Continuation intent detection (incomplete-turn.ts):

  • hasContinuationIntent() — detects promise patterns in the model's visible text
  • hasCompletionLanguage() — detects "done", "finished", "implemented"
  • AUTO_CONTINUATION_INSTRUCTION — "Continue. Take the next concrete action immediately."

The loop (run.ts, ~line 1720):
When GPT-5.4's turn ends with tool results + continuation intent + no completion language, the runner injects the continuation instruction and continues back to the while loop instead of returning. 11-condition safety guard includes budget, abort, timeout, tool errors, messaging, side effects, approval prompts, and completion language.

Commit 3: Review fixes

  • Started-count bypass — both the hasNonPlanToolActivity AND startedCount > planOnlyToolMetaCount guards now have the isSingleActionThenNarrativePattern exception (Codex-connector P1 finding: the started-count guard was neutralizing the fix).
  • Approval safetyOPENAI_GPT5_TOOL_CALL_STYLE now includes the three approval safety lines from the default section (Copilot finding: the override was dropping safety guidance).
  • Side-effect guard — auto-continuation now checks !attempt.replayMetadata.hadPotentialSideEffects (Codex-connector P1 finding: mutating work could be duplicated on retry).
  • JSDoc — 700-char blind spot documented on isSingleActionThenNarrativePattern.

Safety rails on auto-continuation

Condition Prevents
autoContinuationCount < continuationBudget Runaway loops (max 5 turns)
!aborted && !timedOut Continuing after cancel/timeout
!attempt.clientToolCall Continuing when client-side tool execution needed
!attempt.yieldDetected Continuing when model explicitly yielded
!attempt.lastToolError Continuing after a tool error
!attempt.didSendViaMessagingTool Continuing after model already sent a message
!attempt.didSendDeterministicApprovalPrompt Continuing when explicit approval needed
!attempt.replayMetadata.hadPotentialSideEffects Continuing after mutating work (writes, sends)
toolMetas.some(e => e.toolName !== "update_plan") Only continuing when real tool work happened
hasContinuationIntent(...) Only continuing when model shows future-work language
!hasCompletionLanguage(...) Not continuing when model says "done" / "finished"

Tests

  • src/agents/pi-embedded-runner/run.incomplete-turn.test.ts — 16 passed
  • src/agents/system-prompt.test.ts — 52 passed
  • TypeScript check: 0 errors from changed files

Claude is unaffected: continuation loop defaults to "prompt" for non-strict-agentic runs; prompt overlay only applies to GPT-5 on openai/openai-codex providers.

Projected scorecard

Objective Before After
Follow instructions 7/10 9/10
Use tools proactively 7/10 9/10
Multi-step execution 6/10 10/10
No over-confirmation 6/10 9/10
Overall 6.5/10 9.25/10

How this PR relates to the others

…ative loophole, add tool-first reinforcement
Copilot AI review requested due to automatic review settings April 12, 2026 08:28
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9490e00d38

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/agents/pi-embedded-runner/run/incomplete-turn.ts Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Strengthens GPT-5.x agent “act-first” behavior in OpenClaw by (1) tightening the OpenAI prompt overlay to more aggressively reinforce tool execution, and (2) extending the planning-only retry guard to catch “one tool call then planning narration” turns that previously escaped retries.

Changes:

  • Updated GPT-5 execution-bias overlay text to explicitly require tool/action-first and continued multi-step tool use until completion or a real blocker.
  • Added a new heuristic (isSingleActionThenNarrativePattern) so a single non-plan tool call followed by planning prose can still be treated as “planning-only” for retry purposes.
  • Added an OpenAI GPT-5 tool_call_style section override to reinforce direct tool calling without interleaved narration.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
src/agents/pi-embedded-runner/run/incomplete-turn.ts Adds detection to close the “single tool call then plan/permission ask” retry loophole.
extensions/openai/prompt-overlay.ts Strengthens GPT-5 execution bias and introduces a GPT-5 tool-call style override.

Comment thread extensions/openai/prompt-overlay.ts Outdated
Comment thread src/agents/pi-embedded-runner/run/incomplete-turn.ts Outdated
Comment thread src/agents/pi-embedded-runner/run/incomplete-turn.ts
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Apr 12, 2026

Greptile Summary

This PR closes the "one action then narrative" loophole in the GPT-5.4 planning-only retry detector and adds a new prompt-overlay.ts module with stronger execution-bias and tool-call-style overrides. The detection logic in isSingleActionThenNarrativePattern is sound — 2+ non-plan tool calls and mutating single calls remain exempt, while 1 non-mutating tool call followed by planning prose is now correctly eligible for retry.

Two P2 items to consider before merge:

  • The existing test "does not retry planning-only detection after tool activity" continues to pass but now via the hadPotentialSideEffects bail path (because bash is mutating), not the hasNonPlanToolActivity bail. The new primary behavior — 1 non-mutating read-only tool call + planning prose → retry — has no dedicated test.
  • isSingleActionThenNarrativePattern silently falls back to old "no retry" behavior for planning texts longer than 700 characters; this edge case is not documented in the function's JSDoc.

Confidence Score: 5/5

Safe to merge; no P0/P1 defects found. Both findings are non-blocking P2 suggestions around test coverage and JSDoc clarity.

All changes are logically sound. The loophole fix correctly threads the needle between 0-tool, 1-non-mutating-tool, 1-mutating-tool, and 2+-tool scenarios. Existing tests still pass. Remaining findings are purely test-gap and documentation quality concerns.

No files require special attention for correctness; consider adding a test for the 1-non-mutating-tool-call + planning-prose path in run.incomplete-turn.test.ts.

Comments Outside Diff (1)

  1. src/agents/pi-embedded-runner/run.incomplete-turn.test.ts, line 165-178 (link)

    P2 Test coverage gap for new core behavior

    This test still passes after the change, but now for a different reason: bash is in MUTATING_TOOL_NAMES, so hadPotentialSideEffects = true and the function bails at the replayMetadata.hadPotentialSideEffects guard rather than at the old hasNonPlanToolActivity guard. The test title "does not retry planning-only detection after tool activity" no longer accurately describes the new invariant, which is "does not retry after mutating tool activity."

    More importantly, the primary new behavior — 1 non-mutating tool call (e.g. read_file) + planning prose → should retry — has no test coverage at all. Consider adding a case like:

    it("retries after a single read-only tool call followed by planning-only prose", () => {
      const retryInstruction = resolvePlanningOnlyRetryInstruction({
        provider: "openai",
        modelId: "gpt-5.4",
        aborted: false,
        timedOut: false,
        attempt: makeAttemptResult({
          assistantTexts: ["I'll now analyze the structure and make the changes."],
          toolMetas: [{ toolName: "read", meta: "src/foo.ts" }],
        }),
      });
    
      expect(retryInstruction).toContain("Act now");
    });
    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: src/agents/pi-embedded-runner/run.incomplete-turn.test.ts
    Line: 165-178
    
    Comment:
    **Test coverage gap for new core behavior**
    
    This test still passes after the change, but now for a different reason: `bash` is in `MUTATING_TOOL_NAMES`, so `hadPotentialSideEffects = true` and the function bails at the `replayMetadata.hadPotentialSideEffects` guard rather than at the old `hasNonPlanToolActivity` guard. The test title "does not retry planning-only detection after tool activity" no longer accurately describes the new invariant, which is "does not retry after *mutating* tool activity."
    
    More importantly, the primary new behavior — **1 non-mutating tool call (e.g. `read_file`) + planning prose → should retry** — has no test coverage at all. Consider adding a case like:
    
    ```ts
    it("retries after a single read-only tool call followed by planning-only prose", () => {
      const retryInstruction = resolvePlanningOnlyRetryInstruction({
        provider: "openai",
        modelId: "gpt-5.4",
        aborted: false,
        timedOut: false,
        attempt: makeAttemptResult({
          assistantTexts: ["I'll now analyze the structure and make the changes."],
          toolMetas: [{ toolName: "read", meta: "src/foo.ts" }],
        }),
      });
    
      expect(retryInstruction).toContain("Act now");
    });
    ```
    
    How can I resolve this? If you propose a fix, please make it concise.
Prompt To Fix All With AI
This is a comment left during a code review.
Path: src/agents/pi-embedded-runner/run.incomplete-turn.test.ts
Line: 165-178

Comment:
**Test coverage gap for new core behavior**

This test still passes after the change, but now for a different reason: `bash` is in `MUTATING_TOOL_NAMES`, so `hadPotentialSideEffects = true` and the function bails at the `replayMetadata.hadPotentialSideEffects` guard rather than at the old `hasNonPlanToolActivity` guard. The test title "does not retry planning-only detection after tool activity" no longer accurately describes the new invariant, which is "does not retry after *mutating* tool activity."

More importantly, the primary new behavior — **1 non-mutating tool call (e.g. `read_file`) + planning prose → should retry** — has no test coverage at all. Consider adding a case like:

```ts
it("retries after a single read-only tool call followed by planning-only prose", () => {
  const retryInstruction = resolvePlanningOnlyRetryInstruction({
    provider: "openai",
    modelId: "gpt-5.4",
    aborted: false,
    timedOut: false,
    attempt: makeAttemptResult({
      assistantTexts: ["I'll now analyze the structure and make the changes."],
      toolMetas: [{ toolName: "read", meta: "src/foo.ts" }],
    }),
  });

  expect(retryInstruction).toContain("Act now");
});
```

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: src/agents/pi-embedded-runner/run/incomplete-turn.ts
Line: 307-320

Comment:
**Silent 700-char blind spot not documented in JSDoc**

`isSingleActionThenNarrativePattern` returns `false` when `text.length > 700`, which means a model that does 1 `read_file` then writes 800 characters of planning prose is silently treated as "real progress" and is **not retried** — the opposite of the loophole this function aims to close. The behavior is consistent with the downstream cap in `resolvePlanningOnlyRetryInstruction`, but nothing in the JSDoc mentions this edge case. Since a long verbose response is actually a common symptom of the "one step then narrate" pattern, it's worth calling out explicitly:

```ts
 * Note: turns where the text exceeds 700 characters are not matched* they fall through to the downstream text-length guard in
 * resolvePlanningOnlyRetryInstruction and are treated as non-retryable,
 * consistent with the zero-tool-call path.
```

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "agents: strengthen GPT-5.4 execution bia..." | Re-trigger Greptile

Comment thread src/agents/pi-embedded-runner/run/incomplete-turn.ts
@openclaw-barnacle openclaw-barnacle Bot added docs Improvements or additions to documentation size: M and removed size: S labels Apr 12, 2026
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f725a590ac

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/agents/pi-embedded-runner/run/incomplete-turn.ts Outdated
Comment thread src/agents/pi-embedded-runner/run.ts
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e44b709336

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/agents/pi-embedded-runner/run.ts
Comment thread src/agents/agent-scope.ts Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a8055323fa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/agents/pi-embedded-runner/run/incomplete-turn.ts Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Comment thread src/agents/agent-scope.ts Outdated
Comment thread src/config/schema.labels.ts
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a64c7af40e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/agents/pi-embedded-runner/run/incomplete-turn.ts
Comment thread src/config/zod-schema.agent-runtime.ts
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.

Comment thread src/agents/pi-embedded-runner/run.ts
Comment thread src/agents/pi-embedded-runner/run/incomplete-turn.ts
Comment thread src/agents/pi-embedded-runner/run/incomplete-turn.ts
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 82b60b8b6c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/agents/pi-embedded-runner/run/incomplete-turn.ts Outdated
Comment thread extensions/openai/prompt-overlay.ts Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9de4ac9ef6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/agents/pi-embedded-runner/run/incomplete-turn.ts
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.

Comment thread src/agents/agent-scope.ts Outdated
Comment thread src/agents/pi-embedded-runner/run/incomplete-turn.ts Outdated
Comment thread src/agents/pi-embedded-runner/run.incomplete-turn.test.ts
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b3b717b2de

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/agents/pi-embedded-runner/run/incomplete-turn.ts
Comment thread src/agents/pi-embedded-runner/run/incomplete-turn.ts
@pashpashpash
Copy link
Copy Markdown
Contributor

Maintainer update:

We split the narrow GPT-5 prompt / one-action-then-narrative retry slice out of this branch and landed it separately in #65597.

Landed commit:
c848ebc

Source head from this PR at the time of follow-up triage:
b3b717b

The remaining auto-continuation / config-surface work in this PR was intentionally not landed as part of that slice. It is a larger product behavior change and should come back, if desired, as a focused follow-up against current main rather than bundled with the prompt / retry fixes.

Closing this PR as superseded by the landed split.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling docs Improvements or additions to documentation extensions: openai size: M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants