Skip to content

fix(agents): skip bootstrap file injection on continuation messages#44220

Closed
JoshuaLelon wants to merge 2 commits into
openclaw:mainfrom
JoshuaLelon:fix/9157-token-waste
Closed

fix(agents): skip bootstrap file injection on continuation messages#44220
JoshuaLelon wants to merge 2 commits into
openclaw:mainfrom
JoshuaLelon:fix/9157-token-waste

Conversation

@JoshuaLelon

Copy link
Copy Markdown
Contributor

Summary

  • Problem: Bootstrap files (~35k tokens) are re-injected into every Pi session message, even continuations that already have them in context.
  • Why it matters: Wastes ~35k tokens per continuation message, increasing cost and latency.
  • What changed: Check whether the session file exists before calling resolveBootstrapContextForRun(). If the session file exists (continuation), skip bootstrap injection.
  • What did NOT change: First-message behavior; bootstrap files are still injected on the first turn.

Change Type (select all)

  • Bug fix

Scope (select all touched areas)

  • Gateway / orchestration

Linked Issue/PR

User-visible / Behavior Changes

  • Reduced token usage on Pi session continuation messages (~35k tokens saved per message).
  • No behavioral change in responses (context was already present from the first turn).

Security Impact (required)

  • New permissions/capabilities? No
  • Secrets/tokens handling changed? No
  • New/changed network calls? No
  • Command/tool execution surface changed? No
  • Data access scope changed? No

Repro + Verification

Steps

  1. Start a Pi session and send a message (first turn)
  2. Send a follow-up message in the same session
  3. Observe token count in the follow-up is ~35k lower

Expected

  • Continuation messages don't re-inject bootstrap files

Actual (before fix)

  • Every message includes full bootstrap files regardless of session state

Evidence

  • Root cause: resolveBootstrapContextForRun() is called unconditionally in attempt.ts. The session file existence check (fs.stat(params.sessionFile)) reliably distinguishes first vs. continuation messages.

Compatibility / Migration

  • Backward compatible? Yes
  • Config/env changes? No
  • Migration needed? No

Failure Recovery (if this breaks)

  • Revert this commit; bootstrap injection returns to every-message behavior
  • Known bad symptoms: first message of a session missing bootstrap context (would indicate the session file check is wrong)

Risks and Mitigations

  • Risk: Race condition where session file is created between check and bootstrap call
    • Mitigation: Session file is only created after the full attempt completes; the check runs before any writes

@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: XS labels Mar 12, 2026
@greptile-apps

greptile-apps Bot commented Mar 12, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR optimizes Pi session token usage by skipping the resolveBootstrapContextForRun() call (~35k tokens) on continuation messages, gating it behind a single fs.stat(params.sessionFile) check (isFirstMessage). The duplicate fs.stat concern flagged in the prior review has been cleanly resolved by reusing isFirstMessage as hadSessionFile.

  • The repairSessionFileIfNeeded function only repairs existing files and never creates or deletes the session file, confirming hadSessionFile = !isFirstMessage is semantically equivalent to the original fresh fs.stat check.
  • workspaceNotes is now undefined on every continuation turn — it is derived from the empty hookAdjustedBootstrapFiles and controls the "Reminder: commit your changes after edits" line that is injected into the system prompt. Since the system prompt is regenerated on each turn, this reminder will be missing from all follow-up messages. The PR description states "no behavioral change", but this is a real change to the system prompt content on continuation turns that needs to be verified as intentional.

Confidence Score: 3/5

  • Core optimization is correct, but a side-effect on workspaceNotes needs verification before merging.
  • The isFirstMessage / hadSessionFile consolidation is logically sound and the prior duplicate-stat concern is resolved. The unresolved issue is that workspaceNotes — the system-prompt reminder to commit workspace changes — is now derived from the always-empty hookAdjustedBootstrapFiles on continuation turns, silently removing it from the system prompt for all follow-up messages. Whether this is intentional needs confirmation, since the system prompt is not persisted in the session file and must include anything the model needs on every turn.
  • src/agents/pi-embedded-runner/run/attempt.ts — the workspaceNotes derivation at lines 1296-1300 should be reviewed for correctness on continuation turns.

Comments Outside Diff (1)

  1. src/agents/pi-embedded-runner/run/attempt.ts, line 1296-1300 (link)

    workspaceNotes silently dropped from system prompt on continuation turns

    workspaceNotes (the "Reminder: commit your changes in this workspace after edits." line) is derived from hookAdjustedBootstrapFiles, which is now always [] on continuation messages. As a result, workspaceNotes will be undefined for every continuation turn and the reminder will be absent from the system prompt for those requests.

    The system prompt is regenerated from scratch on every turn — it is not stored in the session file and replayed — so any content that should appear on all turns must be included unconditionally. If the workspaceNotes reminder needs to remain in the system prompt for continuation messages (so the model is consistently reminded to commit after edits), the guard must be evaluated independently of the bootstrap-file check, for example:

    // Evaluate once from the original bootstrap files so it is present on all turns,
    // not just the first message.
    const workspaceNotes = isFirstMessage
      ? hookAdjustedBootstrapFiles.some(
          (file) => file.name === DEFAULT_BOOTSTRAP_FILENAME && !file.missing,
        )
        ? ["Reminder: commit your changes in this workspace after edits."]
        : undefined
      : params.workspaceHasBootstrapFile  // pass this flag through params, or derive another way
      ? ["Reminder: commit your changes in this workspace after edits."]
      : undefined;

    If the intent is that this reminder is only needed for the first turn (because it is already present in the session history from the first turn's system prompt), please add a comment explaining that, since the system prompt is not currently stored in session history in a way that would make it available on continuation turns.

Prompt To Fix All With AI
This is a comment left during a code review.
Path: src/agents/pi-embedded-runner/run/attempt.ts
Line: 1296-1300

Comment:
**`workspaceNotes` silently dropped from system prompt on continuation turns**

`workspaceNotes` (the "Reminder: commit your changes in this workspace after edits." line) is derived from `hookAdjustedBootstrapFiles`, which is now always `[]` on continuation messages. As a result, `workspaceNotes` will be `undefined` for every continuation turn and the reminder will be absent from the system prompt for those requests.

The system prompt is regenerated from scratch on every turn — it is not stored in the session file and replayed — so any content that should appear on all turns must be included unconditionally. If the `workspaceNotes` reminder needs to remain in the system prompt for continuation messages (so the model is consistently reminded to commit after edits), the guard must be evaluated independently of the bootstrap-file check, for example:

```typescript
// Evaluate once from the original bootstrap files so it is present on all turns,
// not just the first message.
const workspaceNotes = isFirstMessage
  ? hookAdjustedBootstrapFiles.some(
      (file) => file.name === DEFAULT_BOOTSTRAP_FILENAME && !file.missing,
    )
    ? ["Reminder: commit your changes in this workspace after edits."]
    : undefined
  : params.workspaceHasBootstrapFile  // pass this flag through params, or derive another way
  ? ["Reminder: commit your changes in this workspace after edits."]
  : undefined;
```

If the intent is that this reminder is only needed for the first turn (because it is already present in the session history from the first turn's system prompt), please add a comment explaining that, since the system prompt is not currently stored in session history in a way that would make it available on continuation turns.

How can I resolve this? If you propose a fix, please make it concise.

Last reviewed commit: d228f31

Comment thread src/agents/pi-embedded-runner/run/attempt.ts

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1ff196ae54

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/agents/pi-embedded-runner/run/attempt.ts
@katoue

This comment was marked as spam.

@JoshuaLelon JoshuaLelon force-pushed the fix/9157-token-waste branch from 1ff196a to d228f31 Compare March 12, 2026 16:43
@JoshuaLelon

Copy link
Copy Markdown
Contributor Author

Addressed in d228f31 — replaced the duplicate fs.stat call at line 1364 with const hadSessionFile = !isFirstMessage, eliminating the redundant filesystem round-trip and keeping the two variables explicitly in sync. Added a linking comment at the isFirstMessage declaration.

Also rebased on latest main.

@greptile-apps anything else?

@JoshuaLelon JoshuaLelon force-pushed the fix/9157-token-waste branch from d228f31 to 18970b4 Compare March 12, 2026 17:02

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 18970b4739

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/agents/pi-embedded-runner/run/attempt.ts
Comment thread src/agents/pi-embedded-runner/run/attempt.ts
@JoshuaLelon

Copy link
Copy Markdown
Contributor Author

@codex Re: detecting first turn from transcript content rather than file existence — acknowledged. The current approach (!isFirstMessage from fs.stat) is consistent with the existing pattern at line 1083 and works correctly for the normal flow where the session file is created on first message. Pre-created or partially-written session files from crash recovery are an edge case that would need the SessionManager to expose a "has prior turns" check, which is a larger refactor. Noting this as a potential follow-up.

The duplicate fs.stat call was consolidated as suggested — now reuses the isFirstMessage result computed earlier.

Anything else to address?

JoshuaLelon and others added 2 commits March 12, 2026 15:44
Only inject workspace bootstrap files on the first message of a Pi
session. Continuation messages already have them in context from the
initial turn, so re-injecting wastes ~35k tokens per message.

Fixes openclaw#9157

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@JoshuaLelon JoshuaLelon force-pushed the fix/9157-token-waste branch from 18970b4 to 2c15960 Compare March 12, 2026 20:47
@JoshuaLelon

Copy link
Copy Markdown
Contributor Author

Rebased onto latest upstream/main.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2c159603df

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +1429 to +1432
const isFirstMessage = !(await fs
.stat(params.sessionFile)
.then(() => true)
.catch(() => false));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Determine first-turn state from transcript contents

isFirstMessage is inferred only from fs.stat(sessionFile), but this codebase still treats “file exists, no assistant turn yet” as a first-turn-like state (prepareSessionManagerForRun has an explicit hadSessionFile && header && !hasAssistant branch in src/agents/pi-embedded-runner/session-manager-init.ts:44-52, and runWithOrphanedSingleUserMessage seeds this transcript shape in src/agents/pi-embedded-runner.e2e.test.ts:146-154). In those cases this change skips resolveBootstrapContextForRun, so the first successful reply loses bootstrap file context and bootstrap hook adjustments.

Useful? React with 👍 / 👎.

.stat(params.sessionFile)
.then(() => true)
.catch(() => false);
const hadSessionFile = !isFirstMessage;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Recompute session-file presence after lock acquisition

hadSessionFile now comes from pre-lock isFirstMessage, but acquireSessionWriteLock explicitly retries while waiting on contended locks (src/agents/session-write-lock.ts:482-547), so the transcript can be created by another worker before this run enters the critical section. When that happens, this stale value misclassifies a continuation as first-turn, which skips continuation-only handling like contextEngine.bootstrap and passes the wrong hadSessionFile into session initialization.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling size: XS

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Performance: Workspace file injection wastes 93.5% of token budget

2 participants