perf: add contextInjection option to skip workspace re-injection (#9157)#46813
perf: add contextInjection option to skip workspace re-injection (#9157)#46813anup00900 wants to merge 1 commit into
Conversation
…nclaw#9157) Workspace files (AGENTS.md, SOUL.md, USER.md, etc.) are re-injected into the system prompt on every message, wasting ~35,600 tokens per turn (~93.5% of the token budget in multi-message conversations). Add a contextInjection config option under agents.defaults: - "always" (default): current behavior, inject on every message - "first-message-only": inject only when the session file does not yet exist, reducing token costs by ~93% for subsequent messages The agent retains full workspace context from message openclaw#1 and can use the read tool to re-check files if needed on later messages.
|
@vincentkoc @steipete — this addresses the highly-requested #9157 (12 thumbs-up). Adds an opt-in |
Greptile SummaryThis PR adds a Two things worth addressing before merging:
Confidence Score: 4/5
Prompt To Fix All With AIThis is a comment left during a code review.
Path: src/agents/pi-embedded-runner/run/attempt.ts
Line: 1430-1435
Comment:
**Duplicate `fs.stat` call**
The `fs.stat(params.sessionFile)` call introduced here is duplicated approximately 300 lines later (line 1731–1734) where `hadSessionFile` is computed with the exact same logic:
```typescript
const hadSessionFile = await fs
.stat(params.sessionFile)
.then(() => true)
.catch(() => false);
```
Additionally, `repairSessionFileIfNeeded` runs between the two stat calls (line 1727), which could theoretically produce inconsistent results if the repair alters the file's existence.
Consider moving the bootstrap context resolution to after `hadSessionFile` is computed, and reusing it:
```typescript
const skipContextInjection =
contextInjection === "first-message-only" && hadSessionFile;
```
This eliminates one unnecessary filesystem I/O operation and keeps the "did a session file already exist?" logic in one place.
How can I resolve this? If you propose a fix, please make it concise.
---
This is a comment left during a code review.
Path: src/agents/pi-embedded-runner/run/attempt.ts
Line: 1436-1446
Comment:
**No re-injection after compaction**
When context compaction occurs, the session file still exists on disk, so `skipContextInjection` will remain `true` for every subsequent `runEmbeddedAttempt` call — including the first one after a compaction event. Because compaction replaces or summarises the message history (including the initial bootstrap system prompt content), workspace files like `SOUL.md` and `USER.md` that were only injected on the very first message of the session may no longer be present in the model's effective context after compaction.
The existing `compaction.postCompactionSections` mechanism re-injects named sections from `AGENTS.md`, but it does not cover the full bootstrap context that `resolveBootstrapContextForRun` would have provided.
Consider documenting this tradeoff explicitly in the `contextInjection` JSDoc, e.g.:
```typescript
/**
* ...
* Note: with "first-message-only", workspace context is NOT re-injected after
* context compaction. Use compaction.postCompactionSections to explicitly
* preserve critical sections from AGENTS.md across compaction boundaries.
*/
contextInjection?: "always" | "first-message-only";
```
How can I resolve this? If you propose a fix, please make it concise.Last reviewed commit: e0e0733 |
| const skipContextInjection = | ||
| contextInjection === "first-message-only" && | ||
| (await fs | ||
| .stat(params.sessionFile) | ||
| .then(() => true) | ||
| .catch(() => false)); |
There was a problem hiding this comment.
Duplicate fs.stat call
The fs.stat(params.sessionFile) call introduced here is duplicated approximately 300 lines later (line 1731–1734) where hadSessionFile is computed with the exact same logic:
const hadSessionFile = await fs
.stat(params.sessionFile)
.then(() => true)
.catch(() => false);Additionally, repairSessionFileIfNeeded runs between the two stat calls (line 1727), which could theoretically produce inconsistent results if the repair alters the file's existence.
Consider moving the bootstrap context resolution to after hadSessionFile is computed, and reusing it:
const skipContextInjection =
contextInjection === "first-message-only" && hadSessionFile;This eliminates one unnecessary filesystem I/O operation and keeps the "did a session file already exist?" logic in one place.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/pi-embedded-runner/run/attempt.ts
Line: 1430-1435
Comment:
**Duplicate `fs.stat` call**
The `fs.stat(params.sessionFile)` call introduced here is duplicated approximately 300 lines later (line 1731–1734) where `hadSessionFile` is computed with the exact same logic:
```typescript
const hadSessionFile = await fs
.stat(params.sessionFile)
.then(() => true)
.catch(() => false);
```
Additionally, `repairSessionFileIfNeeded` runs between the two stat calls (line 1727), which could theoretically produce inconsistent results if the repair alters the file's existence.
Consider moving the bootstrap context resolution to after `hadSessionFile` is computed, and reusing it:
```typescript
const skipContextInjection =
contextInjection === "first-message-only" && hadSessionFile;
```
This eliminates one unnecessary filesystem I/O operation and keeps the "did a session file already exist?" logic in one place.
How can I resolve this? If you propose a fix, please make it concise.| const { bootstrapFiles: hookAdjustedBootstrapFiles, contextFiles } = skipContextInjection | ||
| ? { bootstrapFiles: [], contextFiles: [] } | ||
| : await resolveBootstrapContextForRun({ | ||
| workspaceDir: effectiveWorkspace, | ||
| config: params.config, | ||
| sessionKey: params.sessionKey, | ||
| sessionId: params.sessionId, | ||
| warn: makeBootstrapWarn({ sessionLabel, warn: (message) => log.warn(message) }), | ||
| contextMode: params.bootstrapContextMode, | ||
| runKind: params.bootstrapContextRunKind, | ||
| }); |
There was a problem hiding this comment.
No re-injection after compaction
When context compaction occurs, the session file still exists on disk, so skipContextInjection will remain true for every subsequent runEmbeddedAttempt call — including the first one after a compaction event. Because compaction replaces or summarises the message history (including the initial bootstrap system prompt content), workspace files like SOUL.md and USER.md that were only injected on the very first message of the session may no longer be present in the model's effective context after compaction.
The existing compaction.postCompactionSections mechanism re-injects named sections from AGENTS.md, but it does not cover the full bootstrap context that resolveBootstrapContextForRun would have provided.
Consider documenting this tradeoff explicitly in the contextInjection JSDoc, e.g.:
/**
* ...
* Note: with "first-message-only", workspace context is NOT re-injected after
* context compaction. Use compaction.postCompactionSections to explicitly
* preserve critical sections from AGENTS.md across compaction boundaries.
*/
contextInjection?: "always" | "first-message-only";Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/pi-embedded-runner/run/attempt.ts
Line: 1436-1446
Comment:
**No re-injection after compaction**
When context compaction occurs, the session file still exists on disk, so `skipContextInjection` will remain `true` for every subsequent `runEmbeddedAttempt` call — including the first one after a compaction event. Because compaction replaces or summarises the message history (including the initial bootstrap system prompt content), workspace files like `SOUL.md` and `USER.md` that were only injected on the very first message of the session may no longer be present in the model's effective context after compaction.
The existing `compaction.postCompactionSections` mechanism re-injects named sections from `AGENTS.md`, but it does not cover the full bootstrap context that `resolveBootstrapContextForRun` would have provided.
Consider documenting this tradeoff explicitly in the `contextInjection` JSDoc, e.g.:
```typescript
/**
* ...
* Note: with "first-message-only", workspace context is NOT re-injected after
* context compaction. Use compaction.postCompactionSections to explicitly
* preserve critical sections from AGENTS.md across compaction boundaries.
*/
contextInjection?: "always" | "first-message-only";
```
How can I resolve this? If you propose a fix, please make it concise.There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e0e0733a77
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| contextInjection === "first-message-only" && | ||
| (await fs | ||
| .stat(params.sessionFile) | ||
| .then(() => true) | ||
| .catch(() => false)); |
There was a problem hiding this comment.
Use assistant history to detect first turn before skipping context
This gate treats sessionFile existence as “not first message”, but later in the same flow prepareSessionManagerForRun explicitly handles pre-created/header-only transcripts as an initial run state. In that scenario (contextInjection: "first-message-only" + existing file without assistant turns), bootstrap files are skipped even though this is still the first real model turn, so AGENTS/SOUL/USER context is never injected for that session start.
Useful? React with 👍 / 👎.
| const { bootstrapFiles: hookAdjustedBootstrapFiles, contextFiles } = skipContextInjection | ||
| ? { bootstrapFiles: [], contextFiles: [] } | ||
| : await resolveBootstrapContextForRun({ |
There was a problem hiding this comment.
Keep heartbeat context resolution when first-message-only is set
Returning { bootstrapFiles: [], contextFiles: [] } here bypasses resolveBootstrapContextForRun entirely, which also bypasses run-kind-specific behavior. Heartbeat runs set bootstrapContextRunKind: "heartbeat" (see src/auto-reply/reply/agent-runner-execution.ts) and rely on applyContextModeFilter to inject HEARTBEAT.md each run (src/agents/bootstrap-files.ts); with this short-circuit, subsequent heartbeat turns lose that context whenever the session file already exists.
Useful? React with 👍 / 👎.
|
closing this one out. this idea is obsolete now: if there is still a missing edge case here, it should come back as a narrowly scoped follow-up against current |
Summary
Fixes #9157 — workspace files (AGENTS.md, SOUL.md, USER.md, etc.) are re-injected into the system prompt on every single message, wasting ~35,600 tokens per turn (~93.5% of the token budget).
The fix: Add a
contextInjectionconfig option underagents.defaults:{ "agents": { "defaults": { "contextInjection": "first-message-only" } } }"always"(default): current behavior — zero breaking changes"first-message-only": skip workspace file injection when the session file already existsMeasured impact (from issue reporter)
~93% token reduction over a conversation. ~$1.51 saved per 100-message session.
Changes
src/agents/pi-embedded-runner/run/attempt.ts: Check session file existence + config before callingresolveBootstrapContextForRunsrc/config/types.agent-defaults.ts: AddcontextInjectiontypesrc/config/zod-schema.agent-defaults.ts: Add Zod validationTest plan
attempt.spawn-workspace.test.ts— same onmain)"always"preserves current behavior — zero breaking changescontextInjection: "first-message-only"→ verify workspace files injected on msg 1, skipped on msg 2+readworkspace files on subsequent messagesRelated