fix(codex): scale context engine projection#80761
Conversation
|
Codex review: needs changes before merge. Summary Reproducibility: yes. at source level. Current main passes the runtime token budget into context-engine assembly, then renders the assembled Codex prompt through a fixed 24k-character projection cap. Real behavior proof Next step before merge Security Review findings
Review detailsBest possible solution: Land a narrowed Codex projection fix that scales from contextTokenBudget while preserving reserveTokensFloor as a floor-only guard and leaving exact accounting telemetry to #80765. Do we have a high-confidence way to reproduce the issue? Yes, at source level. Current main passes the runtime token budget into context-engine assembly, then renders the assembled Codex prompt through a fixed 24k-character projection cap. Is this the best way to solve the issue? No, not as written. Budget-aware projection is the right boundary, but the patch should keep the default reserve unless reserveTokens is explicitly configured so floor-only settings do not remove headroom. Full review comments:
Overall correctness: patch is incorrect Acceptance criteria:
What I checked:
Likely related people:
Remaining risk / open question:
Codex review notes: model gpt-5.5, reasoning high; reviewed against fe1f30bfd629. |
e625948 to
0e55d18
Compare
|
@pashpashpash tagging also since you're over codex harness. |
0e55d18 to
62f1e3f
Compare
62f1e3f to
61c9550
Compare
|
Merged via squash.
Thanks @100yenadmin! |
Fixes #80760.
Native Codex already asks the OpenClaw context engine to assemble the runtime context, but the app-server projection layer then rendered that assembled result through a fixed 24k-character cap. That meant large LCM/frontier assemblies could be correctly selected upstream and still arrive at Codex as a much smaller quoted context block, which makes the runtime status look like context disappeared after restart or harness changes.
This PR makes the projection cap budget-aware for the active context-engine path:
contextTokenBudgetagents.defaults.compaction.reserveTokens/agents.defaults.compaction.reserveTokensFloorreserve surface, including the default 20k-token floor and small-context cap behaviorrunCodexAppServerAttemptroute, the hard upper bound, and the shared reserve-token cap behaviorThe intent is deliberately narrow: let native Codex actually receive a proportionate slice of the context the engine already assembled, while honoring the same reserve knobs users already configure in
openclaw.json. It does not add new runtime status telemetry or a visible LCM frontier-vs-provider-usage diagnostic; those are still useful follow-ups and are tracked upstream in Martian-Engineering/lossless-claw#658, but this patch fixes the send-side cap that caused the observed mismatch.Validation:
Real behavior proof
/Volumes/LEXAR/repos/openclaw-codex-context-projection-pr, based onopenclaw/openclawmain, with the incident reproduced on installed OpenClaw2026.5.10-beta.5against the liveagent:main:mainLCM session.258000tokens), plus a configured-reserve case that exercisesagents.defaults.compaction.reserveTokens/reserveTokensFloor.$ pnpm exec tsx -e 'import { projectContextEngineAssemblyForCodex, resolveCodexContextEngineProjectionMaxChars, resolveCodexContextEngineProjectionReserveTokens } from "./extensions/codex/src/app-server/context-engine-projection.ts"; const contextTokenBudget = 258000; const cap = resolveCodexContextEngineProjectionMaxChars({ contextTokenBudget }); const assembledMessages = Array.from({ length: 12 }, (_, i) => ({ role: "assistant", content: `${i}:${"x".repeat(5900)}` })); const projected = projectContextEngineAssemblyForCodex({ assembledMessages, originalHistoryMessages: [], prompt: "next", maxRenderedContextChars: cap }); const config = { agents: { defaults: { compaction: { reserveTokens: 12000, reserveTokensFloor: 0 } } } }; const reserveTokens = resolveCodexContextEngineProjectionReserveTokens({ config }); console.log(JSON.stringify({ contextTokenBudget, cap, promptChars: projected.promptText.length, truncated: projected.promptText.includes("[truncated "), configReserveTokens: reserveTokens, configuredReserveCap: resolveCodexContextEngineProjectionMaxChars({ contextTokenBudget: 80000, reserveTokens }) }, null, 2));' { "contextTokenBudget": 258000, "cap": 952000, "promptChars": 71198, "truncated": false, "configReserveTokens": 12000, "configuredReserveCap": 272000 }