Symptom
With compaction.mode: "safeguard" configured, sessions can grow far past contextTokens (e.g. 640k tokens / 3.2x cap) without the safeguard ever firing. Hard compaction must be triggered manually with /compact. Cron sessions are particularly affected — observed ratios up to 756% of context cap.
memoryFlush fires correctly throughout the bloat (its own byte-gate works) but does not trim conversation history; the session JSONL on disk continues growing unbounded.
Configuration
{
"agents": {
"defaults": {
"contextTokens": 200000,
"compaction": {
"mode": "safeguard",
"reserveTokensFloor": 40000,
"maxHistoryShare": 0.3,
"truncateAfterCompaction": true,
"memoryFlush": {
"softThresholdTokens": 20000,
"forceFlushTranscriptBytes": "1mb"
}
}
}
}
}
Root Cause
Two compounding bugs in src/agents/pi-embedded-runner/run/attempt.ts:
Bug 1 — windowed-array race. The context engine's assemble() runs at lines 1738–1753 BEFORE the preemptive compaction check at lines 2319–2324. assemble() returns a windowed subset of messages that fits the engine's internal budget, and activeSession.messages is overwritten with the windowed view. When shouldPreemptivelyCompactBeforePrompt() (src/agents/pi-embedded-runner/run/preemptive-compaction.ts:41) then runs estimateMessagesTokens() on activeSession.messages, it estimates only the window — never the full session. The safeguard becomes blind to the session's true size.
Bug 2 — ownsCompaction hard bypass. At src/agents/pi-embedded-runner/run/attempt.ts:2309, if contextEngine.info.ownsCompaction === true, the entire preemptive check is skipped (returns dummy route: "fits", shouldCompact: false). Per src/context-engine/types.ts:47-60, this flag means the engine "manages its own compaction lifecycle" — but no fallback safeguard exists if the engine fails to actually do so. Codex-style proxy engines that set this flag get no openclaw-side compaction protection at all.
Why memoryFlush works but safeguard doesn't
|
memoryFlush |
safeguard preemptive |
| Token source |
persisted entry.totalTokens (real usage) |
windowed in-memory messages array (estimated) |
| Byte gate |
forceFlushTranscriptBytes reads JSONL stat.size |
none |
Bypassed by ownsCompaction |
no |
yes |
memoryFlush at src/auto-reply/reply/agent-runner-memory.ts:518-863 reads ground-truth disk state. The safeguard reads only the (windowed) in-memory view.
Proposed Fix
Three changes, in priority order:
P1 — Add transcript-bytes hard floor for compaction. Mirror forceFlushTranscriptBytes: add agents.defaults.compaction.forceCompactTranscriptBytes (suggested default "2mb"). In runPreflightCompactionIfNeeded (src/auto-reply/reply/agent-runner-memory.ts:518), add a sibling byte-gate alongside the existing memoryFlush byte-gate. If the JSONL exceeds the threshold, queue a compaction regardless of in-memory token estimate or ownsCompaction flag. This bypasses both bugs by reading disk state directly. Smallest blast radius; most direct fix.
P2 — Heartbeat-triggered context check. Add a periodic (e.g. 60s) check that reads session JSONL stat.size + last-known entry.totalTokens for active sessions; if either crosses threshold, force-queue compaction even when no prompt is arriving. Critical for cron sessions that loop without external prompts — those have no in-band trigger at all today. New module src/agents/pi-embedded-runner/heartbeat-compaction.ts calling the same compaction queue used by P1.
P3 — Pre-window the safeguard check. In src/agents/pi-embedded-runner/run/attempt.ts, run shouldPreemptivelyCompactBeforePrompt() against activeSession.messages BEFORE contextEngine.assemble() (or pass the pre-assemble snapshot). And: when ownsCompaction === true, still run a "last-resort overflow check" against the on-disk session size, even if the in-band preemptive check is skipped. Larger design conversation about engine contracts; punt to follow-up issue if needed.
Tests
src/agents/pi-embedded-runner/run.overflow-compaction.test.ts currently asserts the preemptive check fires when in-memory messages exceed budget — but no test covers the case where messages are windowed before the check runs, or where ownsCompaction === true. Tests for both should be added alongside P1/P2.
Affected versions
Reproduced on 2026.4.22. HEAD on 2026.4.24 has the same code path — preemptive check at attempt.ts:2319 still runs after assemble() at attempt.ts:1738, and the ownsCompaction bypass at line 2309 is unchanged.
cc @jalehman (Compaction, Context Engine) @sebslight (Agent Reliability, Runtime Hardening)
Symptom
With
compaction.mode: "safeguard"configured, sessions can grow far pastcontextTokens(e.g. 640k tokens / 3.2x cap) without the safeguard ever firing. Hard compaction must be triggered manually with/compact. Cron sessions are particularly affected — observed ratios up to 756% of context cap.memoryFlush fires correctly throughout the bloat (its own byte-gate works) but does not trim conversation history; the session JSONL on disk continues growing unbounded.
Configuration
{ "agents": { "defaults": { "contextTokens": 200000, "compaction": { "mode": "safeguard", "reserveTokensFloor": 40000, "maxHistoryShare": 0.3, "truncateAfterCompaction": true, "memoryFlush": { "softThresholdTokens": 20000, "forceFlushTranscriptBytes": "1mb" } } } } }Root Cause
Two compounding bugs in
src/agents/pi-embedded-runner/run/attempt.ts:Bug 1 — windowed-array race. The context engine's
assemble()runs at lines 1738–1753 BEFORE the preemptive compaction check at lines 2319–2324.assemble()returns a windowed subset of messages that fits the engine's internal budget, andactiveSession.messagesis overwritten with the windowed view. WhenshouldPreemptivelyCompactBeforePrompt()(src/agents/pi-embedded-runner/run/preemptive-compaction.ts:41) then runsestimateMessagesTokens()onactiveSession.messages, it estimates only the window — never the full session. The safeguard becomes blind to the session's true size.Bug 2 — ownsCompaction hard bypass. At
src/agents/pi-embedded-runner/run/attempt.ts:2309, ifcontextEngine.info.ownsCompaction === true, the entire preemptive check is skipped (returns dummyroute: "fits",shouldCompact: false). Persrc/context-engine/types.ts:47-60, this flag means the engine "manages its own compaction lifecycle" — but no fallback safeguard exists if the engine fails to actually do so. Codex-style proxy engines that set this flag get no openclaw-side compaction protection at all.Why memoryFlush works but safeguard doesn't
entry.totalTokens(real usage)messagesarray (estimated)forceFlushTranscriptBytesreads JSONLstat.sizeownsCompactionmemoryFlush at
src/auto-reply/reply/agent-runner-memory.ts:518-863reads ground-truth disk state. The safeguard reads only the (windowed) in-memory view.Proposed Fix
Three changes, in priority order:
P1 — Add transcript-bytes hard floor for compaction. Mirror
forceFlushTranscriptBytes: addagents.defaults.compaction.forceCompactTranscriptBytes(suggested default"2mb"). InrunPreflightCompactionIfNeeded(src/auto-reply/reply/agent-runner-memory.ts:518), add a sibling byte-gate alongside the existing memoryFlush byte-gate. If the JSONL exceeds the threshold, queue a compaction regardless of in-memory token estimate orownsCompactionflag. This bypasses both bugs by reading disk state directly. Smallest blast radius; most direct fix.P2 — Heartbeat-triggered context check. Add a periodic (e.g. 60s) check that reads session JSONL
stat.size+ last-knownentry.totalTokensfor active sessions; if either crosses threshold, force-queue compaction even when no prompt is arriving. Critical for cron sessions that loop without external prompts — those have no in-band trigger at all today. New modulesrc/agents/pi-embedded-runner/heartbeat-compaction.tscalling the same compaction queue used by P1.P3 — Pre-window the safeguard check. In
src/agents/pi-embedded-runner/run/attempt.ts, runshouldPreemptivelyCompactBeforePrompt()againstactiveSession.messagesBEFOREcontextEngine.assemble()(or pass the pre-assemble snapshot). And: whenownsCompaction === true, still run a "last-resort overflow check" against the on-disk session size, even if the in-band preemptive check is skipped. Larger design conversation about engine contracts; punt to follow-up issue if needed.Tests
src/agents/pi-embedded-runner/run.overflow-compaction.test.tscurrently asserts the preemptive check fires when in-memory messages exceed budget — but no test covers the case where messages are windowed before the check runs, or whereownsCompaction === true. Tests for both should be added alongside P1/P2.Affected versions
Reproduced on 2026.4.22. HEAD on 2026.4.24 has the same code path — preemptive check at
attempt.ts:2319still runs afterassemble()atattempt.ts:1738, and theownsCompactionbypass at line 2309 is unchanged.cc @jalehman (Compaction, Context Engine) @sebslight (Agent Reliability, Runtime Hardening)