Skip to content

compaction.mode=safeguard never triggers automatically when context engine windows messages or sets ownsCompaction #71325

@joeywrightphoto

Description

@joeywrightphoto

Symptom

With compaction.mode: "safeguard" configured, sessions can grow far past contextTokens (e.g. 640k tokens / 3.2x cap) without the safeguard ever firing. Hard compaction must be triggered manually with /compact. Cron sessions are particularly affected — observed ratios up to 756% of context cap.

memoryFlush fires correctly throughout the bloat (its own byte-gate works) but does not trim conversation history; the session JSONL on disk continues growing unbounded.

Configuration

{
  "agents": {
    "defaults": {
      "contextTokens": 200000,
      "compaction": {
        "mode": "safeguard",
        "reserveTokensFloor": 40000,
        "maxHistoryShare": 0.3,
        "truncateAfterCompaction": true,
        "memoryFlush": {
          "softThresholdTokens": 20000,
          "forceFlushTranscriptBytes": "1mb"
        }
      }
    }
  }
}

Root Cause

Two compounding bugs in src/agents/pi-embedded-runner/run/attempt.ts:

Bug 1 — windowed-array race. The context engine's assemble() runs at lines 1738–1753 BEFORE the preemptive compaction check at lines 2319–2324. assemble() returns a windowed subset of messages that fits the engine's internal budget, and activeSession.messages is overwritten with the windowed view. When shouldPreemptivelyCompactBeforePrompt() (src/agents/pi-embedded-runner/run/preemptive-compaction.ts:41) then runs estimateMessagesTokens() on activeSession.messages, it estimates only the window — never the full session. The safeguard becomes blind to the session's true size.

Bug 2 — ownsCompaction hard bypass. At src/agents/pi-embedded-runner/run/attempt.ts:2309, if contextEngine.info.ownsCompaction === true, the entire preemptive check is skipped (returns dummy route: "fits", shouldCompact: false). Per src/context-engine/types.ts:47-60, this flag means the engine "manages its own compaction lifecycle" — but no fallback safeguard exists if the engine fails to actually do so. Codex-style proxy engines that set this flag get no openclaw-side compaction protection at all.

Why memoryFlush works but safeguard doesn't

memoryFlush safeguard preemptive
Token source persisted entry.totalTokens (real usage) windowed in-memory messages array (estimated)
Byte gate forceFlushTranscriptBytes reads JSONL stat.size none
Bypassed by ownsCompaction no yes

memoryFlush at src/auto-reply/reply/agent-runner-memory.ts:518-863 reads ground-truth disk state. The safeguard reads only the (windowed) in-memory view.

Proposed Fix

Three changes, in priority order:

P1 — Add transcript-bytes hard floor for compaction. Mirror forceFlushTranscriptBytes: add agents.defaults.compaction.forceCompactTranscriptBytes (suggested default "2mb"). In runPreflightCompactionIfNeeded (src/auto-reply/reply/agent-runner-memory.ts:518), add a sibling byte-gate alongside the existing memoryFlush byte-gate. If the JSONL exceeds the threshold, queue a compaction regardless of in-memory token estimate or ownsCompaction flag. This bypasses both bugs by reading disk state directly. Smallest blast radius; most direct fix.

P2 — Heartbeat-triggered context check. Add a periodic (e.g. 60s) check that reads session JSONL stat.size + last-known entry.totalTokens for active sessions; if either crosses threshold, force-queue compaction even when no prompt is arriving. Critical for cron sessions that loop without external prompts — those have no in-band trigger at all today. New module src/agents/pi-embedded-runner/heartbeat-compaction.ts calling the same compaction queue used by P1.

P3 — Pre-window the safeguard check. In src/agents/pi-embedded-runner/run/attempt.ts, run shouldPreemptivelyCompactBeforePrompt() against activeSession.messages BEFORE contextEngine.assemble() (or pass the pre-assemble snapshot). And: when ownsCompaction === true, still run a "last-resort overflow check" against the on-disk session size, even if the in-band preemptive check is skipped. Larger design conversation about engine contracts; punt to follow-up issue if needed.

Tests

src/agents/pi-embedded-runner/run.overflow-compaction.test.ts currently asserts the preemptive check fires when in-memory messages exceed budget — but no test covers the case where messages are windowed before the check runs, or where ownsCompaction === true. Tests for both should be added alongside P1/P2.

Affected versions

Reproduced on 2026.4.22. HEAD on 2026.4.24 has the same code path — preemptive check at attempt.ts:2319 still runs after assemble() at attempt.ts:1738, and the ownsCompaction bypass at line 2309 is unchanged.

cc @jalehman (Compaction, Context Engine) @sebslight (Agent Reliability, Runtime Hardening)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions