Skip to content

Embedded agent auto-compaction retries without reducing 873k-token Paperclip prompt #75740

@coolmanns

Description

@coolmanns

Summary

A long-running embedded/Paperclip agent session reached context overflow, auto-compaction retried three times, but the estimated prompt size did not decrease at all. The run remained at ~873k estimated prompt tokens against a ~242k prompt budget.

This looks like auto-compaction reporting attempts without actually reducing the selected prompt window for this session shape.

Environment

  • OpenClaw: 2026.4.29 (a448042)
  • Provider/model: openai-codex/gpt-5.5
  • Session key: agent:roi-ceo:paperclip
  • Node: v22.22.2
  • OS: Linux x64

Log excerpt

[context-overflow-diag] sessionKey=agent:roi-ceo:paperclip provider=openai-codex/gpt-5.5 source=promptError messages=49 sessionFile=/home/coolmann/.openclaw/agents/roi-ceo/sessions/9bbbe46d-d3af-486f-9f63-92b711c60d3d.jsonl diagId=ovf-mon4azsa-YdtnzQ compactionAttempts=1 observedTokens=unknown error=Context overflow: prompt too large for the model (precheck).
context overflow detected (attempt 2/3); attempting auto-compaction for openai-codex/gpt-5.5
[context-overflow-precheck] sessionKey=agent:roi-ceo:paperclip provider=openai-codex/gpt-5.5 route=compact_only estimatedPromptTokens=873084 promptBudgetBeforeReserve=242144 overflowTokens=630940 toolResultReducibleChars=0 reserveTokens=20000 effectiveReserveTokens=20000 sessionFile=/home/coolmann/.openclaw/agents/roi-ceo/sessions/9bbbe46d-d3af-486f-9f63-92b711c60d3d.jsonl
[context-overflow-diag] sessionKey=agent:roi-ceo:paperclip provider=openai-codex/gpt-5.5 source=promptError messages=49 sessionFile=/home/coolmann/.openclaw/agents/roi-ceo/sessions/9bbbe46d-d3af-486f-9f63-92b711c60d3d.jsonl diagId=ovf-mon4b3l8-X2J3QA compactionAttempts=2 observedTokens=unknown error=Context overflow: prompt too large for the model (precheck).
context overflow detected (attempt 3/3); attempting auto-compaction for openai-codex/gpt-5.5
[context-overflow-precheck] sessionKey=agent:roi-ceo:paperclip provider=openai-codex/gpt-5.5 route=compact_only estimatedPromptTokens=873084 promptBudgetBeforeReserve=242144 overflowTokens=630940 toolResultReducibleChars=0 reserveTokens=20000 effectiveReserveTokens=20000 sessionFile=/home/coolmann/.openclaw/agents/roi-ceo/sessions/9bbbe46d-d3af-486f-9f63-92b711c60d3d.jsonl
[context-overflow-diag] sessionKey=agent:roi-ceo:paperclip provider=openai-codex/gpt-5.5 source=promptError messages=49 sessionFile=/home/coolmann/.openclaw/agents/roi-ceo/sessions/9bbbe46d-d3af-486f-9f63-92b711c60d3d.jsonl diagId=ovf-mon4b7eg-Bk-OzQ compactionAttempts=3 observedTokens=unknown error=Context overflow: prompt too large for the model (precheck).

Session file shape

Local inspection of the JSONL session file showed:

size_bytes=17248811
lines=11546
roles={'session': 1, 'model_change': 1, 'thinking_level_change': 1, 'custom': 2252, 'user': 2272, 'assistant': 4687, 'toolResult': 2332}

Largest individual events were only around 20k chars, and the recent tail showed many repeated medium-sized wake/prompt/assistant/tool events, e.g. user events around ~2.4k chars and assistant events around ~1-3k chars.

So this was not one huge reducible tool result. It was accumulated conversation/session history that eventually made the selected prompt window enormous.

Expected behavior

When auto-compaction is attempted after a precheck overflow, one of these should happen:

  1. The selected prompt window is reduced below budget.
  2. The old session history is summarized/rotated/truncated according to policy.
  3. The run fails once with a clear actionable error instead of retrying identical oversized prompts.

Actual behavior

Auto-compaction attempted three times, but the next precheck still reported the same estimated prompt size:

estimatedPromptTokens=873084
promptBudgetBeforeReserve=242144
overflowTokens=630940
toolResultReducibleChars=0

Operational workaround

I archived the bloated local session file and replaced the active file with an empty one so the agent can start fresh:

~/.openclaw/agents/roi-ceo/sessions/archive-context-overflow-20260501/9bbbe46d-d3af-486f-9f63-92b711c60d3d.jsonl

That is a workaround, not a root-cause fix.

Impact

Long-running embedded/Paperclip agents can accumulate enough session history to become unrecoverable by normal auto-compaction. The logs imply compaction is happening, but no effective reduction occurs, which makes the failure hard to reason about operationally.

Metadata

Metadata

Assignees

No one assigned

    Labels

    staleMarked as stale due to inactivity

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions