Skip to content

Diagnose and mitigate large tool-result retention in long sessions #4184

@qqqys

Description

@qqqys

Parent

Part of #3000 and builds on /doctor memory diagnostics.

Goal

Introduce large tool-result and history retention diagnostics, then use those signals to design a safe offload/preview path that reduces OOM risk in long sessions.

Motivation

For agent CLIs, OOM often comes from retaining oversized tool outputs or duplicating history during compression, not just from a traditional leak. Claude Code-style behavior suggests a useful pattern: keep structured metadata and previews in the live context, persist large payloads out of the hot path, and make diagnostics show where memory is being retained.

Proposed solution

Phase 1: diagnostics only

  • Track aggregate size of tool results retained in live session/history.
  • Report largest tool result size, total retained tool-result bytes/chars, and count of oversized results.
  • Report whether large outputs are also present in UI history or compression inputs.
  • Expose these aggregate signals through /doctor memory without content.

Phase 2: mitigation design

  • Define a threshold for large tool result preview/offload.
  • Persist full payloads to a local diagnostic/session artifact store when needed.
  • Keep only a preview, metadata, and retrieval pointer in hot history.
  • Ensure model-facing context receives either a bounded preview or summarized representation.

Non-goals

  • Do not upload tool results.
  • Do not include private file contents or command output in diagnostics.
  • Do not silently drop data without a retrievable local pointer.
  • Do not block the baseline /doctor memory PR on offload implementation.

Acceptance criteria

  • /doctor memory can identify large retained tool results by size/count only.
  • A design note documents the offload/preview state transition and privacy model.
  • Follow-up implementation has tests showing large outputs are bounded in hot memory/context while still recoverable locally.

Metadata

Metadata

Assignees

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions