Skip to content

fix(compressor): fix context compression Pass 3, make it JSON-safe for tool_call arguments and add test#11821

Closed
otherview wants to merge 1 commit into
NousResearch:mainfrom
otherview:main
Closed

fix(compressor): fix context compression Pass 3, make it JSON-safe for tool_call arguments and add test#11821
otherview wants to merge 1 commit into
NousResearch:mainfrom
otherview:main

Conversation

@otherview

Copy link
Copy Markdown

What does this PR do?

This PR fixes a context compression bug where Pass 3 could corrupt tool_calls.function.arguments by raw string truncation (args[:200] + "...[truncated]"), producing invalid JSON and causing downstream json.loads(arguments) failures (for example Unterminated string).

The fix keeps Pass 3, but makes it JSON-safe:

  • Parse arguments as JSON before modifying.
  • Shrink large payloads structurally (recursive trim of long strings/lists/dicts).
  • Re-serialize with json.dumps(...) so output stays valid JSON.
  • Preserve pre-existing invalid argument strings unchanged (no mutation).

This approach preserves the intent of Pass 3 (reduce oversized args) without violating the tool-call JSON contract.

Related Issue

Fixes #

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • Updated agent/context_compressor.py:
    • Added _shrink_json_for_tool_args(...) recursive helper.
    • Replaced raw-truncate Pass 3 logic with JSON-safe parse/shrink/serialize flow.
    • Preserved invalid pre-existing arguments verbatim (skip mutation on parse failure).
    • Updated method docstring to reflect JSON-safe argument shrinking behavior.
  • Updated tests/agent/test_context_compressor.py:
    • Added regression test ensuring large args are shrunk but remain valid JSON.
    • Added regression test ensuring malformed existing args are not modified.

How to Test

  1. Reproduce old behavior (before fix): run a long session with large tool arguments and compaction; observe JSON parse failures on truncated args.
  2. Run tests:
    • pytest -q tests/agent/test_context_compressor.py
  3. Verify fixed behavior:
    • Large tool_calls.function.arguments are reduced in size.
    • json.loads(arguments) succeeds for shrunk payloads.
    • Already-invalid argument strings are unchanged after pruning.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: macOS

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A

Screenshots / Logs

  • N/A (logic + tests change; no UI changes).

@otherview otherview changed the title fix(compressor): make Pass 3 JSON-safe for tool_call arguments and test fix(compressor): fix context compression Pass 3, make it JSON-safe for tool_call arguments and add test Apr 17, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Closing as duplicate of PR #12259 (merged). You independently diagnosed the same bug on Apr 17 within hours of #11788 and #11617, and your test coverage was excellent. We went with #11788's implementation because it was similar in approach but with slightly simpler logic and came with its own test suite. Credited in the salvage PR body — thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants