Skip to content

Fix/underestimates token count#16117

Closed
SimbaKingjoe wants to merge 2 commits into
NousResearch:mainfrom
SimbaKingjoe:fix/underestimates-token-count
Closed

Fix/underestimates token count#16117
SimbaKingjoe wants to merge 2 commits into
NousResearch:mainfrom
SimbaKingjoe:fix/underestimates-token-count

Conversation

@SimbaKingjoe

Copy link
Copy Markdown
Contributor

What does this PR do?

_find_tail_cut_by_tokens calls len(content) to estimate message tokens, but
when content is a multimodal list (e.g. [{"type": "text", "text": "..."}, {"type": "image_url", ...}]),
len() returns the number of blocks (~2) instead of total character count.
Every multimodal message is therefore estimated as ~10 tokens regardless of
actual size, causing tail protection to expand far beyond the token budget.

The fix sums text lengths across content blocks when content is a list —
mirroring the pattern already used correctly in the sibling method
_prune_old_tool_results (line 488).

Related Issue

Fixes #16087
Related: #14694 (anti-thrashing cascade), supersedes #14395 (closed without merge)

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✅ Tests (adding or improving test coverage)

Changes Made

  • agent/context_compressor.py — _find_tail_cut_by_tokens(): replace
    len(content) with sum(len(p.get("text", "")) for p in content if isinstance(p, dict)) if isinstance(content, list) else len(content)
  • tests/agent/test_context_compressor.py — added
    TestTokenBudgetTailProtection::test_multimodal_list_content_counted_correctly

How to Test

  1. Run the new regression test:
    uv run pytest tests/agent/test_context_compressor.py::TestTokenBudgetTailProtection::test_multimodal_list_content_counted_correctly -v

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent loop, run_agent.py, prompt builder labels Apr 26, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Note: PR #16113 also targets the same issue #16087 with an equivalent fix. Both replace len(content) with summed text lengths for multimodal list content in _find_tail_cut_by_tokens.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent loop, run_agent.py, prompt builder P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: underestimates token count for multimodal messages, causing oversized tail protection and ineffective context compression

2 participants