Measured before/after on realistic multi-tool assistant turns — token counts via tiktoken (o200k_base) on the serialized tool_calls, to show the undercount concretely:

The old walk undercounts a 4-call turn ~4.3× (41 vs 176); the envelope estimate lands within ~16% of the tokenizer, and the gap widens with parallel-call count — precisely when tail protection matters most. These use short arguments; with large arguments the absolute miss is bigger still (matching the 73 vs ~1,090 figure in #28053).

On output quality / information loss — since this moves a compression boundary, worth being explicit rather than hand-waving it:

• It does not change when compression fires. should_compress() gates on real last_prompt_tokens vs threshold_tokens; these budget walks only run inside compress() to place the protect/summarize boundary. Compression frequency is unchanged.
• What changes: on tool-call-heavy turns the tail now reaches tail_token_budget at roughly the intended size instead of ~2× over it, so slightly more of that recent tail becomes eligible for summarization — i.e. it converges on the budget the compressor was designed around.
• The floors are preserved: min_protect / protect_last_n still hold and _ensure_last_user_message_in_tail keeps the most recent user turn verbatim, so the tail can't be stripped below the floor.
• The behavior it removes was itself a quality risk: the over-protection made compression run well below its target ratio, so context kept growing toward the hard window — where the provider truncates uncontrollably (often dropping the system prompt / oldest turns) and "lost-in-the-middle" degrades reasoning. Honoring the budget avoids that failure mode.

Net: this restores the designed tail budget rather than making compression more aggressive. If the effective ~2× over-protection turns out to be desirable, the clean lever is to raise tail_token_budget explicitly — not to lean on the estimation undercount. Happy to add that as a follow-up knob if you'd prefer.

Verification Review

Approach: Correct fix for a long-standing compression budget undercount. The tail-protection walks estimated tokens from content + function.arguments only, dropping the tool_call envelope (id, type, function.name, JSON structure). For parallel tool-call turns this undercounted by 2-15x, causing the protected tail to overshoot tail_token_budget.

Strengths:

• Single extraction point: _estimate_msg_budget_tokens() replaces 3 inline copies, eliminating drift
• Full envelope: counts len(str(tc)) for each tool_call, covering id/type/name/JSON structure
• Defensive: non-dict tool_calls are silently skipped (no crash on malformed data)
• Good test coverage: 5 tests covering envelope vs arguments-only comparison, scaling with parallel calls, plain messages, and non-dict edge cases
• Real-world validation: the tail-cut test uses 20 heavy turns to demonstrate the budget now stops early

No issues found. The fix is well-scoped and the tests validate the exact regression scenario from #28053.