Skip to content

refactor(gateway): use get_custom_provider_context_length() helper + pass custom_providers to @context path#18844

Open
quinnmacro wants to merge 1 commit into
NousResearch:mainfrom
quinnmacro:refactor/custom-providers-context-length-helper
Open

refactor(gateway): use get_custom_provider_context_length() helper + pass custom_providers to @context path#18844
quinnmacro wants to merge 1 commit into
NousResearch:mainfrom
quinnmacro:refactor/custom-providers-context-length-helper

Conversation

@quinnmacro

@quinnmacro quinnmacro commented May 2, 2026 •
edited
Loading

Copy link
Copy Markdown

Summary

Three targeted fixes for the custom_providers context-length resolution chain:

1. Bug fix: @context expansion path ignores custom_providers

The @context reference expansion path calls get_model_context_length() without passing custom_providers. This skips step 0b (per-model context_length from custom_providers), so users who configure custom_providers[].models.<m>.context_length without a top-level model.context_length get wrong injection limits.

2. Refactor: replace inline parsing with get_custom_provider_context_length() helper

The hygiene-run path has a 27-line inline custom_providers parsing loop. PR #15844 introduced get_custom_provider_context_length() as the single source of truth. This replaces the inline loop with a helper call.

3. Bug fix: run_conversation() result dict missing context_length field (NEW)

The gateway runtime_footer reads context_length from agent_result, but run_conversation() never included this field in its return dict. This caused the footer's context_pct display to always be skipped (None).

The CLI status bar works fine because _get_status_bar_snapshot() reads context_length directly from context_compressor. But the gateway footer only sees the result dict — so agent_result.get("context_length") was always None.

Fix: Add "context_length" to the result dict returned by run_conversation(), derived from context_compressor.context_length, matching the same source the CLI uses.

New commit: fix/context-length-result-dict branch (1 line change in run_agent.py).

Related

Test Plan

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/gateway Gateway runner, session dispatch, delivery labels May 2, 2026
@alt-glitch

alt-glitch commented May 2, 2026

Copy link
Copy Markdown
Collaborator

Related to #15844 (merged), which introduced the helper this PR now uses in the remaining code paths.

1 similar comment
@alt-glitch

alt-glitch commented May 2, 2026

Copy link
Copy Markdown
Collaborator

Related to #15844 (merged), which introduced the helper this PR now uses in the remaining code paths.

@dinedal dinedal mentioned this pull request May 3, 2026
Closed
19 tasks
@quinnmacro

quinnmacro commented May 9, 2026

Copy link
Copy Markdown
Author

New commit: fix/context-length-result-dict

While investigating why the gateway runtime footer still shows blank context_pct after #15844, I found a deeper bug:

Root cause: run_conversation() in run_agent.py never includes "context_length" in its return dict. The gateway footer reads context_length from agent_result.get("context_length") (run.py L6297), which is always None.

The CLI status bar works because _get_status_bar_snapshot() reads context_length directly from context_compressor — it never goes through the result dict. So #15844 fixed the CLI path but left the gateway path broken.

Fix: 1-line addition in run_agent.py L14637:

"context_length": getattr(self.context_compressor, "context_length", 0) or 0,

Branch: fix/context-length-result-dict on quinnmacro/hermes-upstream-fork

@quinnmacro quinnmacro force-pushed the refactor/custom-providers-context-length-helper branch from f0918e8 to b764842 Compare May 15, 2026 00:58
@quinnmacro quinnmacro force-pushed the refactor/custom-providers-context-length-helper branch 2 times, most recently from 1549dba to 1c7ebc2 Compare May 30, 2026 07:50
@quinnmacro

quinnmacro commented May 30, 2026

Copy link
Copy Markdown
Author

Rebased onto latest upstream/main (v2026.5.29)

Clean single commit: +1 line in agent/conversation_loop.py (moved from run_agent.py per the 3-module refactor).

What this fixes (real-world impact)

The gateway runtime footer shows context usage percentage (context_pct) for every message. Since v0.15.x, it's been blank for all users — not because the data is missing, but because run_conversation() returns a dict that doesn't include context_length.

The CLI status bar works fine because it reads context_compressor.context_length directly. But the gateway path goes through the result dict, so agent_result.get("context_length") returns None, and the footer skips the display entirely.

The fix

One line, zero side effects:

"context_length": getattr(agent.context_compressor, "context_length", 0) or 0,

This matches the exact same source the CLI uses (context_compressor.context_length), so the gateway and CLI now show consistent numbers.

What I verified

Path Before After
Gateway footer context_pct Always blank (None) Shows correct %
CLI status bar Already correct Unchanged
/modelinfo command Already correct Unchanged
/model switch display Already correct Unchanged

This is the last remaining gap from the custom_providers context-length work (#15844 merged). Ready for merge whenever.

@banditburai banditburai mentioned this pull request Jun 1, 2026
Open
@quinnmacro
fix(gateway): add context_length to run_conversation result dict
d4f02d1 
The gateway footer reads agent_result.get('context_length') to display
context usage percentage. After the 3-module refactor (v2026.5.29), the
result dict was extracted to agent/turn_finalizer.py — but context_length
was never included, leaving the gateway footer blank.

The CLI status bar works fine because it reads context_compressor directly.
This 1-line fix makes gateway and CLI consistent.

Adapted to v2026.6.5: result dict now in agent/turn_finalizer.py.
@quinnmacro quinnmacro force-pushed the refactor/custom-providers-context-length-helper branch from 1c7ebc2 to d4f02d1 Compare June 11, 2026 08:10
@quinnmacro

quinnmacro commented Jun 11, 2026

Copy link
Copy Markdown
Author

Rebased onto v2026.6.5 (commit c94e93a)

Adapted to the turn_finalizer.py extraction refactor:

  • context_length field added to result dict in agent/turn_finalizer.py (was conversation_loop.py)
  • 1-line fix, zero side effects

Gateway footer context_pct now shows correct value. Diff: +1/-0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Bug: /model switch to named custom provider ignores custom_providers model context_length

2 participants

@quinnmacro @alt-glitch