Skip to content

fix(model_metadata): add xAI Grok context length fallbacks#7093

Merged
teknium1 merged 1 commit into
mainfrom
hermes/hermes-5bbf4839
Apr 10, 2026
Merged

fix(model_metadata): add xAI Grok context length fallbacks#7093
teknium1 merged 1 commit into
mainfrom
hermes/hermes-5bbf4839

Conversation

@teknium1

Copy link
Copy Markdown
Contributor

Cherry-picked from #7039 by @Julientalbot onto current main.

Summary

xAI's /v1/models endpoint does not return context_length metadata. Users pointing at https://api.x.ai/v1 via a custom provider fall through to the 128k probe-down default, losing up to 93% of the usable window (e.g. 128k instead of 2M for grok-4.20).

Adds DEFAULT_CONTEXT_LENGTHS entries for the Grok family — same pattern as Claude, Gemma, MiniMax, Kimi, and GLM.

Test plan

pytest tests/agent/test_model_metadata.py — 77 passed

xAI /v1/models does not return context_length metadata, so Hermes
probes down to the 128k default whenever a user configures a custom
provider pointing at https://api.x.ai/v1. This forces every xAI user
to manually override model.context_length in config.yaml (2M for
Grok 4.20 / 4.1-fast / 4-fast) or lose most of the usable context
window.

Add DEFAULT_CONTEXT_LENGTHS entries for the Grok family so the
fallback lookup returns the correct value via substring matching.
Values sourced from models.dev (2026-04) and cross-checked against
the xAI /v1/models listing:

  - grok-4.20-*          2,000,000  (reasoning, non-reasoning, multi-agent)
  - grok-4-1-fast-*      2,000,000
  - grok-4-fast-*        2,000,000
  - grok-4 / grok-4-0709   256,000
  - grok-code-fast-1       256,000
  - grok-3*                131,072
  - grok-2 / latest        131,072
  - grok-2-vision*           8,192
  - grok (catch-all)       131,072

Keys are ordered longest-first so that specific variants match before
the catch-all, consistent with the existing Claude/Gemma/MiniMax entries.

Add TestDefaultContextLengths.test_grok_models_context_lengths and
test_grok_substring_matching to pin the values and verify the full
lookup path. All 77 tests in test_model_metadata.py pass.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants