Skip to content

fix(model-metadata): prioritize curated defaults over OpenRouter for known providers#23950

Closed
nicoechaniz wants to merge 1 commit into
NousResearch:mainfrom
nicoechaniz:fix/kimi-context-length-resolution
Closed

fix(model-metadata): prioritize curated defaults over OpenRouter for known providers#23950
nicoechaniz wants to merge 1 commit into
NousResearch:mainfrom
nicoechaniz:fix/kimi-context-length-resolution

Conversation

@nicoechaniz

@nicoechaniz nicoechaniz commented May 11, 2026

Copy link
Copy Markdown
Contributor

Problem

Users configuring provider: kimi directly (not through OpenRouter) get a context-window rejection:

Model kimi-k2.6 has a context window of 32,768 tokens, which is below the minimum 64,000 required by Hermes Agent.

The real context window is 262,144 tokens. 32,768 comes from OpenRouter's community-maintained metadata, which the resolution chain consults before the project's own curated DEFAULT_CONTEXT_LENGTHS table.

Root Cause

Two issues in the context-length resolution chain (agent/model_metadata.py):

  1. PROVIDER_TO_MODELS_DEV was missing "kimi" and "moonshot" entries — only kimi-coding and kimi-coding-cn were mapped. Users setting provider: kimi got None from the models.dev lookup.

  2. After models.dev returned None, the chain fell through to OpenRouter (step 6) before reaching DEFAULT_CONTEXT_LENGTHS (step 8). OpenRouter reports 32,768 for kimi-k2.6, which is incorrect but was accepted as authoritative because the curated defaults were never consulted.

Fix

  1. Add "kimi" and "moonshot" to PROVIDER_TO_MODELS_DEV → "kimi-for-coding". This is consistent with the existing kimi-coding and kimi-coding-cn entries. Functionally, models.dev still uses different model IDs (k2p6/k2p5), so the lookup usually misses — but the mapping now exists for consistency.

  2. Gate OpenRouter behind not effective_provider. If the provider is known (inferred from URL or set in config), skip OpenRouter's community-maintained metadata and fall through to the project's own curated DEFAULT_CONTEXT_LENGTHS table. OpenRouter data can be incorrect and should not override the project's defaults for models belonging to known providers.

  3. Add explicit DEFAULT_CONTEXT_LENGTHS entries for kimi-k2.6, kimi-k2.5, kimi-k2, k2p6, k2p5 (all 262,144) as a safety net.

Behavior changes

Scenario Before After
provider: kimi, model kimi-k2.6 32,768 ❌ 262,144 ✅
provider: kimi, model kimi-k2.5 32,768 ❌ 262,144 ✅
provider: openrouter, model kimi-k2.6 32,768 262,144¹
Unknown provider, model kimi-k2.6 32,768 32,768²

¹ Fixed via models.dev now resolving the kimi provider mapping.
² Unchanged — OpenRouter data is the only available source for unknown providers.

Testing

  • All 94 tests in tests/agent/test_model_metadata.py pass
  • All 47 tests in tests/agent/test_models_dev.py and tests/agent/test_kimi_coding_anthropic_thinking.py pass

@nicoechaniz
fix(model-metadata): prioritize curated defaults over OpenRouter for …
c67de94 
…known providers

PROVIDER_TO_MODELS_DEV was missing 'kimi' and 'moonshot' provider
mappings, causing the models.dev lookup to return None for users who
configure provider: kimi directly.

The context-length resolution chain then fell through to OpenRouter's
community-maintained metadata (step 6), which reports 32,768 tokens for
kimi-k2.6 — incorrect; the real context window is 262,144 tokens.

Fix:

1. Add 'kimi' and 'moonshot' to PROVIDER_TO_MODELS_DEV pointing to
   kimi-for-coding, matching the existing kimi-coding/kimi-coding-cn
   entries.

2. Gate OpenRouter fallback behind 'not effective_provider' so that
   known providers skip third-party metadata and go straight to the
   project's curated DEFAULT_CONTEXT_LENGTHS table.  OpenRouter data
   is community-maintained and should not override the project's own
   defaults for models belonging to known providers.

3. Add explicit DEFAULT_CONTEXT_LENGTHS entries for kimi-k2.6,
   kimi-k2.5, kimi-k2, k2p6, and k2p5 (all 262,144) as a safety net.

Users who set provider: kimi in config.yaml now resolve to 262,144
tokens without needing an explicit model.context_length override.
OpenRouter users and unknown-provider paths are unchanged.
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent loop, run_agent.py, prompt builder provider/kimi Kimi / Moonshot provider/ollama Ollama / local models P2 Medium — degraded but workaround exists labels May 11, 2026
kshitijk4poor pushed a commit to kshitijk4poor/hermes-agent that referenced this pull request May 11, 2026
@nicoechaniz @kshitijk4poor
fix(model-metadata): skip OpenRouter for known providers, add kimi/mo…
a4f143f 
…onshot to PROVIDER_TO_MODELS_DEV

Based on PR NousResearch#23950 by @nicoechaniz.

- Add "kimi" and "moonshot" to PROVIDER_TO_MODELS_DEV → kimi-for-coding
- Gate OpenRouter metadata step behind "if not effective_provider":
  known providers should not be overridden by community-maintained OR data
- Keep the targeted Kimi-family 32k guard as a secondary safety net
  inside the OR gate (for unknown providers with Kimi models)

Co-authored-by: nicoechaniz <nicoechaniz@altermundi.net>
@kshitijk4poor kshitijk4poor mentioned this pull request May 11, 2026
Merged
@kshitijk4poor

kshitijk4poor commented May 11, 2026

Copy link
Copy Markdown
Collaborator

Merged via PR #23980 with your authorship preserved. Thank you @nicoechaniz!

Kept from your PR:

  • "kimi" and "moonshot" → kimi-for-coding in PROVIDER_TO_MODELS_DEV
  • OpenRouter gated behind if not effective_provider: for known providers

Note: the DEFAULT_CONTEXT_LENGTHS entries (kimi-k2.6, k2p6, k2p5) were skipped — the existing "kimi": 262144 catch-all already covers these, and the new /api/show probe provides the authoritative source for Ollama-hosted models.

kshitijk4poor pushed a commit that referenced this pull request May 11, 2026
@nicoechaniz @kshitijk4poor
fix(model-metadata): skip OpenRouter for known providers, add kimi/mo…
e2b713c 
…onshot to PROVIDER_TO_MODELS_DEV

Based on PR #23950 by @nicoechaniz.

- Add "kimi" and "moonshot" to PROVIDER_TO_MODELS_DEV → kimi-for-coding
- Gate OpenRouter metadata step behind "if not effective_provider":
  known providers should not be overridden by community-maintained OR data
- Keep the targeted Kimi-family 32k guard as a secondary safety net
  inside the OR gate (for unknown providers with Kimi models)

Co-authored-by: nicoechaniz <nicoechaniz@altermundi.net>
@briandevans briandevans mentioned this pull request May 12, 2026
Closed
4 tasks
@nicoechaniz nicoechaniz deleted the fix/kimi-context-length-resolution branch May 13, 2026 21:05
02356abc pushed a commit to 02356abc/hermes-agent that referenced this pull request May 14, 2026
@nicoechaniz @kshitijk4poor
fix(model-metadata): skip OpenRouter for known providers, add kimi/mo…
9f1951d 
…onshot to PROVIDER_TO_MODELS_DEV

Based on PR NousResearch#23950 by @nicoechaniz.

- Add "kimi" and "moonshot" to PROVIDER_TO_MODELS_DEV → kimi-for-coding
- Gate OpenRouter metadata step behind "if not effective_provider":
  known providers should not be overridden by community-maintained OR data
- Keep the targeted Kimi-family 32k guard as a secondary safety net
  inside the OR gate (for unknown providers with Kimi models)

Co-authored-by: nicoechaniz <nicoechaniz@altermundi.net>
jsboige pushed a commit to jsboige/hermes-agent that referenced this pull request May 14, 2026
@nicoechaniz @kshitijk4poor
fix(model-metadata): skip OpenRouter for known providers, add kimi/mo…
99eb2a2 
…onshot to PROVIDER_TO_MODELS_DEV

Based on PR NousResearch#23950 by @nicoechaniz.

- Add "kimi" and "moonshot" to PROVIDER_TO_MODELS_DEV → kimi-for-coding
- Gate OpenRouter metadata step behind "if not effective_provider":
  known providers should not be overridden by community-maintained OR data
- Keep the targeted Kimi-family 32k guard as a secondary safety net
  inside the OR gate (for unknown providers with Kimi models)

Co-authored-by: nicoechaniz <nicoechaniz@altermundi.net>
@kshitijk4poor kshitijk4poor mentioned this pull request May 17, 2026
Closed
AlexFoxD pushed a commit to AlexFoxD/hermes-agent that referenced this pull request May 21, 2026
@nicoechaniz @AlexFoxD
fix(model-metadata): skip OpenRouter for known providers, add kimi/mo…
518d2b6 
…onshot to PROVIDER_TO_MODELS_DEV

Based on PR NousResearch#23950 by @nicoechaniz.

- Add "kimi" and "moonshot" to PROVIDER_TO_MODELS_DEV → kimi-for-coding
- Gate OpenRouter metadata step behind "if not effective_provider":
  known providers should not be overridden by community-maintained OR data
- Keep the targeted Kimi-family 32k guard as a secondary safety net
  inside the OR gate (for unknown providers with Kimi models)

Co-authored-by: nicoechaniz <nicoechaniz@altermundi.net>
gweeteve pushed a commit to gweeteve/hermes-agent that referenced this pull request Jun 2, 2026
@nicoechaniz @kshitijk4poor
fix(model-metadata): skip OpenRouter for known providers, add kimi/mo…
14c591f 
…onshot to PROVIDER_TO_MODELS_DEV

Based on PR NousResearch#23950 by @nicoechaniz.

- Add "kimi" and "moonshot" to PROVIDER_TO_MODELS_DEV → kimi-for-coding
- Gate OpenRouter metadata step behind "if not effective_provider":
  known providers should not be overridden by community-maintained OR data
- Keep the targeted Kimi-family 32k guard as a secondary safety net
  inside the OR gate (for unknown providers with Kimi models)

Co-authored-by: nicoechaniz <nicoechaniz@altermundi.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

comp/agent Core agent loop, run_agent.py, prompt builder P2 Medium — degraded but workaround exists provider/kimi Kimi / Moonshot provider/ollama Ollama / local models type/bug Something isn't working

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@nicoechaniz @kshitijk4poor @alt-glitch