Skip to content

Bug: llama.cpp router mode reports n_ctx=0, treated as valid context window #1280

@Aaronontheweb

Description

@Aaronontheweb

Problem

When llama.cpp runs in router mode, the /v1/models and /props endpoints do not populate context window metadata. The probe returns n_ctx: 0 (unset/sentinel value), which NetClaw treats as a legitimate context window of 0 tokens.

Root Cause

ProbeHelpers.TryReadInt32() returns the raw integer value when present — including 0:

internal static int? TryReadInt32(JsonElement element, string propertyName)
{
    return element.TryGetProperty(propertyName, out var property) &&
           property.ValueKind == JsonValueKind.Number &&
           property.TryGetInt32(out var value)
        ? value  // returns 0, not null!
        : null;
}

Then in ModelCapabilityResolution.ResolveModelCapabilities(), the validation check:

if (model.ContextWindow is int configured
    && detected?.ContextWindowTokens is int detected  // 0 matches is int!
    && configured > detected)  // 120000 > 0 → throws
{
    throw new InvalidOperationException(
        $"Models:Main:ContextWindow ({configured}) exceeds the " +
        $"provider-reported effective context window ({detected}).");
}

is int matches 0 as a valid integer, so the validation incorrectly concludes the user-set context window exceeds the provider-reported value.

Impact

  • Users running llama.cpp in router mode see startup failures with confusing error messages
  • Users running llama.cpp with models that expose n_ctx: 0 in metadata hit the same issue
  • The error message says to "Reduce the configured ContextWindow" — but the real fix is either setting a correct ContextWindow in config OR fixing the zero-detection logic

Reproduction

  1. Start llama.cpp in router mode (e.g., llama-router)
  2. Configure NetClaw with a model that has an explicit ContextWindow set
  3. Observe: daemon refuses to start with "ContextWindow exceeds the provider-reported effective context window (0)"

Proposed Fix

Either:

  1. In TryReadInt32: Return null when the parsed value is 0 (value > 0 guard)
  2. In LlamaCppBackendStrategy: Treat n_ctx == 0 as "not reported" and return null
  3. In ModelCapabilityResolution: Skip the validation check when detectedContextWindow == 0

Option 1 is the cleanest since n_ctx: 0 in llama.cpp always means "unset" — it is never a legitimate context window size.

Tweet Reference

User reported: https://x.com/rare47/status/2061456687500775842

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingtoolsIssues related to agent tools: file_read, web_search, shell_execute, image processing, etc.

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions