Walkthrough

This PR adds vLLM runtime context window auto-detection: a helper reads max_model_len from vLLM /v1/models, validates it, and sets NEMOCLAW_CONTEXT_WINDOW when appropriate. The helper is exported via local.ts, invoked during onboarding, and covered by unit and integration tests.

Changes

vLLM Runtime Context Window Auto-Detection

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Suggested labels

Local Models, fix, enhancement: inference

Suggested reviewers

• cv

Poem

🐇 I found a model list in a vLLM stream,
max_model_len shining like a dream,
I parsed with care, warned when a value was odd,
Set the window when sensible — nod, nod, nod,
Hoppity hops, the context fits the team.

❌ Failed checks (1 warning)

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: medium

Required E2E

• cloud-onboard-e2e (medium): Runs the real install/onboard flow on a clean runner. Although it uses cloud inference rather than vLLM, it is the closest existing merge-blocking E2E for src/lib/onboard.ts changes and guards against regressions in provider selection/onboarding state.
• inference-routing-e2e (medium): Validates end-to-end inference routing through OpenShell gateway/proxy paths and provider configuration. The changed local inference/context-window plumbing can affect runtime inference behavior, so this provides the best existing route-level E2E signal.

Optional E2E

• onboard-inference-smoke-e2e (low): Hermetic regression E2E for onboard inference validation and diagnostics. Useful adjacent confidence for onboarding/inference changes, but it does not exercise the local vLLM /v1/models max_model_len path directly.
• gpu-e2e (high): Exercises real local inference onboarding and sandbox inference on a GPU runner, but it uses Ollama rather than vLLM. Run if extra confidence is desired for local-provider wiring and context/environment interactions.

New E2E recommendations

• local vLLM onboarding (high): No existing E2E appears to start or mock a vLLM-compatible /v1/models endpoint during real onboarding and assert that max_model_len becomes NEMOCLAW_CONTEXT_WINDOW only after validation succeeds and is preserved through sandbox inference.
  • Suggested test: Add a local-vllm-context-window-e2e job that runs onboarding with NEMOCLAW_PROVIDER=vllm against a hermetic vLLM/OpenAI-compatible mock returning /v1/models with max_model_len, verifies the configured context window, and performs an inference.local chat completion from the sandbox.

Dispatch hint

• Workflow: nightly-e2e.yaml
• jobs input: cloud-onboard-e2e,inference-routing-e2e

E2E Scenario Advisor

Base: origin/main
Head: HEAD
Confidence: medium

Required scenario E2E

• None.

Optional scenario E2E

• ubuntu-repo-cloud-openclaw: The PR changes onboarding and local inference helper code, but the modified path is specific to local vLLM runtime context detection and there is no vLLM scenario ID in the dispatchable ROUTES table. This Ubuntu repo cloud OpenClaw scenario is the closest low-cost adjacent coverage for onboarding, gateway, sandbox, and inference-local wiring.
  • Dispatch: gh workflow run e2e-scenarios.yaml --ref <pr-head-ref> --field scenarios=ubuntu-repo-cloud-openclaw

Relevant changed files

• src/lib/inference/local.ts
• src/lib/inference/vllm-runtime-context.ts
• src/lib/onboard.ts

🛠️ Needs attention

• None.

🔎 Worth checking

• Source-of-truth review needed: src/lib/inference/vllm-runtime-context.ts tolerant parsing of vLLM /v1/models max_model_len: The advisor marked localized patch analysis as needs_followup.
  • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Evidence: The helper implements compatibility no-ops and warnings, while the analogous Ollama helper includes a comment documenting source boundary, tolerated states, regression coverage, and removal condition.
• Document the source boundary for tolerant vLLM max_model_len parsing (src/lib/inference/vllm-runtime-context.ts:26): The helper intentionally tolerates missing, empty, malformed, non-positive, unsafe, and over-ceiling max_model_len values from the vLLM /v1/models response. The invalid states are tested, but the code does not record the source-of-truth rationale or removal condition. That leaves future maintainers without the same context already present in the analogous Ollama runtime-context helper.
  • Recommendation: Add a short comment near the parsing logic that identifies the external vLLM-compatible /v1/models boundary, the tolerated invalid states, why NemoClaw cannot enforce that producer contract in this PR, the regression coverage, and when the fallback/no-op behavior can be removed.
  • Evidence: src/lib/inference/vllm-runtime-context.ts silently returns for absent/empty max_model_len and warns for malformed or over-ceiling values; src/lib/inference/ollama-runtime-context.ts documents the same class of compatibility tolerance with source boundary, regression coverage, and removal condition.

🌱 Nice ideas

• None.

Current findings:

• Source-of-truth review needed: src/lib/inference/vllm-runtime-context.ts tolerant parsing of vLLM /v1/models max_model_len: The advisor marked localized patch analysis as needs_followup.
  • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Evidence: The helper implements compatibility no-ops and warnings, while the analogous Ollama helper includes a comment documenting source boundary, tolerated states, regression coverage, and removal condition.
• Document the source boundary for tolerant vLLM max_model_len parsing (src/lib/inference/vllm-runtime-context.ts:26): The helper intentionally tolerates missing, empty, malformed, non-positive, unsafe, and over-ceiling max_model_len values from the vLLM /v1/models response. The invalid states are tested, but the code does not record the source-of-truth rationale or removal condition. That leaves future maintainers without the same context already present in the analogous Ollama runtime-context helper.
  • Recommendation: Add a short comment near the parsing logic that identifies the external vLLM-compatible /v1/models boundary, the tolerated invalid states, why NemoClaw cannot enforce that producer contract in this PR, the regression coverage, and when the fallback/no-op behavior can be removed.
  • Evidence: src/lib/inference/vllm-runtime-context.ts silently returns for absent/empty max_model_len and warns for malformed or over-ceiling values; src/lib/inference/ollama-runtime-context.ts documents the same class of compatibility tolerance with source boundary, regression coverage, and removal condition.