Walkthrough

Adds available GPU memory (availableMemoryMB) to detection, a registry of Ollama models with memory/download metadata, and refactors local model selection, prompts, non-interactive onboarding, and model-size fallbacks to prefer models that fit currently available memory.

Changes

Available Memory Detection and Registry-Based Model Selection

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related issues

• [DGX Spark][Ollama] Onboarding selects qwen3.6:35b despite insufficient currently available GPU memory #4113: Addresses the same root cause by routing non-interactive selection through GPU-aware helpers to avoid choosing qwen3.6 when availableMemoryMB is insufficient.

Suggested labels

Provider: Ollama, documentation, enhancement: testing, v0.0.50

Suggested reviewers

• ericksoa
• cv
• jyaunches

Poem

🐇 I hopped through GPUs late at night,
Counting free memory by soft moonlight.
When giants don't fit the space at hand,
I nudge you toward a smaller, safer land.
Models load happy — the rabbit's delight.

❌ Failed checks (1 warning)

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

• gpu-e2e (high (~30 minutes; self-hosted GPU)): Required because the PR changes real local Ollama onboarding/model selection and GPU-memory detection. This job exercises the source install, NEMOCLAW_PROVIDER=ollama onboarding, Ollama install/start/pull, auth proxy setup, sandbox creation, and live sandbox inference on a NVIDIA GPU runner, catching regressions in the changed detectGpu → model selection → validation path.

Optional E2E

• gpu-double-onboard-e2e (high (~30 minutes; self-hosted GPU)): Useful adjacent coverage for re-running Ollama onboarding with an existing sandbox/token and recovered model state. The PR changes non-interactive and recovered/requested Ollama model resolution, but proxy token consistency itself is not the primary diff, so this is optional rather than merge-blocking.
• ollama-proxy-e2e (medium (~15 minutes; installs/pulls small Ollama model)): Useful lower-scope validation because src/lib/inference/ollama/proxy.ts was touched and the local Ollama flow depends on the authenticated proxy. It validates real Ollama inference through the proxy, token enforcement, persistence, recovery, and container reachability, but it does not directly exercise GPU available-memory model selection.
• gpu-repo-local-ollama-openclaw (high (self-hosted GPU scenario)): Scenario-runner equivalent coverage for the local Ollama OpenClaw profile with smoke, local-ollama-inference, and ollama-proxy suites. Good confidence if validating the newer scenario framework, but overlaps with gpu-e2e for this PR.

New E2E recommendations

• local Ollama available-memory downgrade (high): Existing GPU E2E normally runs on an idle GPU and validates the happy path, but it is unlikely to exercise the new behavior where total memory is large while currently available memory is too low and onboarding downgrades from qwen3.6:35b/nemotron to qwen2.5:7b.
  • Suggested test: Add a local Ollama onboarding E2E that creates a low-available-memory condition (for example by pre-allocating GPU memory or by using a controlled nvidia-smi/free shim in an E2E harness), runs non-interactive NEMOCLAW_PROVIDER=ollama onboarding, asserts the oversize-model warning/fallback, and verifies sandbox inference succeeds with the fallback model.
• macOS Apple Silicon local Ollama sizing (medium): The PR adds macOS vm_stat-based availableMemoryMB handling, but the existing macOS scenario is cloud/Docker-optional and does not validate local Ollama model sizing on Apple Silicon.
  • Suggested test: Add a macOS local Ollama onboarding or dry-run E2E/assertion that verifies Apple Silicon available-memory detection influences the starter model menu/default without requiring Docker-dependent sandbox suites on GitHub-hosted macOS.

Dispatch hint

• Workflow: .github/workflows/nightly-e2e.yaml
• jobs input: gpu-e2e

E2E Scenario Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required scenario E2E

• None. No scenario workflow, scenario metadata, scenario runtime, or validation-suite files changed.

Optional scenario E2E

• None.

Relevant changed files

• None.

🛠️ Needs attention

• Inference/onboarding hotspots grew further instead of being extracted: Codebase drift check: the PR patches files that still exist and have active recent history, including inference/network and onboarding host-glue paths. The behavioral change is in-scope, but the implementation continues adding memory-detection and model-selection logic to already-large security-adjacent files rather than extracting focused helpers. This keeps Docker/NIM detection, local inference networking, Ollama proxy behavior, and onboarding selection glue concentrated in monoliths, increasing future review and regression risk.
  • Recommendation: Extract Ollama selection policy and GPU available-memory probing/parsing into focused modules, and move new local/nim/proxy regression coverage out of the large hotspot test files where practical. At minimum, offset this PR's hotspot growth by extracting existing helper code before merge.
  • Evidence: This is the prior advisor hotspot finding and it still applies. Trusted monolith deltas show src/lib/inference/nim.test.ts 1241→1361 (+120), src/lib/inference/local.test.ts 839→953 (+114), src/lib/inference/nim.ts 710→794 (+84), src/lib/inference/local.ts 1014→1089 (+75), and src/lib/inference/ollama/proxy.ts 812→834 (+22). Drift evidence confirms these files still exist and have active recent history; src/lib/onboard.ts also overlaps many open PRs even though this patch is net-zero there.

🔎 Worth checking

• Missing explicit unified-memory fallback coverage for absent or malformed MemAvailable (src/lib/inference/nim.test.ts:633): The new tests cover availableMemoryMB propagation on primary NVIDIA, GB10/Spark, Orin, Jetson/Tegra, macOS, and a primary-path memory.free parse failure. However, the unified-memory Linux path still has a fixture whose `free -m` output lacks the `available` column without asserting the resulting `availableMemoryMB` contract. A regression in `readHostAvailableMemoryMB` could silently treat malformed output as usable or unintentionally size against an invalid value.
  • Recommendation: Add a focused `detectGpu` test for a unified-memory/Spark or Jetson path where `free -m` lacks or malforms the `available` column, and assert that `availableMemoryMB` is omitted so downstream selection intentionally falls back to `totalMemoryMB`. Keep the positive propagation and primary-path parse-failure assertions already added.
  • Evidence: This prior advisor finding still applies. The mixed unified-memory fixture in `src/lib/inference/nim.test.ts` returns `free -m` output with only `total used free`, but only asserts name/gpus behavior. The PR added `omits availableMemoryMB when memory.free fails to parse on the primary path` and macOS parse-failure coverage, but not the requested unified-memory `MemAvailable` fallback assertion.

🌱 Nice ideas

• None.

Current findings:

• Inference/onboarding hotspots grew further instead of being extracted: Codebase drift check: the PR patches files that still exist and have active recent history, including inference/network and onboarding host-glue paths. The behavioral change is in-scope, but the implementation continues adding memory-detection and model-selection logic to already-large security-adjacent files rather than extracting focused helpers. This keeps Docker/NIM detection, local inference networking, Ollama proxy behavior, and onboarding selection glue concentrated in monoliths, increasing future review and regression risk.
  • Recommendation: Extract Ollama selection policy and GPU available-memory probing/parsing into focused modules, and move new local/nim/proxy regression coverage out of the large hotspot test files where practical. At minimum, offset this PR's hotspot growth by extracting existing helper code before merge.
  • Evidence: This is the prior advisor hotspot finding and it still applies. Trusted monolith deltas show src/lib/inference/nim.test.ts 1241→1361 (+120), src/lib/inference/local.test.ts 839→953 (+114), src/lib/inference/nim.ts 710→794 (+84), src/lib/inference/local.ts 1014→1089 (+75), and src/lib/inference/ollama/proxy.ts 812→834 (+22). Drift evidence confirms these files still exist and have active recent history; src/lib/onboard.ts also overlaps many open PRs even though this patch is net-zero there.
• Missing explicit unified-memory fallback coverage for absent or malformed MemAvailable (src/lib/inference/nim.test.ts:633): The new tests cover availableMemoryMB propagation on primary NVIDIA, GB10/Spark, Orin, Jetson/Tegra, macOS, and a primary-path memory.free parse failure. However, the unified-memory Linux path still has a fixture whose `free -m` output lacks the `available` column without asserting the resulting `availableMemoryMB` contract. A regression in `readHostAvailableMemoryMB` could silently treat malformed output as usable or unintentionally size against an invalid value.
  • Recommendation: Add a focused `detectGpu` test for a unified-memory/Spark or Jetson path where `free -m` lacks or malforms the `available` column, and assert that `availableMemoryMB` is omitted so downstream selection intentionally falls back to `totalMemoryMB`. Keep the positive propagation and primary-path parse-failure assertions already added.
  • Evidence: This prior advisor finding still applies. The mixed unified-memory fixture in `src/lib/inference/nim.test.ts` returns `free -m` output with only `total used free`, but only asserts name/gpus behavior. The PR added `omits availableMemoryMB when memory.free fails to parse on the primary path` and macOS parse-failure coverage, but not the requested unified-memory `MemAvailable` fallback assertion.