Walkthrough

Adds a docker pull progress watchdog (parsing, dedupe keys, timeouts), tests the stall/max timeout semantics and output tailing, and integrates the watchdog into vLLM image pulls with an increased 12-hour safety budget.

Changes

Docker Pull Watchdog

Sequence Diagram(s)

sequenceDiagram
  participant User
  participant Code as dockerPullWithProgressWatchdog
  participant Spawn as spawnImpl
  participant Docker as "docker pull" child
  User->>Code: dockerPullWithProgressWatchdog(imageRef, opts)
  Code->>Spawn: spawn("docker", ["pull", imageRef], {cwd, env, stdio})
  Spawn-->>Docker: start process
  loop stream lines
    Docker-->>Code: stderr/stdout lines
    Code->>Code: dockerPullProgressSignature(line) -> update lastProgress when new
  end
  alt stall or max timeout exceeded
    Code->>Docker: kill(SIGTERM) then schedule SIGKILL
    Docker-->>Code: close(status 124)
    Code-->>User: resolve(timeoutKind: "stall"|"max", status: 124, output)
  else process exits normally
    Docker-->>Code: close(exit status)
    Code-->>User: resolve(status, output, timeoutKind: null)
  end

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Suggested labels

Docker, fix

Suggested reviewers

• cv
• ericksoa

"I'm a rabbit on a nightly jog,
Watching docker layers in the fog,
I mark new progress, tail the logs,
If stalls appear I wave my flags,
Timeouts sent with gentle hops and clogs."

❌ Failed checks (1 warning)

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

• full-e2e (high; live Docker/OpenShell sandbox plus NVIDIA Endpoints inference): Required baseline because src/lib/inference/vllm.ts is loaded through onboard and the change affects installer/onboarding control flow. This verifies install.sh, onboard, sandbox creation, and live inference still work for the primary user journey.
• gpu-e2e (high; requires NVKS GPU runner and model pull): Closest existing GPU/local-inference E2E. It does not exercise managed vLLM directly, but it validates that local GPU onboarding, Docker GPU runtime setup, sandbox inference wiring, and real assistant flow still work after changes in local inference code.

Optional E2E

• onboard-inference-smoke-e2e (low; hermetic mocked route, no live sandbox): Low-cost regression guard that imports onboard and verifies onboarding fails closed when an inference route is configured but runtime-broken. Useful adjacent coverage for inference/onboard module-loading changes, but it does not cover vLLM image pull behavior.
• inference-routing-e2e (medium; Docker/OpenShell plus selective live provider checks): Adjacent confidence for provider routing through inference.local and credential isolation. Optional because this PR specifically changes managed vLLM image pull/install rather than generic provider routing.

New E2E recommendations

• managed vLLM install / docker pull watchdog (high): Existing E2E coverage appears to exercise Ollama GPU and cloud inference, but not NEMOCLAW_PROVIDER=install-vllm with the managed vLLM pull/download/start/readiness path changed here. A regression in watchdog progress parsing, stall timeout, max timeout, or async pullImage would not be caught by current required jobs.
  • Suggested test: Add a GPU-runner E2E for NEMOCLAW_EXPERIMENTAL=1 NEMOCLAW_PROVIDER=install-vllm that runs install/onboard, pulls or uses a cached vLLM image, waits for /v1/models, registers vllm-local, and proves a sandbox inference request succeeds.
• docker pull watchdog diagnostics (medium): Unit tests cover parsing and timer behavior with fake child processes, but no E2E validates real Docker CLI pull output, bounded diagnostic tails, and failure messaging on an actual runner.
  • Suggested test: Add a PR-safe script E2E using a small public image and a controlled fake/stalled docker wrapper to assert dockerPullWithProgressWatchdog behavior from the built CLI path without requiring the full vLLM image.

Dispatch hint

• Workflow: .github/workflows/nightly-e2e.yaml
• jobs input: full-e2e,gpu-e2e

E2E Scenario Advisor

Base: origin/main
Head: HEAD
Confidence: medium

Required scenario E2E

• ubuntu-repo-cloud-openclaw: Smallest standard scenario that exercises repo-checkout onboarding with Docker running and cloud inference. The Docker adapter barrel exports the changed pull helper, and this scenario catches import/runtime regressions in Docker-backed onboarding and sandbox startup; no dispatchable ROUTES scenario directly exercises managed vLLM install.
  • Dispatch: gh workflow run e2e-scenarios.yaml --ref <pr-head-ref> --field scenarios=ubuntu-repo-cloud-openclaw

Optional scenario E2E

• gpu-repo-local-ollama-openclaw: Adjacent GPU/Docker local-inference path and closest available special-runner coverage for containerized local inference changes, though it uses Ollama rather than the changed vLLM install path.
  • Dispatch: gh workflow run e2e-scenarios.yaml --ref <pr-head-ref> --field scenarios=gpu-repo-local-ollama-openclaw

Relevant changed files

• src/lib/adapters/docker/pull.ts
• src/lib/inference/vllm.ts

🛠️ Needs attention

• None.

🔎 Worth checking

• Add vLLM caller coverage for watchdog wiring (src/lib/inference/vllm.ts:230): The new Docker pull watchdog helper has direct unit coverage, but the production vLLM install path changed from a synchronous shell timeout to an awaited watchdog call without caller-level tests. That leaves the caller contract unverified: passing the profile image, mapping pullTimeoutSec to maxTimeoutMs, forwarding progress logging, awaiting the pull before later install steps, and converting stall/max failures into user-facing reasons.
  • Recommendation: Add a focused unit test around pullImage() or installVllm() with dockerPullWithProgressWatchdog mocked. Cover at least success, stall timeout, max timeout, generic non-zero failure, and aborting later install steps after pull failure. Also identify targeted runtime/sandbox validation for real Docker pull progress output and signal behavior.
  • Evidence: Repository search found watchdog tests only in src/lib/adapters/docker/pull.test.ts. The changed caller now does await dockerPullWithProgressWatchdog(profile.image, { maxTimeoutMs: profile.pullTimeoutSec * 1000, logLine: emit }) and maps timeoutKind values in src/lib/inference/vllm.ts:230-244, but no test exercises pullImage() or installVllm() wiring.

🌱 Nice ideas

• None.

Current findings:

• Add vLLM caller coverage for watchdog wiring (src/lib/inference/vllm.ts:230): The new Docker pull watchdog helper has direct unit coverage, but the production vLLM install path changed from a synchronous shell timeout to an awaited watchdog call without caller-level tests. That leaves the caller contract unverified: passing the profile image, mapping pullTimeoutSec to maxTimeoutMs, forwarding progress logging, awaiting the pull before later install steps, and converting stall/max failures into user-facing reasons.
  • Recommendation: Add a focused unit test around pullImage() or installVllm() with dockerPullWithProgressWatchdog mocked. Cover at least success, stall timeout, max timeout, generic non-zero failure, and aborting later install steps after pull failure. Also identify targeted runtime/sandbox validation for real Docker pull progress output and signal behavior.
  • Evidence: Repository search found watchdog tests only in src/lib/adapters/docker/pull.test.ts. The changed caller now does await dockerPullWithProgressWatchdog(profile.image, { maxTimeoutMs: profile.pullTimeoutSec * 1000, logLine: emit }) and maps timeoutKind values in src/lib/inference/vllm.ts:230-244, but no test exercises pullImage() or installVllm() wiring.