Skip to content

[MacOS] No clear error message when Ollama backend is stopped #995

@zNeill

Description

@zNeill

Description

When the inference backend (Ollama) is stopped and a request is made to inference.local from inside the sandbox, the OpenShell gateway silently drops the connection with no HTTP response body. The user sees no error message — just a curl exit code 6 after ~5 seconds. The gateway should return a clear HTTP error response indicating the backend is unreachable.

Working State
With Ollama running, inference through the gateway works correctly:

# Inside sandbox — returns valid completion
curl -s https://inference.local/v1/chat/completions \
-H  Content-Type: application/json  \
-d '{ model : llama3.2:1b , messages :[{ role : user , content : say hello }]}'

Response:

{ id : chatcmpl-100 , object : chat.completion , created :1773620673, model : llama3.2:1b , choices :[{ index :0, message :{ role : assistant , content : Hello. How can I assist you today? }, finish_reason : stop }], usage :{ prompt_tokens :27, completion_tokens :10, total_tokens :37}}

Steps to reproduce

  1. Complete nemoclaw setup successfully
  2. Set inference route to ollama-local: openshell inference set --provider ollama-local --model llama3.2:1b
  3. Verify inference works inside sandbox (pre-condition above)
  4. Stop Ollama on the host: pkill ollama
  5. Verify Ollama is down: curl -s http://localhost:11434/api/tags returns connection refused
  6. Inside sandbox, re-run inference: curl -s https://inference.local/v1/chat/completions \ -H Content-Type: application/json \ -d '{ model : llama3.2:1b , messages :[{ role : user , content : say hello }]}'
Actual Result

Steps to reproduce

No steps provided.


[NVB# 5982629]

[NVB#5982629]

Metadata

Metadata

Assignees

Labels

NV QABugs found by the NVIDIA QA TeamUATIssues flagged for User Acceptance Testing.

Type

No fields configured for Bug.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions