Bug Description
vision_analyze fails with a timeout error when using a local model for the auxiliary vision client. The default 30s timeout in async_call_llm is too short for local models where image encoding alone can take 30s+ to complete (on very constrained hardware or large/slow models on consumer hardware etc)
Steps to Reproduce
- Use locally hosted model with vision
- Ask hermes to describe an image
- Wait 30~ seconds for timeout to occur
Expected Behavior
Image is analyzed successfully and the result is passed to the main model
Actual Behavior
Vision pre-analysis times out before the server returns the first token:
httpx.ReadTimeout
openai.APITimeoutError: Request timed out.
ERROR tools.vision_tools: Error analyzing image: Request timed out.
Affected Component
Tools (terminal, file ops, web, code execution, etc.)
Messaging Platform (if gateway-related)
No response
Operating System
Arch Linux (6.19.7-1)
Python Version
3.11.15
Hermes Version
v0.4.0 (2026.3.18)
Relevant Logs / Traceback
Expand: ERROR tools.vision_tools: Error analyzing image: Request timed out
2026-03-19 20:08:23,828 ERROR tools.vision_tools: Error analyzing image: Request timed out.
Traceback (most recent call last):
File "/home/smt/.hermes/hermes-agent/venv/lib/python3.11/site-packages/httpx/_transports/default.py", line 101, in map_httpcore_exceptions
yield
File "/home/smt/.hermes/hermes-agent/venv/lib/python3.11/site-packages/httpx/_transports/default.py", line 394, in handle_async_request
resp = await self._pool.handle_async_request(req)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/smt/.hermes/hermes-agent/venv/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 256, in handle_async_request
raise exc from None
File "/home/smt/.hermes/hermes-agent/venv/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 236, in handle_async_request
response = await connection.handle_async_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/smt/.hermes/hermes-agent/venv/lib/python3.11/site-packages/httpcore/_async/connection.py", line 103, in handle_async_request
return await self._connection.handle_async_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/smt/.hermes/hermes-agent/venv/lib/python3.11/site-packages/httpcore/_async/http11.py", line 136, in handle_async_request
raise exc
File "/home/smt/.hermes/hermes-agent/venv/lib/python3.11/site-packages/httpcore/_async/http11.py", line 106, in handle_async_request
) = await self._receive_response_headers(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/smt/.hermes/hermes-agent/venv/lib/python3.11/site-packages/httpcore/_async/http11.py", line 177, in _receive_response_headers
event = await self._receive_event(timeout=timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/smt/.hermes/hermes-agent/venv/lib/python3.11/site-packages/httpcore/_async/http11.py", line 217, in _receive_event
data = await self._network_stream.read(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/smt/.hermes/hermes-agent/venv/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 32, in read
with map_exceptions(exc_map):
File "/home/smt/.local/share/uv/python/cpython-3.11.15-linux-x86_64-gnu/lib/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/home/smt/.hermes/hermes-agent/venv/lib/python3.11/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
raise to_exc(exc) from exc
httpcore.ReadTimeout
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/smt/.hermes/hermes-agent/venv/lib/python3.11/site-packages/openai/_base_client.py", line 1604, in request
response = await self._client.send(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/smt/.hermes/hermes-agent/venv/lib/python3.11/site-packages/httpx/_client.py", line 1629, in send
response = await self._send_handling_auth(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/smt/.hermes/hermes-agent/venv/lib/python3.11/site-packages/httpx/_client.py", line 1657, in _send_handling_auth
response = await self._send_handling_redirects(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/smt/.hermes/hermes-agent/venv/lib/python3.11/site-packages/httpx/_client.py", line 1694, in _send_handling_redirects
response = await self._send_single_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/smt/.hermes/hermes-agent/venv/lib/python3.11/site-packages/httpx/_client.py", line 1730, in _send_single_request
response = await transport.handle_async_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/smt/.hermes/hermes-agent/venv/lib/python3.11/site-packages/httpx/_transports/default.py", line 393, in handle_async_request
with map_httpcore_exceptions():
File "/home/smt/.local/share/uv/python/cpython-3.11.15-linux-x86_64-gnu/lib/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/home/smt/.hermes/hermes-agent/venv/lib/python3.11/site-packages/httpx/_transports/default.py", line 118, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.ReadTimeout
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/smt/.hermes/hermes-agent/tools/vision_tools.py", line 310, in vision_analyze_tool
response = await async_call_llm(**call_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/smt/.hermes/hermes-agent/agent/auxiliary_client.py", line 1512, in async_call_llm
return await client.chat.completions.create(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/smt/.hermes/hermes-agent/venv/lib/python3.11/site-packages/openai/resources/chat/completions/completions.py", line 2714, in create
return await self._post(
^^^^^^^^^^^^^^^^^
File "/home/smt/.hermes/hermes-agent/venv/lib/python3.11/site-packages/openai/_base_client.py", line 1884, in post
return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/smt/.hermes/hermes-agent/venv/lib/python3.11/site-packages/openai/_base_client.py", line 1622, in request
raise APITimeoutError(request=request) from err
openai.APITimeoutError: Request timed out.
Root Cause Analysis (optional)
inside of vision_analyze_tool() the following call_kwargs is defined:
call_kwargs = {
"task": "vision",
"messages": messages,
"temperature": 0.1,
"max_tokens": 2000,
}
without a timeout, the timeout ends up defaulting somewhere up the chain, many of them like async_call_llm() in agent/auxiliary_client.py have a default of 30
as a quick workaround I was able to add "timeout": 180, to call_kwargs which stopped the timeout
Proposed Fix (optional)
some kind of configuration or envvar would be useful as just bumping the default could obviously lead to some unwanted behavior for people who know their models should be returning fast, but for people playing around with local models and constrained hardware, some way to raise it would be useful
potentially also a more generous default if the provider is detected as custom/local would give a smoother experience for people using a wider range of configurations
Are you willing to submit a PR for this?
Bug Description
vision_analyzefails with a timeout error when using a local model for the auxiliary vision client. The default 30s timeout inasync_call_llmis too short for local models where image encoding alone can take 30s+ to complete (on very constrained hardware or large/slow models on consumer hardware etc)Steps to Reproduce
Expected Behavior
Image is analyzed successfully and the result is passed to the main model
Actual Behavior
Vision pre-analysis times out before the server returns the first token:
Affected Component
Tools (terminal, file ops, web, code execution, etc.)
Messaging Platform (if gateway-related)
No response
Operating System
Arch Linux (6.19.7-1)
Python Version
3.11.15
Hermes Version
v0.4.0 (2026.3.18)
Relevant Logs / Traceback
Expand: ERROR tools.vision_tools: Error analyzing image: Request timed out
Root Cause Analysis (optional)
inside of
vision_analyze_tool()the following call_kwargs is defined:without a timeout, the timeout ends up defaulting somewhere up the chain, many of them like
async_call_llm()inagent/auxiliary_client.pyhave a default of 30as a quick workaround I was able to add
"timeout": 180,to call_kwargs which stopped the timeoutProposed Fix (optional)
some kind of configuration or envvar would be useful as just bumping the default could obviously lead to some unwanted behavior for people who know their models should be returning fast, but for people playing around with local models and constrained hardware, some way to raise it would be useful
potentially also a more generous default if the provider is detected as custom/local would give a smoother experience for people using a wider range of configurations
Are you willing to submit a PR for this?