feat: concurrent tool execution with ThreadPoolExecutor#1152
Merged
Conversation
When the model returns multiple tool calls in a single response, they are now executed concurrently using a thread pool instead of sequentially. This significantly reduces wall-clock time when multiple independent tools are batched (e.g. parallel web_search, read_file, terminal calls). Architecture: - _execute_tool_calls() dispatches to sequential or concurrent path - Single tool calls and batches containing 'clarify' use sequential path - Multiple non-interactive tools use ThreadPoolExecutor (max 8 workers) - Results are collected and appended to messages in original order - _invoke_tool() extracted as shared tool invocation helper Safety: - Pre-flight interrupt check skips all tools if interrupted - Per-tool exception handling: one failure doesn't crash the batch - Result truncation (100k char limit) applied per tool - Budget pressure injection after all tools complete - Checkpoints taken before file-mutating tools - CLI spinner shows batch progress, then per-tool completion messages Tests: 10 new tests covering dispatch logic, ordering, error handling, interrupt behavior, truncation, and _invoke_tool routing.
angelburgosrosado
pushed a commit
to angelburgosrosado/hermes-agent
that referenced
this pull request
Apr 27, 2026
…f47f71c0 feat: concurrent tool execution with ThreadPoolExecutor
02356abc
pushed a commit
to 02356abc/hermes-agent
that referenced
this pull request
May 14, 2026
…f47f71c0 feat: concurrent tool execution with ThreadPoolExecutor
olympus-terminal
pushed a commit
to olympus-terminal/hermes-agent
that referenced
this pull request
May 16, 2026
…f47f71c0 feat: concurrent tool execution with ThreadPoolExecutor
Egavasyug
pushed a commit
to Egavasyug/hermes-agent
that referenced
this pull request
Jun 10, 2026
…f47f71c0 feat: concurrent tool execution with ThreadPoolExecutor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When the model returns multiple tool calls in a single response, they are now executed concurrently using a thread pool instead of sequentially. This significantly reduces wall-clock time when multiple independent tools are batched (e.g. parallel
web_search,read_file,terminalcalls).Architecture
_execute_tool_calls()now dispatches between sequential and concurrent paths_execute_tool_calls_sequential): unchanged original behavior, used for:clarify(requires interactive user input)_execute_tool_calls_concurrent): usesThreadPoolExecutorwith up to 8 workers_invoke_tool()extracted as shared tool invocation helper (handles both agent-level tools and registry-dispatched tools)Safety Considerations
_NEVER_PARALLEL_TOOLSfrozenset controls which tools force sequential execution_MAX_TOOL_WORKERS = 8caps thread pool sizeDisplay/UX
Tests
10 new tests in
TestConcurrentToolExecution:_invoke_toolrouting for regular and agent-level toolsFixed 1 existing test (
test_interrupt.py) that used MagicMock and needed real dispatch methods bound.3374 tests passing, 0 new failures.