Description
Currently, all non-agent tool calls execute strictly sequentially, even when consecutive calls are read-only (file reads, searches, web fetches). This adds unnecessary latency when the model issues multiple read operations in a single turn — a very common pattern.
Proposed Enhancement
Introduce READ_ONLY_KINDS (Kind.Read, Kind.Search, Kind.Fetch) and batch contiguous read-only tool calls for parallel execution via Promise.all. Write/execute tools and agent calls continue to follow existing behavior (agents already parallelized, writes sequential).
The scheduler detects contiguous runs of read-only tools and executes each batch concurrently, then resumes sequential processing for the next non-read-only tool.
Impact
- Significant wall-clock time reduction per turn when model issues multiple reads
- No change to write ordering semantics
- Adds
isReadOnly property to DeclarativeTool base class for extensibility
Description
Currently, all non-agent tool calls execute strictly sequentially, even when consecutive calls are read-only (file reads, searches, web fetches). This adds unnecessary latency when the model issues multiple read operations in a single turn — a very common pattern.
Proposed Enhancement
Introduce
READ_ONLY_KINDS(Kind.Read,Kind.Search,Kind.Fetch) and batch contiguous read-only tool calls for parallel execution viaPromise.all. Write/execute tools and agent calls continue to follow existing behavior (agents already parallelized, writes sequential).The scheduler detects contiguous runs of read-only tools and executes each batch concurrently, then resumes sequential processing for the next non-read-only tool.
Impact
isReadOnlyproperty toDeclarativeToolbase class for extensibility