Skip to content

feat(core): parallelize contiguous read-only tool calls #2563

@netbrah

Description

@netbrah

Description

Currently, all non-agent tool calls execute strictly sequentially, even when consecutive calls are read-only (file reads, searches, web fetches). This adds unnecessary latency when the model issues multiple read operations in a single turn — a very common pattern.

Proposed Enhancement

Introduce READ_ONLY_KINDS (Kind.Read, Kind.Search, Kind.Fetch) and batch contiguous read-only tool calls for parallel execution via Promise.all. Write/execute tools and agent calls continue to follow existing behavior (agents already parallelized, writes sequential).

The scheduler detects contiguous runs of read-only tools and executes each batch concurrently, then resumes sequential processing for the next non-read-only tool.

Impact

  • Significant wall-clock time reduction per turn when model issues multiple reads
  • No change to write ordering semantics
  • Adds isReadOnly property to DeclarativeTool base class for extensibility

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions