Skip to content

perf(graph): concurrent LLM community summaries with FuturesUnordered #1266

@bug-ops

Description

@bug-ops

Context

Phase 5 Community Detection (epic #1222, issue #1228) generates LLM summaries sequentially for each detected community.

Problem

At 20 communities with 3-15s per LLM call (depending on provider), the sequential loop can take up to 5 minutes. The task runs in a background tokio::spawn but holds the extraction task slot for the entire duration.

Suggested Fix

Use FuturesUnordered with a concurrency limit of 3 for LLM summary generation. This bounds concurrent API calls (avoiding rate limits) while reducing total latency by ~3x.

use futures::stream::FuturesUnordered;
use futures::StreamExt;

let mut futs = FuturesUnordered::new();
// ... push up to 3 concurrent summary tasks

Source

PERF-02 from Phase 5 performance analysis. TODO comment exists at community.rs line 168.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestllmzeph-llm crate (Ollama, Claude)performancePerformance improvements

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions