Context
Phase 5 Community Detection (epic #1222, issue #1228) generates LLM summaries sequentially for each detected community.
Problem
At 20 communities with 3-15s per LLM call (depending on provider), the sequential loop can take up to 5 minutes. The task runs in a background tokio::spawn but holds the extraction task slot for the entire duration.
Suggested Fix
Use FuturesUnordered with a concurrency limit of 3 for LLM summary generation. This bounds concurrent API calls (avoiding rate limits) while reducing total latency by ~3x.
use futures::stream::FuturesUnordered;
use futures::StreamExt;
let mut futs = FuturesUnordered::new();
// ... push up to 3 concurrent summary tasks
Source
PERF-02 from Phase 5 performance analysis. TODO comment exists at community.rs line 168.
Context
Phase 5 Community Detection (epic #1222, issue #1228) generates LLM summaries sequentially for each detected community.
Problem
At 20 communities with 3-15s per LLM call (depending on provider), the sequential loop can take up to 5 minutes. The task runs in a background tokio::spawn but holds the extraction task slot for the entire duration.
Suggested Fix
Use
FuturesUnorderedwith a concurrency limit of 3 for LLM summary generation. This bounds concurrent API calls (avoiding rate limits) while reducing total latency by ~3x.Source
PERF-02 from Phase 5 performance analysis. TODO comment exists at community.rs line 168.