-
Notifications
You must be signed in to change notification settings - Fork 0
feat: coordination metrics from distributed systems theory (Amdahl ceiling, straggler gap) #703
Copy link
Copy link
Open
Labels
prio:highImportant, should be prioritizedImportant, should be prioritizedscope:medium1-3 days of work1-3 days of workspec:architectureDESIGN_SPEC Section 15 - Technical ArchitectureDESIGN_SPEC Section 15 - Technical Architecturespec:budgetDESIGN_SPEC Section 10 - Cost & Budget ManagementDESIGN_SPEC Section 10 - Cost & Budget Managementspec:task-workflowDESIGN_SPEC Section 6 - Task & Workflow EngineDESIGN_SPEC Section 6 - Task & Workflow Enginetype:featureNew feature implementationNew feature implementationv0.7Minor version v0.7Minor version v0.7v0.7.8Patch release v0.7.8Patch release v0.7.8
Description
Context
Deep dive on "Language Model Teams as Distributed Systems" (arXiv:2603.12229) -- Princeton/MIT/Cambridge/NYU study applying distributed computing theory to LLM agent teams.
Key empirical findings:
- Decentralized teams: median 0.88x speedup (slower than single agent), 19 vs 4 test failures
- Centralized preassigned: median 1.36x speedup, O(n) messages
- Token cost scales with team size (rho=0.40) while speedup does not (rho=-0.07)
- Amdahl's Law confirmed (H=61.4, p<0.001): speedup ceiling =
1 / (1-p)
Action Items
- Amdahl ceiling for team sizing: Compute
S_max = 1/(1-p)from task parallelizability before dispatching; use as team size ceiling in auto-topology selector. Currently only qualitative rules. - Straggler gap metric: Add
straggler_gap(slowest_agent - mean_completion) as 6th coordination metric. Decentralized-topology diagnostic. - Token/speedup ratio alert: Add
token_multiplier / latency_speedupper-run ratio; alert when >2.0 for decentralized runs. Paper: median 1.17 decentralized. - O(n^2) message alert: Flag when decentralized team size grows past the point where O(n^2) coordination overhead dominates.
- File-locking priority: Paper shows 19 vs 4 test failures without file-level locking in decentralized dispatch. Bump priority for file-level locking when decentralized dispatch goes production.
Risks
- Paper tested homogeneous teams only; SynthOrg's role-differentiated agents mean exact numbers won't transfer -- directional findings are robust
- Model cost heterogeneity not modeled (Senior agents cost 5-20x more per token)
- O(n^2) message volume accelerates context fill, triggering more compaction (unmodeled interaction)
References
- arXiv:2603.12229
- GitHub: distributed-llm-teams
- Prior: arXiv:2512.08296 (Kim et al.) -- already integrated
- Related: research: audit engine against multi-agent failure patterns (swarm drift, microservices anti-patterns) #690 (multi-agent failure pattern audit)
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
prio:highImportant, should be prioritizedImportant, should be prioritizedscope:medium1-3 days of work1-3 days of workspec:architectureDESIGN_SPEC Section 15 - Technical ArchitectureDESIGN_SPEC Section 15 - Technical Architecturespec:budgetDESIGN_SPEC Section 10 - Cost & Budget ManagementDESIGN_SPEC Section 10 - Cost & Budget Managementspec:task-workflowDESIGN_SPEC Section 6 - Task & Workflow EngineDESIGN_SPEC Section 6 - Task & Workflow Enginetype:featureNew feature implementationNew feature implementationv0.7Minor version v0.7Minor version v0.7v0.7.8Patch release v0.7.8Patch release v0.7.8