Skip to content

feat: coordination metrics from distributed systems theory (Amdahl ceiling, straggler gap) #703

@Aureliolo

Description

@Aureliolo

Context

Deep dive on "Language Model Teams as Distributed Systems" (arXiv:2603.12229) -- Princeton/MIT/Cambridge/NYU study applying distributed computing theory to LLM agent teams.

Key empirical findings:

  • Decentralized teams: median 0.88x speedup (slower than single agent), 19 vs 4 test failures
  • Centralized preassigned: median 1.36x speedup, O(n) messages
  • Token cost scales with team size (rho=0.40) while speedup does not (rho=-0.07)
  • Amdahl's Law confirmed (H=61.4, p<0.001): speedup ceiling = 1 / (1-p)

Action Items

  • Amdahl ceiling for team sizing: Compute S_max = 1/(1-p) from task parallelizability before dispatching; use as team size ceiling in auto-topology selector. Currently only qualitative rules.
  • Straggler gap metric: Add straggler_gap (slowest_agent - mean_completion) as 6th coordination metric. Decentralized-topology diagnostic.
  • Token/speedup ratio alert: Add token_multiplier / latency_speedup per-run ratio; alert when >2.0 for decentralized runs. Paper: median 1.17 decentralized.
  • O(n^2) message alert: Flag when decentralized team size grows past the point where O(n^2) coordination overhead dominates.
  • File-locking priority: Paper shows 19 vs 4 test failures without file-level locking in decentralized dispatch. Bump priority for file-level locking when decentralized dispatch goes production.

Risks

  • Paper tested homogeneous teams only; SynthOrg's role-differentiated agents mean exact numbers won't transfer -- directional findings are robust
  • Model cost heterogeneity not modeled (Senior agents cost 5-20x more per token)
  • O(n^2) message volume accelerates context fill, triggering more compaction (unmodeled interaction)

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    prio:highImportant, should be prioritizedscope:medium1-3 days of workspec:architectureDESIGN_SPEC Section 15 - Technical Architecturespec:budgetDESIGN_SPEC Section 10 - Cost & Budget Managementspec:task-workflowDESIGN_SPEC Section 6 - Task & Workflow Enginetype:featureNew feature implementationv0.7Minor version v0.7v0.7.8Patch release v0.7.8

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions