-
Notifications
You must be signed in to change notification settings - Fork 0
Implement coordination error taxonomy with opt-in classification pipeline (DESIGN_SPEC §10.5 M5) #146
Copy link
Copy link
Closed
Labels
prio:mediumShould do, but not blockingShould do, but not blockingscope:medium1-3 days of work1-3 days of workspec:agent-systemDESIGN_SPEC Section 3 - Agent SystemDESIGN_SPEC Section 3 - Agent Systemspec:budgetDESIGN_SPEC Section 10 - Cost & Budget ManagementDESIGN_SPEC Section 10 - Cost & Budget Managementspec:communicationDESIGN_SPEC Section 5 - Communication ArchitectureDESIGN_SPEC Section 5 - Communication Architecturespec:task-workflowDESIGN_SPEC Section 6 - Task & Workflow EngineDESIGN_SPEC Section 6 - Task & Workflow Enginetype:featureNew feature implementationNew feature implementationtype:testTest coverage, test infrastructureTest coverage, test infrastructure
Description
Context
When coordination metrics collection is enabled (see #135), the system can optionally classify coordination errors into structured categories. This enables targeted diagnosis of multi-agent coordination problems — e.g., if coordination failures spike, the topology may be too complex; if context omissions spike, the orchestrator's synthesis is insufficient.
This is an M5 feature because it requires semantic analysis infrastructure (embedding computation, contradiction detection) and baseline comparison data from M4 coordination metrics.
Acceptance Criteria
Error Taxonomy Categories
- Logical contradiction — agent asserts both "X is true" and "X is false"; detected via semantic contradiction analysis on agent outputs
- Numerical drift — accumulated computational errors from cascading rounding/unit conversion (>5% deviation); detected via numerical comparison against ground truth or cross-agent verification
- Context omission — failure to reference previously established entities/state required for current reasoning; detected via missing-reference analysis across agent conversation history
- Coordination failure — MAS-specific: message misinterpretation, task allocation conflicts, state synchronization errors; detected at the protocol/orchestration layer
Classification Pipeline
- Opt-in via
coordination_metrics.error_taxonomy.enabled: trueconfig - Configurable category selection (can enable/disable individual categories)
- Pipeline runs post-execution — never blocks agent work
- Classification results logged as structured events to observability layer
- Pipeline gracefully degrades if semantic analysis is unavailable
Testing
- Unit tests for each error category detection logic
- Integration test: multi-agent task with known error patterns → verify classification
- Verify pipeline never blocks execution (async/post-execution enforcement)
Dependencies
- Implement LLM call categorization, coordination metrics suite, and orchestration tracking (DESIGN_SPEC §10.5 M4) #135 — Coordination metrics suite (M4) — must exist before error taxonomy can classify errors
- M5 memory/persistence infrastructure for storing classification results
- Embedding computation capability for
context_omissionandlogical_contradictiondetection
Design Spec Reference
- §10.5 — M4/M5: Coordination Error Taxonomy
- §16.3 — Agent Scaling Research (Kim et al., 2025 + Cemri et al., 2025 MAST)
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
prio:mediumShould do, but not blockingShould do, but not blockingscope:medium1-3 days of work1-3 days of workspec:agent-systemDESIGN_SPEC Section 3 - Agent SystemDESIGN_SPEC Section 3 - Agent Systemspec:budgetDESIGN_SPEC Section 10 - Cost & Budget ManagementDESIGN_SPEC Section 10 - Cost & Budget Managementspec:communicationDESIGN_SPEC Section 5 - Communication ArchitectureDESIGN_SPEC Section 5 - Communication Architecturespec:task-workflowDESIGN_SPEC Section 6 - Task & Workflow EngineDESIGN_SPEC Section 6 - Task & Workflow Enginetype:featureNew feature implementationNew feature implementationtype:testTest coverage, test infrastructureTest coverage, test infrastructure