Problem
Two related gaps in execution trace data that limit structural credit assignment and post-hoc analysis:
1. No per-agent attribution in coordination results
CoordinationResult is frozen (model_config = ConfigDict(frozen=True)). There is no way to attach per-agent contribution data without modifying the model. When a coordinated task fails, attribution is imprecise -- the failure is attributed to the executing agent, not to the upstream agent whose bad output caused the downstream failure.
2. No node type tagging on TurnRecord
TurnRecord does not record which node types (LLM call, tool invocation, quality check, budget check, stagnation check) executed in that turn. Execution trace analyzers and evaluation pipelines cannot distinguish a turn that failed during a tool call from one that failed during a quality check.
Source: docs/research/acg-formalism-evaluation.md, closes research #848.
Solution
CoordinationResultWithAttribution
Introduce a frozen wrapper (do not modify CoordinationResult):
@dataclass(frozen=True)
class AgentContribution:
agent_id: str
subtask_id: str
outcome: Literal["success", "failure", "partial"]
failure_category: str | None # from infer_failure_category()
token_usage: int
@dataclass(frozen=True)
class CoordinationResultWithAttribution:
result: CoordinationResult
contributions: tuple[AgentContribution, ...]
Populate AgentContribution in _post_execution_pipeline using the existing infer_failure_category() keyword heuristic applied per-agent subtask result. Feed into PerformanceTracker.record_task_metric() for trend detection. Surface in GET /tasks/{id} response metadata.
NodeType enum on TurnRecord
class NodeType(Enum):
LLM_CALL = "llm_call"
TOOL_INVOCATION = "tool_invocation"
QUALITY_CHECK = "quality_check"
BUDGET_CHECK = "budget_check"
STAGNATION_CHECK = "stagnation_check"
Add node_types: tuple[NodeType, ...] = () to TurnRecord as an optional field with a default of () so existing persisted records remain valid without migration. Populate during turn execution. Consumers must treat an empty tuple as "unknown composition".
Files
src/synthorg/engine/coordination/models.py -- add AgentContribution, CoordinationResultWithAttribution
src/synthorg/engine/coordination/ -- populate attribution in post-execution pipeline
src/synthorg/engine/models.py (or wherever TurnRecord lives) -- add NodeType enum + field
src/synthorg/hr/performance/tracker.py -- accept AgentContribution input
Problem
Two related gaps in execution trace data that limit structural credit assignment and post-hoc analysis:
1. No per-agent attribution in coordination results
CoordinationResultis frozen (model_config = ConfigDict(frozen=True)). There is no way to attach per-agent contribution data without modifying the model. When a coordinated task fails, attribution is imprecise -- the failure is attributed to the executing agent, not to the upstream agent whose bad output caused the downstream failure.2. No node type tagging on TurnRecord
TurnRecorddoes not record which node types (LLM call, tool invocation, quality check, budget check, stagnation check) executed in that turn. Execution trace analyzers and evaluation pipelines cannot distinguish a turn that failed during a tool call from one that failed during a quality check.Source:
docs/research/acg-formalism-evaluation.md, closes research #848.Solution
CoordinationResultWithAttribution
Introduce a frozen wrapper (do not modify
CoordinationResult):Populate
AgentContributionin_post_execution_pipelineusing the existinginfer_failure_category()keyword heuristic applied per-agent subtask result. Feed intoPerformanceTracker.record_task_metric()for trend detection. Surface inGET /tasks/{id}response metadata.NodeType enum on TurnRecord
Add
node_types: tuple[NodeType, ...] = ()toTurnRecordas an optional field with a default of()so existing persisted records remain valid without migration. Populate during turn execution. Consumers must treat an empty tuple as "unknown composition".Files
src/synthorg/engine/coordination/models.py-- addAgentContribution,CoordinationResultWithAttributionsrc/synthorg/engine/coordination/-- populate attribution in post-execution pipelinesrc/synthorg/engine/models.py(or whereverTurnRecordlives) -- addNodeTypeenum + fieldsrc/synthorg/hr/performance/tracker.py-- acceptAgentContributioninput