Context

Two complementary findings on improving engine observability:

1. AgentProcessBench -- 1,000 trajectories, 8,509 human-labeled step annotations. Ternary step labeling (correct/neutral-exploratory/incorrect) with error-propagation rules. Current stagnation detection operates at task level only.
2. MADQA Benchmark -- Agents trapped in loops despite having answers. Novel accuracy-effort trade-off metric (outcome quality vs steps consumed).

Action Items

• Add step-level quality signals to approval gate triggers (not just task-boundary outcomes)
• Implement ternary step classification: correct / neutral-exploratory / incorrect
• Expose accuracy-effort ratio in observability layer (task quality / normalized steps)
• Wire accuracy-effort metric into budget module for cost-per-outcome analysis
• Address finding: weaker models terminate early, inflating "correct step" ratios (trap for HR performance tracking)

References

• AgentProcessBench
• MADQA Benchmark
• Research log entry Evaluate and set up CLI framework (Typer/Click/argparse) #27 (2026-03-14)