Skip to content

DebateResolver: fall back to AuthorityResolver when judge evaluator raises #1117

@Aureliolo

Description

@Aureliolo

Problem

DebateResolver makes a single LLM call to the judge evaluator. If the evaluator raises (provider error, timeout), the exception propagates through ConflictResolutionService.resolve() uncaught -- the resolution attempt fails entirely with no fallback.

DebateResolver already falls back to AuthorityResolver when no JudgeEvaluator is configured. The same fallback should apply when the evaluator raises.

Source: docs/research/multi-agent-failure-audit.md (R5), closes research #690.

Solution

Wrap the evaluator call in DebateResolver with a try/except. On exception, log a warning with the error context and fall back to AuthorityResolver.

try:
    winner = await self._judge_evaluator.evaluate(conflict, arguments)
except Exception:
    logger.warning(CONFLICT_DEBATE_EVALUATOR_FAILED, ...)
    return AuthorityResolver().resolve(conflict)

Requires a new CONFLICT_DEBATE_EVALUATOR_FAILED event constant.

Files

  • src/synthorg/communication/conflict_resolution/debate_strategy.py
  • src/synthorg/observability/events/ -- add event constant

Metadata

Metadata

Assignees

No one assigned

    Labels

    prio:lowNice to have, can deferscope:smallLess than 1 day of workspec:communicationDESIGN_SPEC Section 5 - Communication Architecturetype:fixBug fixes and corrections

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions