feat: add declarative tool review gates for workflow run outputs (#369)#370
Conversation
Implements #369 — deterministic, non-LLM validation gates that inspect structured pipeline outputs and drive stage transitions. Changes: - pkg/hub/pipeline/pipeline.go: Add Gate, GateResultTrigger, OutputMatchesTrigger types and stage lookup helpers. Gate evaluates pass/fail conditions against named JSON outputs from prior run actions. - pkg/hub/pipeline_runner.go: evaluateGate(), persistGateResult(), loadGateResult(), hasFailedRequiredGate(), autoTransitionAfterGate() — gate evaluation logic integrated into runOnEnter after run actions complete. - pkg/hub/linear.go: Block [DONE] signal when any required gate has failed. - pkg/hub/db.go: Add pipeline_gate_results table (v10 migration) for persisting gate verdicts. - Tests: gate parsing, evaluation (pass/fail/skipped/error), transitions, required-gate PR blocking, JSON path walking.
|
Reviews (1): Last reviewed commit: "feat: add declarative tool review gates ..." | Re-trigger Greptile |
- Block both 'fail' and 'error' verdicts for required gates in runOnEnter
- hasFailedRequiredGate now checks verdict IN ('fail','error')
- Distinct chat message for 'error' vs 'fail' verdicts
- Add test for error-verdict gate blocking [DONE]
- Add test for error-verdict in hasFailedRequiredGate
- Comment on loadGateResult noting future UI use
|
StageForOutputMatches was defined and tested but never called from production code. checkPipelineMessageTriggers now falls back to output_matches evaluation when message_contains doesn't match, so stages with output_matches triggers can actually transition at runtime. Adds TestCheckPipelineMessageTriggersOutputMatches to verify the end-to-end trigger evaluation.
|
Reviews (3): Last reviewed commit: "fix: wire output_matches trigger into ch..." | Re-trigger Greptile |
- persistPipelineOutput now parses JSON stdout regardless of exit code, so validation tools that emit structured output on nonzero exit can still have their output inspected by gates. - runOnEnter continues to gate evaluation when a stage has a gate and the run produced a named output, even if the command exited nonzero. This avoids requiring continue_on_error=true on every gate stage. - Distinction: nonzero exit + valid JSON -> gate evaluates normally; nonzero exit + no/invalid JSON -> gate verdict 'error'. - Tests: nonzero exit with valid JSON (gate evaluates to fail), nonzero exit with no JSON (gate verdict error).
|
…SkippedAsPass is set
When a gate's output is missing and TreatSkippedAsPass is true, the gate
verdict is 'skipped'. Auto-transition now normalizes this to 'pass' so
that gate_result: { verdict: pass } triggers correctly fire, honouring
the contract stated in the PR description.
Also fixes persistPipelineOutput to parse JSON stdout regardless of exit
code, so validation tools that emit structured output on nonzero exit can
still have their output inspected by gates.
Tests: skipped-as-pass auto-transition, nonzero exit with valid JSON
(gate evaluates to fail), nonzero exit with no JSON (gate verdict error).
output_matches triggers query persistent DB state (pipeline outputs). Without a one-shot guard, every subsequent call to checkPipelineMessageTriggers would re-evaluate and re-transition the claw back to the same stage, causing backwards regressions and re-executing on_enter actions. Changes: - Add pipeline_stage_history table (v11 migration) to track visited stages - Add recordPipelineStageVisit / hasVisitedPipelineStage helpers in pipeline_state.go - transitionPipelineStageWithContext records each stage visit on successful transition - checkPipelineMessageTriggers skips output_matches triggers for stages the claw has already visited This gives output_matches a natural one-shot mechanism: it fires once when the output first matches, and never again.
|
Reviews (5): Last reviewed commit: "fix: prevent output_matches trigger from..." | Re-trigger Greptile |
Implements #369.
Adds deterministic, non-LLM validation gates that inspect structured pipeline outputs (e.g. CodeBuild, E2E tests, security scanners) and drive stage transitions without involving an LLM judge.
Pipeline YAML additions
gateon a stage: inspects a named output from a priorrunactionpass/failconditions: JSON path + expected valuesrequired: blocks PR creation if gate failstreat_skipped_as_pass: missing output treated as passgate_resulttrigger: auto-transition on a specific gate verdictoutput_matchestrigger: general primitive for JSON path matchingExample pipeline