Add AI advisor dispatch to autorevert lambda (shadow mode)#7839
Merged
izaitsevfb merged 3 commits intomainfrom Mar 19, 2026
Merged
Add AI advisor dispatch to autorevert lambda (shadow mode)#7839izaitsevfb merged 3 commits intomainfrom
izaitsevfb merged 3 commits intomainfrom
Conversation
When autorevert detects a failure pattern with a clean partition (2+ failures, 1+ success, no unknown gap), it dispatches the Claude autorevert advisor workflow for analysis. Shadow mode: the verdict is not consumed yet — dispatch is fire-and-forget for data collection. - DispatchAdvisor emitted from process_valid_autorevert_pattern() - SignalActionProcessor.dispatch_advisors() handles shuffle, dedup, cap (8/wf+commit) - Logged to misc.autorevert_events_v2 as action='advisor' - Signal pattern JSON written to /tmp for debugging - HUD shows "AI" badge and advisor dispatch table - AdvisorAction enum (skip/log/run), default: run - 122 tests passing (19 new)
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
- Cap successful+prior commits to 12 total (trim oldest prior) - Update prior partition label to avoid misleading the advisor about pre-existing failures
Contributor
Author
Evaluation Results (pytorch/pytorch, 2026-03-19)Manually dispatched 5 advisor runs on real trunk signals. State at time of testing:
4/5 correct. Run 1 used the old prompt format (before the v3 flattened signal pattern + prior partition label fix). Runs 2-5 used the current prompt and were all correct. |
jeanschmidt
approved these changes
Mar 19, 2026
3 tasks
izaitsevfb
added a commit
that referenced
this pull request
Apr 2, 2026
## Summary Closes the loop on the AI advisor: autorevert now reads verdict results from `misc.autorevert_advisor_verdicts` (populated via S3 ingestion from the advisor workflow) and uses them to make faster decisions. ## Verdict Handling | Advisor Verdict | Autorevert Action | |----------------|-------------------| | `revert` | Produce `AutorevertPattern` immediately — skip restarts | | `not_related` | Early exit as `Ineligible(ADVISOR_NOT_RELATED)` | | `garbage` | Suppress signal for 2h since verdict timestamp | | `unsure` | Continue normal restart-to-confirm flow | ## Changes - **`signal.py`**: `AdvisorVerdict` enum, `AIAdvisorResult` dataclass, `SignalCommit.advisor_result` field, `_check_advisor_verdict()` method, restructured `process_valid_autorevert_pattern()` (flakiness check moved after advisor) - **`signal_extraction_datasource.py`**: `fetch_advisor_verdicts()` — CH query for verdicts by commit SHAs - **`signal_extraction.py`**: `_attach_advisor_verdicts()` — post-processing step that wires CH data to SignalCommit objects - **`tests/test_signal.py`**: 8 new tests covering all verdict types, 2h expiry, signal key matching, flaky signal handling ## Dependencies - [#7839](#7839) (merged) — advisor dispatch from lambda - [#7906](#7906) (merged) — CH table + S3 ingestion - [pytorch/pytorch#178810](pytorch/pytorch#178810) (pending) — workflow S3 upload step ## Test plan - [x] 60 tests passing (8 new) - [x] End-to-end verified: ciforge advisor run → S3 → CH → row queryable - [ ] Deploy and verify verdicts are read during live autorevert runs tested locally: ``` python -m pytorch_auto_revert --dry-run autorevert-checker trunk --hours 18 --as-of "2026-03-31 18:12" --hud-html python -m pytorch_auto_revert --dry-run autorevert-checker trunk --hours 18 --hud-html ``` example of the result: [2026-04-01T00-11-54.124373-00-00.html](https://github.com/user-attachments/files/26392664/2026-04-01T00-11-54.124373-00-00.html) [2026-04-01T01-22-24.845831-00-00.html](https://github.com/user-attachments/files/26393283/2026-04-01T01-22-24.845831-00-00.html)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds shadow-mode AI advisor dispatch to the autorevert lambda. When a clean failure partition is detected (2+ failures, 1+ success, no unknown gap), the lambda dispatches the
claude-autorevert-advisor.ymlworkflow (pytorch/pytorch#177404) for AI-powered failure analysis.Shadow mode: verdicts are not consumed — dispatch is fire-and-forget for accuracy data collection.
Changes
DispatchAdvisordataclass emitted fromprocess_valid_autorevert_pattern()(pure functional)SignalActionProcessor.dispatch_advisors()— shuffled dispatch with per-signal dedup and cap (8 per workflow+commit)AdvisorActionenum (skip/log/run), CLI--advisor-action, envADVISOR_ACTIONmisc.autorevert_events_v2asaction='advisor'/tmp/advisor-patterns/for debuggingadvisor_dispatcheskey (forward/backward compatible)AdvisorAction.RUN(lambda dispatches advisors on deploy, no gha-infra changes needed)Test plan