Strengthen reviewer skill: add step-back analysis dimensions#13504
Merged
JanProvaznik merged 1 commit intodotnet:mainfrom Apr 8, 2026
Merged
Strengthen reviewer skill: add step-back analysis dimensions#13504JanProvaznik merged 1 commit intodotnet:mainfrom
JanProvaznik merged 1 commit intodotnet:mainfrom
Conversation
Add rules for historical context analysis, root-cause vs symptom fixes, N-participant race generalization, assertion strength validation, and API alternative awareness to the expert-reviewer agent. - Dim 4 (Tests): flag weak assertions that pass with wrong output - Dim 10 (Design): check design intent alignment, API alternatives, and borrowed-pattern assumptions - Dim 22 (Correctness): generalize N=2 to N=3+ participants, flag symptom-only fixes - Workflow: add historical context step before dimension dispatch - Tasks instructions: document Microsoft.IO.Redist API availability Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Updates the repo’s review guidance to help reviewers “step back” from mechanical diff-checking and catch design-level issues (especially around concurrency, TOCTOU patterns, and weak test assertions), incorporating lessons from PR #13477.
Changes:
- Strengthen the Tests dimension to flag weak assertions that could pass with incorrect behavior.
- Expand Design and Correctness dimensions to emphasize design-intent alignment, avoiding workaround chains when better APIs exist, validating borrowed patterns, and generalizing race fixes beyond 2 participants.
- Update workflow to include a historical context step (read linked issue + original feature PR) and document
Microsoft.IO.RedistAPI availability for tasks targeting .NET Framework.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| .github/instructions/tasks.instructions.md | Adds guidance to check Microsoft.IO.* backported APIs (via Microsoft.IO.Redist) before implementing workaround-based I/O patterns in tasks. |
| .github/agents/expert-reviewer.md | Extends the reviewer checklist with “step-back” design/correctness checks and adds a historical-context workflow step. |
ViktorHofer
approved these changes
Apr 8, 2026
JanKrivanek
approved these changes
Apr 8, 2026
dfederm
pushed a commit
to dfederm/msbuild
that referenced
this pull request
Apr 9, 2026
…13504) Learnings from reviewing PR dotnet#13477 — the 24-dimension checklist missed design-level issues that required stepping back from the diff. ### Changes - **Dim 4 (Tests)**: Flag weak assertions that would pass with incorrect output - **Dim 10 (Design)**: Check alignment with original design intent, flag workarounds when better APIs exist in existing dependencies, validate borrowed patterns - **Dim 22 (Correctness)**: Generalize 2-participant fixes to N participants, flag symptom-only fixes - **Workflow**: Add historical context step (read linked issue + original feature PR) before dispatching dimension agents - **Tasks instructions**: Document \Microsoft.IO.Redist\ API availability for .NET Framework ### Why The review agents evaluated the diff mechanically but missed: 1. The fix patches a TOCTOU symptom; a better API (\Microsoft.IO.File.Move(overwrite: true)\) exists in an already-referenced package 2. The fix handles 2 concurrent writers but fails with 3+ 3. Test assertions were too weak to verify correct behavior 4. The approach was borrowed from the VS editor (which never has 'file doesn't exist' case) without validating assumptions Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Learnings from reviewing PR #13477 — the 24-dimension checklist missed design-level issues that required stepping back from the diff.
Changes
Why
The review agents evaluated the diff mechanically but missed: