Add VMR codeflow health check to pipelines skill#13326
Merged
JanProvaznik merged 4 commits intodotnet:mainfrom Mar 5, 2026
Merged
Add VMR codeflow health check to pipelines skill#13326JanProvaznik merged 4 commits intodotnet:mainfrom
JanProvaznik merged 4 commits intodotnet:mainfrom
Conversation
Add check-vmr-codeflow.ps1 that finds open codeflow PRs from dotnet/msbuild to dotnet/dotnet, gets dotnet-unified-build pipeline runs from dnceng-public/public, extracts failure details, and correlates them with included upstream msbuild PRs. Update SKILL.md with VMR pipeline reference info, data collection step, table rendering instructions, subagent template for VMR codeflow failure investigation, and troubleshooting section. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends the pipelines-health-check skill to also monitor VMR (dotnet/dotnet) codeflow PRs and the dotnet-unified-build pipeline. When MSBuild source changes are flowed into the VMR, the new tooling detects pipeline failures and correlates them back to the specific upstream MSBuild PRs that likely caused them.
Changes:
- New
check-vmr-codeflow.ps1script that discovers open codeflow PRs via GitHub API, retrieves Azure DevOps pipeline runs for each PR branch, extracts and categorizes failures from build timelines, and correlates failures with included upstream MSBuild PRs. - Updated
SKILL.mdto document the new data collection step, add a VMR Codeflow PRs table section, define new problem types (VMR codeflow failure/stale), and add a subagent prompt template for VMR codeflow investigation.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
.github/skills/pipelines-health-check/check-vmr-codeflow.ps1 |
New script: searches for open codeflow PRs, retrieves pipeline runs and timelines, categorizes failures, and emits structured JSON |
.github/skills/pipelines-health-check/SKILL.md |
Updated skill documentation: adds VMR triggers, pipeline table row, key URLs, data collection step, table rendering instructions, problem types, and subagent prompt for VMR codeflow failures |
You can also share your feedback on Copilot code review. Take the survey.
- Rename $matches to $regexMatches to avoid shadowing PS automatic variable - Add short reference regex (msbuild#NNN) and fix misleading comment - Reorder failure category checks: specific BinaryToolTask before general MSB4216 - Add DefinitionId parameter to Get-PipelineRunsForPR for pipeline filtering - Add GITHUB_TOKEN support via shared $githubHeaders for rate limit mitigation - Fix broken dotnet-unified-build URL in SKILL.md (was using non-numeric definitionId) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace all unauthenticated Invoke-RestMethod GitHub API calls with gh CLI which handles authentication automatically. This eliminates rate limiting issues (60 req/hr unauthenticated vs 5000 req/hr authenticated) and properly surfaces failures instead of silently degrading. Add Invoke-GitHubApi helper that wraps gh api with error handling. Add gh CLI availability and auth check at script startup. Update SKILL.md to document gh CLI dependency. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add Sanitize-ErrorString to both check-pipeline-health.ps1 and check-vmr-codeflow.ps1 to strip control characters and truncate error messages to 500 chars, preventing broken JSON output - Replace verbose stages/allJobs arrays with compact jobSummary counts in VMR script, reducing output from 173KB to ~84KB - Cap upstream PR detail fetching at 30 per codeflow PR; remaining PR numbers are listed in additionalUpstreamPRNumbers array - Increase ConvertTo-Json depth from 8 to 10 to avoid truncation - Update SKILL.md with guidance on script timing (use initial_wait of 120s+) and note that JSON is now safe to parse directly Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This was referenced Mar 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Extends the pipelines-health-check skill to monitor VMR (dotnet/dotnet) codeflow PRs and the \dotnet-unified-build\ pipeline.
Changes
*New: \check-vmr-codeflow.ps1*
*Updated: \SKILL.md*
Motivation
When MSBuild source changes are codeflowed into the VMR, the \dotnet-unified-build\ pipeline validates them. Failures there (like the recent MSB4216 task host errors from the app host changes) weren't covered by the existing skill. This addition lets the agent automatically detect VMR failures and correlate them back to the specific msbuild PRs that likely caused them.
Testing
Tested locally — script successfully found 2 open codeflow PRs (#5183, #5204), identified TASK_HOST failures, and correlated them with upstream PRs.