[skills] Add AZDO build investigator skill#10885
Merged
jonathanpeppers merged 4 commits intomainfrom Mar 4, 2026
Merged
Conversation
Add a Copilot skill for investigating Azure DevOps pipeline build
failures. Given a build URL, it fetches run details, identifies failed
jobs/tasks via the timeline API, downloads .binlog artifacts, and
analyzes them with binlogtool to find root causes.
Example usage:
❯ what are the errors here: #10881
◐ The user is asking about errors in a GitHub PR. This is a CI/build failure investigation task. There's an `azdo-build-investigator` skill available that can investigate Azure DevOps pipeline build failures
and analyze .binlog files. However, this is a GitHub PR URL, not an AZDO URL directly. Let me check if the skill can handle GitHub PR URLs - the skill description says "Use this when users share an AZDO
build URL, a GitHub PR URL, or ask about a failing pipeline, build errors, or CI failures." So yes, this skill can handle GitHub PR URLs.
Let me invoke the `azdo-build-investigator` skill.
● skill(azdo-build-investigator)
◐ Let me follow the skill workflow to investigate the GitHub PR. The URL is #10881.
First, let me check if `gh` CLI and `az` CLI are available, and then get the failing checks from the PR.
● Get failing checks from PR #10881
$ gh pr checks 10881 --repo dotnet/android --json "name,state,link,bucket" 2>&1
└ 39 lines...
◐ There's one failing check: `Xamarin.Android-PR` with a link to `https://devdiv.visualstudio.com/0bdbc590-a062-4c3f-b0f6-9383f67865ee/_build/results?buildId=13452282`. Let me parse this:
- Org: `devdiv` (or `devdiv.visualstudio.com`)
- Project: `0bdbc590-a062-4c3f-b0f6-9383f67865ee`
- Build ID: `13452282`
Let me get the build timeline.
● Get failed build timeline records
$ az devops invoke --area build --resource timeline --route-parameters buildId=13452282 --org https://devdiv.visualstudio.com --project 0bdbc590-a062-4c3f-b0f6-9383f67865ee --query "records[?result=='failed']
| [].{name:name, type:type, issues:issues, errorCount:errorCount}" --output json 2>&1
└ 6 lines...
● Get failed build timeline records with project as route param
$ az devops invoke --area build --resource timeline --route-parameters project=0bdbc590-a062-4c3f-b0f6-9383f67865ee buildId=13452282 --org https://devdiv.visualstudio.com --query "records[?result=='failed'] |
[].{name:name, type:type, issues:issues, errorCount:errorCount}" --output json 2>&1
└ 182 lines...
◐ Now I have a clear picture of the failure. The `Xamarin.Android-PR` check is failing in the `Pack .NET Maui` task with 78 RS0016 errors. These are all Roslyn public API analyzer errors — the new symbols
added by this PR are not registered in the declared public API files.
The affected files are in `src\AI\src\Essentials.AI\Platform\MaciOS\`:
1. `NLEmbeddingExtensions.cs` - `NLEmbeddingExtensions` class and `AsIEmbeddingGenerator` method
2. `NLEmbeddingGenerator.cs` - `NLEmbeddingGenerator` class, constructor, `GenerateAsync`, `Dispose`
3. `AppleIntelligenceChatClient.cs` - `AppleIntelligenceChatClient` class, constructor, `GetResponseAsync`, `GetStreamingResponseAsync`
The root cause is that these new public types and members need to be registered in the `PublicAPI.Shipped.txt` or `PublicAPI.Unshipped.txt` file tracked by the `[MAUIAI0001]` analyzer — this is actually a
MAUI repo issue where the integration build is failing. The fix requires updating the public API declaration file.
● The failing check is Xamarin.Android-PR — specifically the "Pack .NET Maui" task, with 78 RS0016 errors.
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Root Cause: Missing Public API Declarations
The new Apple Intelligence / MAUI AI types added by this PR are not registered in the public API surface file. All errors are RS0016 from the Roslyn PublicApiAnalyzers:
"Symbol is not part of the declared public API"
The affected symbols are all in src\AI\src\Essentials.AI\Platform\MaciOS\ (a MAUI repo path, not this repo):
┌──────────────────────────────────┬─────────────────────────────────────────────────────────────────────┐
│ File │ New Public Symbols │
├──────────────────────────────────┼─────────────────────────────────────────────────────────────────────┤
│ AppleIntelligenceChatClient.cs │ Class, constructor, GetResponseAsync, GetStreamingResponseAsync │
├──────────────────────────────────┼─────────────────────────────────────────────────────────────────────┤
│ NLEmbeddingGenerator.cs │ Class, constructor, GenerateAsync, Dispose │
├──────────────────────────────────┼─────────────────────────────────────────────────────────────────────┤
│ NLEmbeddingExtensions.cs │ Class, AsIEmbeddingGenerator extension method │
└──────────────────────────────────┴─────────────────────────────────────────────────────────────────────┘
Fix
The PublicAPI.Unshipped.txt file for the [MAUIAI0001] project needs to be updated to include all these new symbols. This is done in the MAUI repo — you need to add entries like:
[MAUIAI0001]Microsoft.Maui.Essentials.AI.AppleIntelligenceChatClient
[MAUIAI0001]Microsoft.Maui.Essentials.AI.AppleIntelligenceChatClient.AppleIntelligenceChatClient() -> void
[MAUIAI0001]Microsoft.Maui.Essentials.AI.AppleIntelligenceChatClient.GetResponseAsync(...) -> ...
...
to the PublicAPI.Unshipped.txt file in that project. The full list of symbols to add can be found in the 78 error messages from the build log.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new GitHub Copilot skill to help investigate Azure DevOps pipeline build failures by resolving PR URLs to failing checks, querying timelines/logs via Azure DevOps APIs, and analyzing .binlog artifacts with binlogtool.
Changes:
- Introduces a new
azdo-build-investigatorskill definition and workflow documentation. - Documents prerequisites (
gh,az+azure-devopsextension,binlogtool) and a step-by-step investigation process. - Provides common error-pattern heuristics tailored to the dotnet/android repo.
- Add project to --route-parameters for timeline and logs endpoints
- Use {orgUrl} extracted from input instead of hardcoding dev.azure.com
- Condense skill content to reduce token usage
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Clarify prerequisite requirements (az always, gh/binlogtool conditional) - Fix logs endpoint to save raw text to file instead of requesting JSON Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a Copilot skill for investigating Azure DevOps pipeline build failures. Given a build URL, it fetches run details, identifies failed jobs/tasks via the timeline API, downloads .binlog artifacts, and analyzes them with binlogtool to find root causes.
Example usage: