Skip to content

feat(scripts): add per-violation CI annotations and colorized console output#637

Merged
WilliamBerryiii merged 4 commits intomainfrom
feat/632-ci-output-action-version-consistency
Feb 17, 2026
Merged

feat(scripts): add per-violation CI annotations and colorized console output#637
WilliamBerryiii merged 4 commits intomainfrom
feat/632-ci-output-action-version-consistency

Conversation

@WilliamBerryiii
Copy link
Copy Markdown
Member

@WilliamBerryiii WilliamBerryiii commented Feb 17, 2026

Description

Add per-violation Write-CIAnnotation calls and colorized Write-Host output to both Test-DependencyPinning.ps1 and Test-ActionVersionConsistency.ps1. Previously, CI annotations fired only on fatal errors (catch block) and console output used plain Write-Output. Contributors now see inline PR annotations per violation and colored terminal output grouped by file, with severity-based icons and levels.

Also adds Write-CIStepSummary to Test-ActionVersionConsistency.ps1 with a markdown violations table, and removes the redundant ::warning annotation loop from dependency-pinning-scan.yml since the script now handles annotations natively.

Closes #632

Related Issue(s)

Type of Change

Code & Documentation

  • New feature (non-breaking change that adds functionality)
  • Bug fix (non-breaking change that fixes an issue)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Performance improvement

Infrastructure & Configuration

  • CI/CD pipeline changes
  • GitHub Actions workflow changes
  • Build system changes
  • Security configuration
  • Script/automation changes

AI Artifacts

  • Instructions file (.instructions.md)
  • Prompt template (.prompt.md)
  • Agent definition (.agent.md)
  • Skill package (SKILL.md)
  • Copilot configuration (copilot-instructions.md)

Other

  • Refactoring (no functional changes)
  • Code style/formatting
  • Test coverage improvements
  • Other (please describe):

Testing

  • Test-DependencyPinning.Tests.ps1: 68/68 tests passing, 80.67% code coverage
    • Added 7 tests for CI annotations per violation (severity mapping, file/line inclusion, message content, multi-violation)
    • Added 3 tests for Write-PinningLog CI annotation forwarding (Warning, Error, Info-excluded)
    • Added 2 tests for per-violation console output (colored output, success message)
  • Test-ActionVersionConsistency.Tests.ps1: 62/62 tests passing, 94.62% code coverage
    • Added 3 tests for Write-ConsistencyLog CI annotation forwarding
    • Added 5 tests for CI annotations per violation
    • Added 8 tests for CI step summary (pass/fail status, violation counts, table headers, mixed violations)

Checklist

Required Checks

  • I have performed a self-review of my own code
  • My changes generate no new warnings or errors
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

AI Artifact Contributions

  • Frontmatter follows conventions in contributing guide
  • applyTo patterns are specific and accurate
  • No secrets, tokens, or sensitive data in artifact content
  • Tested with GitHub Copilot to verify behavior

Required Automated Checks

  • Linting passes (npm run lint:all)
  • Tests pass (npm run test:ps)

Security Considerations

  • No sensitive information (API keys, passwords, tokens) included
  • Security-related changes have been reviewed for vulnerabilities
  • Dependencies have been checked for known vulnerabilities

Additional Notes

Changes by file

File Summary
scripts/security/Test-DependencyPinning.ps1 Converted Write-PinningLog from Write-Output to Write-Host with color mapping (Info→Cyan, Warning→Yellow, Error→Red, Success→Green). Added CI annotation forwarding for Warning/Error levels. Added per-violation loop emitting Write-CIAnnotation with severity mapping (High→Error, Medium→Warning, Low→Notice) and colorized Write-Host output grouped by file with severity icons.
scripts/security/Test-ActionVersionConsistency.ps1 Added CI annotation forwarding to Write-ConsistencyLog for Warning/Error levels. Added per-violation Write-CIAnnotation loop with severity mapping. Added Write-CIStepSummary with pass/fail status and markdown violations table.
.github/workflows/dependency-pinning-scan.yml Removed redundant ::warning annotation loop (26 lines including ConvertTo-GHAEscaped function) since Test-DependencyPinning.ps1 now handles annotations natively via Write-CIAnnotation.
scripts/tests/security/Test-DependencyPinning.Tests.ps1 Added 12 Pester tests across 3 new contexts for CI annotations per violation, Write-PinningLog CI annotation forwarding, and per-violation console output.
scripts/tests/security/Test-ActionVersionConsistency.Tests.ps1 Added 16 Pester tests across 3 new contexts for Write-ConsistencyLog CI annotation forwarding, CI annotations per violation, and CI step summary content.

🔧 - Generated by Copilot

…ionConsistency

- add per-violation Write-CIAnnotation with severity mapping
- add Write-CIStepSummary with individual violation rows table
- add Pester tests for annotation output and step summary content

🔧 - Generated by Copilot
@WilliamBerryiii WilliamBerryiii requested a review from a team as a code owner February 17, 2026 02:34
Copilot AI review requested due to automatic review settings February 17, 2026 02:34
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Feb 17, 2026

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Feb 17, 2026

Codecov Report

❌ Patch coverage is 65.95745% with 16 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.46%. Comparing base (5fa6328) to head (6070f33).
⚠️ Report is 111 commits behind head on main.

Files with missing lines Patch % Lines
scripts/security/Test-DependencyPinning.ps1 44.44% 15 Missing ⚠️
scripts/security/Test-ActionVersionConsistency.ps1 95.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #637      +/-   ##
==========================================
+ Coverage   85.15%   85.46%   +0.31%     
==========================================
  Files          23       23              
  Lines        4506     4541      +35     
==========================================
+ Hits         3837     3881      +44     
+ Misses        669      660       -9     
Flag Coverage Δ
pester 85.46% <65.95%> (+0.31%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
scripts/security/Test-ActionVersionConsistency.ps1 93.06% <95.00%> (+9.00%) ⬆️
scripts/security/Test-DependencyPinning.ps1 83.19% <44.44%> (-1.02%) ⬇️

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request enhances the Test-ActionVersionConsistency.ps1 script with GitHub Actions CI integration by adding per-violation annotations and a detailed step summary table. The changes make version consistency failures visible directly in the GitHub UI without requiring log file downloads.

Changes:

  • Added per-violation Write-CIAnnotation calls with severity-based level mapping (High → Error, Medium → Warning, default → Notice)
  • Added Write-CIStepSummary with pass/fail status, metrics table, and individual violation details
  • Added 13 comprehensive Pester tests covering annotation emission, suppression, targeting, and step summary content validation

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
scripts/security/Test-ActionVersionConsistency.ps1 Implements CI annotation loop and step summary generation with conditional pass/fail content
scripts/tests/security/Test-ActionVersionConsistency.Tests.ps1 Adds 13 tests verifying annotation levels, file/line targeting, summary status, counts, and table structure

@WilliamBerryiii WilliamBerryiii modified the milestone: v2.4.0 Feb 17, 2026
… output

- convert Write-PinningLog and Write-ConsistencyLog to Write-Host with color mapping
- add CI annotation forwarding for Warning and Error log levels
- emit per-violation Write-CIAnnotation with severity-based levels
- add per-violation colorized Write-Host output grouped by file
- remove redundant ::warning annotation loop from dependency-pinning-scan workflow
- add Pester tests for CI annotation forwarding and console output

🔒 - Generated by Copilot
@WilliamBerryiii WilliamBerryiii changed the title feat(scripts): add CI output to Test-ActionVersionConsistency feat(scripts): add per-violation CI annotations and colorized console output Feb 17, 2026
Copilot AI review requested due to automatic review settings February 17, 2026 21:03
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

- remove first annotation block that hardcoded Warning level for all violations
- keep second block with correct severity-to-level mapping (High→Error, Medium→Warning)
- update tests to assert correct icon and level for High severity violations

🐛 - Generated by Copilot
@WilliamBerryiii WilliamBerryiii merged commit bd7d512 into main Feb 17, 2026
18 checks passed
WilliamBerryiii pushed a commit that referenced this pull request Feb 20, 2026
🤖 I have created a release *beep* *boop*
---


##
[3.0.0](hve-core-v2.3.10...hve-core-v3.0.0)
(2026-02-20)


### ⚠ BREAKING CHANGES

* **skills:** migrate PR reference generation to self-contained skill
([#669](#669))
* restructure RPI collection to HVE Core naming convention
([#668](#668))

### ✨ Features

* **agents:** add agile-coach agent
([#562](#562))
([de8d86c](de8d86c))
* **agents:** add DT coach agent with tiered instruction loading
([#656](#656))
([206d3a7](206d3a7))
* **agents:** add product manager advisor and UX/UI designer agents
([#627](#627))
([539eb8a](539eb8a))
* **agents:** add system architecture reviewer for design trade-offs and
ADR creation ([#626](#626))
([de5cfd6](de5cfd6))
* **build:** pin devcontainer image and align tool parity
([#704](#704))
([6258b1c](6258b1c))
* **design-thinking:** add manufacturing industry context template
([#682](#682))
([ce864bf](ce864bf))
* **instructions:** add DT coaching state protocol for session
persistence ([#654](#654))
([5a5be4e](5a5be4e))
* **instructions:** add dt-coaching-identity ambient instruction
([#642](#642))
([6209a0d](6209a0d))
* **instructions:** add dt-method-01-deep for advanced scope
conversation techniques
([#673](#673))
([cc92ef9](cc92ef9))
* **instructions:** add dt-method-03-deep for advanced input synthesis
techniques ([#676](#676))
([0079a4f](0079a4f))
* **instructions:** add dt-method-09-deep instructions for Method 9
advanced coaching
([#703](#703))
([150b2a6](150b2a6))
* **instructions:** add dt-method-sequencing ambient instruction
([#650](#650))
([e465b2f](e465b2f))
* **instructions:** add dt-quality-constraints and design-thinking
collection ([#645](#645))
([17002bd](17002bd))
* **instructions:** add DT-to-RPI handoff contract specification
([#679](#679))
([87f9962](87f9962))
* **instructions:** add energy industry context template
([#687](#687))
([41088d8](41088d8))
* **instructions:** add healthcare industry context template
([#686](#686))
([b2d5281](b2d5281))
* **instructions:** add Method 1 Scope Conversations coaching knowledge
([#651](#651))
([93e2d48](93e2d48))
* **instructions:** add Method 2 Design Research coaching knowledge
([#652](#652))
([30f7f3b](30f7f3b))
* **instructions:** add Method 3 Input Synthesis coaching knowledge
([#653](#653))
([1efdb7d](1efdb7d))
* **instructions:** add Method 7 High-Fidelity Prototypes coaching
instruction ([#666](#666))
([9233eab](9233eab))
* **instructions:** add pull request instructions for PR generation
workflow ([#706](#706))
([73d23eb](73d23eb))
* **instructions:** create DT curriculum content (9 modules)
([#690](#690))
([9f7378f](9f7378f)),
closes [#617](#617)
* **instructions:** create dt-method-02-deep.instructions.md
([#700](#700))
([4d4d0ca](4d4d0ca))
* **instructions:** create dt-method-06-lofi-prototypes.instructions.md
([#684](#684))
([4d5f757](4d5f757))
* **instructions:** create dt-method-07-deep.instructions.md
([#678](#678))
([d3ec70d](d3ec70d))
* **instructions:** Create dt-method-08-deep.instructions.md
([#683](#683))
([d9e1115](d9e1115))
* **instructions:** create dt-method-08-testing.instructions.md
([#681](#681))
([3008ad8](3008ad8))
* **instructions:** create dt-method-09-iteration.instructions.md
([#685](#685))
([9d7f4f5](9d7f4f5))
* **instructions:** create dt-rpi-research-context.instructions.md
([#689](#689))
([34c7b89](34c7b89))
* **instructions:** create manufacturing reference learning scenario
([#692](#692))
([1bd3994](1bd3994))
* **instructions:** Design Thinking Method 4 brainstorming instruction
file ([#664](#664))
([06f90b0](06f90b0))
* **prompts:** add DT start-project prompt for coaching initialization
([#657](#657))
([ce583d5](ce583d5))
* **prompts:** add dt-resume-coaching prompt for session recovery
([#665](#665))
([11b93cb](11b93cb))
* **prompts:** create dt-handoff-problem-space.prompt.md
([#688](#688))
([277963d](277963d))
* **scripts:** add collection-level maturity field with validation,
gating, and notices
([#697](#697))
([7b1c8e8](7b1c8e8))
* **scripts:** add per-violation CI annotations and colorized console
output ([#637](#637))
([bd7d512](bd7d512))
* **skills:** edit SKILL frontmatter schema, add CI validation, and
documentation ([#625](#625))
([0138a78](0138a78))
* **skills:** mandate unit testing and document language support
([#636](#636))
([9263617](9263617))
* **skills:** migrate PR reference generation to self-contained skill
([#669](#669))
([cf8805f](cf8805f))


### 🐛 Bug Fixes

* **collections:** migrate artifacts into collection-based
subdirectories
([#658](#658))
([dfa5261](dfa5261))
* **instructions:** optimize Phase 1 DT token budgets and close
[#564](https://github.com/microsoft/hve-core/issues/564)/[#565](https://github.com/microsoft/hve-core/issues/565)
gaps ([#675](#675))
([4f42f00](4f42f00))
* **scripts:** add CI annotations and step summary to copyright header
check ([#638](#638))
([5fa6328](5fa6328))
* **scripts:** add grouped link-lang console diagnostics and failure
summary ([#661](#661))
([4d6871f](4d6871f))
* **scripts:** add per-violation Write-Host and Write-CIAnnotation
output to Test-DependencyPinning
([#640](#640))
([9d3b71d](9d3b71d))
* **scripts:** align agent frontmatter schema with VS Code spec
([#469](#469))
([254d445](254d445))
* **scripts:** optimize PSScriptAnalyzer linting performance in WSL2
([#667](#667))
([f120b93](f120b93))
* **scripts:** stabilize YAML display key ordering in collection
manifest ([#701](#701))
([73c0d2c](73c0d2c))
* **scripts:** use text stubs for plugin links when symlinks unavailable
([#695](#695))
([d7650a3](d7650a3))
* **skills:** fix powershell test coverage in pr-reference skill
([#699](#699))
([408e6b7](408e6b7))


### 📚 Documentation

* **dt:** add Method 5 Concepts and Method 6 Lo-Fi Prototypes
instructions ([#693](#693))
([cfdcf11](cfdcf11))
* **hve-guide:** add role-based guides and project lifecycle
documentation ([#663](#663))
([17a85da](17a85da))


### ♻️ Refactoring

* restructure RPI collection to HVE Core naming convention
([#668](#668))
([120dde0](120dde0))
* **scripts:** consolidate duplicate logging into shared SecurityHelpers
module ([#655](#655))
([627a877](627a877))
* **scripts:** use shared SecurityHelpers and CIHelpers modules in
security scripts
([#705](#705))
([3a0baa7](3a0baa7))


### 🔧 Maintenance

* **deps-dev:** bump markdownlint-cli2 from 0.20.0 to 0.21.0 in the
npm-dependencies group
([#609](#609))
([1486dd7](1486dd7))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: hve-core-release-please[bot] <254602402+hve-core-release-please[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Test-ActionVersionConsistency.ps1 missing Write-CIAnnotation per violation and Write-CIStepSummary

4 participants