Summary

Test-ActionVersionConsistency.ps1 validates SHA-pinned action version comment consistency across GitHub Actions workflows, but its CI integration is minimal: Write-CIAnnotation fires only in the catch block (line 425) for fatal errors, and Write-CIStepSummary is never called. Contributors see per-violation Write-Host output in the console log but get no inline PR annotations and no job summary tab content.

Current Behavior

• Console output: Per-violation Write-Host via a log helper function (line 96) with color coding. ✅
• CI annotations: Write-CIAnnotation only in catch block (line 425) — fires on fatal script errors, not per-violation. ❌
• Step summary: No Write-CIStepSummary call anywhere in the script. ❌
• Workflow: No dedicated workflow file exists — the script runs via npm run lint:version-consistency only.

Expected Behavior

1. Each version mismatch or missing comment should produce a Write-CIAnnotation (level Warning) with the workflow file path, action reference, and description of the inconsistency — so GitHub renders inline annotations on PR diffs.
2. A Write-CIStepSummary should produce a markdown table in the job summary tab showing all scanned workflows, the actions checked, and their consistency status.

Root Cause

The script's CI output integration was not extended beyond the catch-block pattern. The validation loop writes to console via Write-Host but never calls Write-CIAnnotation per violation. Write-CIStepSummary is absent entirely.

Files Requiring Changes

Additional Context

• The script imports SecurityClasses.psm1 from scripts/security/Modules/SecurityClasses.psm1.
• The CIHelpers module is already available — no new imports needed.
• Write-Host is safe — PSAvoidUsingWriteHost is explicitly excluded in scripts/linting/PSScriptAnalyzer.psd1.
• No workflow file changes needed since the script has no dedicated workflow.

Fix Guidance

Per-Violation Annotations

In the validation results processing section, after each Write-Host violation line:

Write-CIAnnotation -Message "Action version mismatch: $($violation.Action) in $($violation.File) — SHA comment says $($violation.CommentVersion) but latest is $($violation.ActualVersion)" `
    -Level Warning `
    -File $violation.File

Step Summary

After all validations complete and before the exit decision:

$summaryLines = @(
    "## Action Version Consistency Results"
    ""
    "| Workflow File | Action | Comment Version | Status |"
    "|-------------|--------|----------------|--------|"
)

foreach ($result in $allResults) {
    $status = if ($result.IsConsistent) { "✅ Consistent" } else { "❌ Mismatch" }
    $summaryLines += "| $($result.File) | $($result.Action) | $($result.CommentVersion) | $status |"
}

$summaryLines += ""
$summaryLines += "**Total**: $($allResults.Count) actions checked, $($inconsistentCount) inconsistent"

Write-CIStepSummary -Summary ($summaryLines -join "`n")

Unit Testing and Code Coverage Requirements

Codecov Configuration

The repository enforces an auto-incrementing project coverage threshold (+1% over base) and an 80% patch target (codecov.yml). All new or modified lines must meet the patch coverage gate.

Pester Coverage

• Config: scripts/tests/pester.config.ps1 — JaCoCo format, CoveragePercentTarget = 80
• Coverage path: scripts/security/ is already in the coverage scan scope
• Run: npm run test:ps

Current Test Gap

The existing test file scripts/tests/security/Test-ActionVersionConsistency.Tests.ps1 has no mocks for Write-CIAnnotation, Write-CIStepSummary, or Write-Host. All CI helper mock infrastructure must be built from scratch:

1. Mock Write-CIAnnotation — add mock with -ParameterFilter to assert correct -File, -Level Warning, and -Message content per violation. Currently only called in catch block.
2. Mock Write-CIStepSummary — add mock and Should -Invoke assertion verifying the markdown summary includes workflow file names, action references, and consistency status.
3. Mock Write-Host — add content verification for per-violation console output lines.

RPI Phase Testing Guidance

• Research: Audit Test-ActionVersionConsistency.Tests.ps1 for CI helper coverage gaps; document the validation loop structure and catch-block-only annotation pattern.
• Plan: Design test cases for per-violation Write-CIAnnotation, Write-CIStepSummary markdown content, and Write-Host content verification.
• Implement: Add mock infrastructure for all CI helper functions; verify npm run test:ps passes with patch coverage ≥ 80%.
• Review: Confirm no coverage regressions in the pester flag on Codecov.

RPI Framework Starter Prompts

Research Phase

Research CI output coverage in scripts/security/Test-ActionVersionConsistency.ps1. Document: (1) the log helper function (line 96) providing per-violation Write-Host output, (2) the catch-only Write-CIAnnotation pattern (line 425), (3) the absence of Write-CIStepSummary, (4) that no dedicated workflow file exists — script runs via npm run lint:version-consistency, (5) the SecurityClasses.psm1 module dependency, (6) existing Pester test coverage and the complete absence of CI helper mocks, and (7) codecov.yml and scripts/tests/pester.config.ps1 coverage requirements (80% patch target). Compare with Invoke-PSScriptAnalyzer.ps1 as the per-violation annotation reference and Test-DependencyPinning.ps1 as the step summary reference.

Plan Phase

Plan CI output improvements for Test-ActionVersionConsistency.ps1. The plan should cover: (1) inserting Write-CIAnnotation calls with Warning level per version mismatch — determine the correct insertion point in the validation results loop, (2) building a markdown summary table from validation results and calling Write-CIStepSummary after all validations complete, (3) adding Pester tests with mocks for Write-CIAnnotation (per-violation) and Write-CIStepSummary to verify call count, arguments, and markdown content, and (4) ensuring patch coverage meets the 80% codecov gate.

Implement Phase

Implement CI output improvements for Test-ActionVersionConsistency.ps1. Steps: (1) In the validation results loop, after each Write-Host mismatch line, add Write-CIAnnotation -Message "..." -Level Warning -File $file with the action name, expected version, and actual version. (2) After all validations, build a markdown table from results (columns: Workflow File, Action, Comment Version, Status) and call Write-CIStepSummary -Summary $markdown. (3) In Test-ActionVersionConsistency.Tests.ps1, add mocks for Write-CIAnnotation and Write-CIStepSummary with -ParameterFilter assertions; ensure patch coverage ≥ 80%. Run npm run lint:ps and npm run test:ps to validate.

Review Phase

Review CI output changes to Test-ActionVersionConsistency.ps1. Verify: (1) Write-CIAnnotation is called once per version mismatch with Warning level and correct file path, (2) Write-CIStepSummary produces a well-formed markdown table with all scanned actions, (3) existing catch-block Write-CIAnnotation is unmodified, (4) Pester tests mock and assert both CI helpers, (5) npm run lint:ps passes, (6) no regressions in existing tests, (7) patch coverage meets the 80% codecov gate, and (8) output formats align with other scripts for consistency.

References

• Affected script: scripts/security/Test-ActionVersionConsistency.ps1
• Module dependency: scripts/security/Modules/SecurityClasses.psm1
• Tests: scripts/tests/security/Test-ActionVersionConsistency.Tests.ps1
• No dedicated workflow — runs via npm run lint:version-consistency
• Reference implementations: scripts/linting/Invoke-PSScriptAnalyzer.ps1 (per-violation annotations), scripts/security/Test-DependencyPinning.ps1 (step summary)
• CI helpers module: scripts/lib/Modules/CIHelpers.psm1
• Codecov config: codecov.yml (80% patch target, auto +1% project threshold)
• Pester config: scripts/tests/pester.config.ps1 (JaCoCo format, 80% coverage target)