Skip to content

Link-lang check produces no per-issue console output in GitHub Actions logs #629

@WilliamBerryiii

Description

@WilliamBerryiii

Summary

When the link-lang check workflow fails, contributors viewing the GitHub Action run see only a count summary (Found N URLs with 'en-us' language paths) and a generic failure message. No per-issue details (file, line, URL) appear in the console log. Contributors must download the link-lang-check-results.json artifact and manually parse it to understand what failed.

Current Behavior

The Invoke-LinkLanguageCheckCore function in scripts/linting/Invoke-LinkLanguageCheck.ps1 writes CI annotations, a step summary, and a JSON log file — but the console output (what a user sees when expanding the GitHub Actions step log) contains only:

🔍 Checking for URLs with language paths...
Found 3 URLs with 'en-us' language paths

The "Check results and fail if needed" step in link-lang-check.yml then outputs only:

Link language check failed and soft-fail is false. Failing the job.

Neither step tells the contributor which files, lines, or URLs are failing.

Compounding factors

  1. The "Run Link Language Check" step uses continue-on-error: true, so it appears green in the UI — the user is less likely to expand it.
  2. The actual failing step ("Check results and fail if needed") has zero useful content.
  3. While CI annotations exist, they only appear in the PR Files Changed tab, not in the Actions log view.

Expected Behavior

The console output should include per-issue detail lines so contributors can immediately identify failures, matching the pattern established by Invoke-PSScriptAnalyzer.ps1:

🔍 Checking for URLs with language paths...
Found 3 URLs with 'en-us' language paths

📄 docs/guide.md
  ⚠️ Line 15: https://docs.microsoft.com/en-us/azure/overview
  ⚠️ Line 42: https://docs.microsoft.com/en-us/dotnet/core

📄 README.md
  ⚠️ Line 7: https://learn.microsoft.com/en-us/windows/overview

❌ Link language check failed with 3 issue(s) in 2 file(s).

Root Cause

In Invoke-LinkLanguageCheck.ps1, the issues-found branch (approximately lines 52-103) immediately jumps from a count summary to CI annotations and JSON export. There is no Write-Host loop that outputs per-issue details grouped by file — unlike Invoke-PSScriptAnalyzer.ps1 which writes 📄 Analyzing: $filePath headers and per-issue ⚠️ [Severity] Rule: Message (Line N) lines.

Files Requiring Changes

File Change
scripts/linting/Invoke-LinkLanguageCheck.ps1 Add per-issue Write-Host output grouped by file between the count summary (line 52) and the annotation loop (line 54)
.github/workflows/link-lang-check.yml Update the "Check results and fail if needed" step to read the JSON log and print a failure summary instead of a generic message
scripts/tests/linting/Invoke-LinkLanguageCheck.Tests.ps1 Add test coverage for the new console output behavior

Reproduction Steps

  1. Introduce a language-path URL (e.g., https://docs.microsoft.com/en-us/azure) into any tracked markdown file.
  2. Run npm run lint:links.
  3. Observe that the console output contains only a count — no file, line, or URL details.
  4. Open the generated logs/link-lang-check-results.json to find the actual details (the only way to discover what failed).

Fix Guidance

Follow the pattern from Invoke-PSScriptAnalyzer.ps1 (lines 91-127):

# After the count summary line, before the annotation loop
$groupedByFile = $results | Group-Object -Property file
foreach ($fileGroup in $groupedByFile) {
    Write-Host "`n📄 $($fileGroup.Name)" -ForegroundColor Cyan
    foreach ($item in $fileGroup.Group) {
        Write-Host "  ⚠️ Line $($item.line_number): $($item.original_url)" -ForegroundColor Yellow
    }
}

For the workflow file's failure step:

if ($env:LINK_LANG_FAILED -eq 'true') {
    $logFile = 'logs/link-lang-check-results.json'
    if (Test-Path $logFile) {
        $data = Get-Content $logFile -Raw | ConvertFrom-Json
        Write-Host "❌ Link language check failed with $($data.summary.total_issues) issue(s) in $($data.summary.files_affected) file(s):" -ForegroundColor Red
        foreach ($issue in $data.issues) {
            Write-Host "  ⚠️ $($issue.file):$($issue.line_number)$($issue.original_url)" -ForegroundColor Yellow
        }
    }
    exit 1
}

Unit Testing and Code Coverage Requirements

Codecov Configuration

The repository enforces an auto-incrementing project coverage threshold (+1% over base) and an 80% patch target (codecov.yml). All new or modified lines in scripts/linting/Invoke-LinkLanguageCheck.ps1 must meet the patch coverage gate.

Pester Coverage

  • Config: scripts/tests/pester.config.ps1 — JaCoCo format, CoveragePercentTarget = 80
  • Coverage path: scripts/linting/ is already in the coverage scan scope
  • Run: npm run test:ps

Current Test Gap

The existing test file scripts/tests/linting/Invoke-LinkLanguageCheck.Tests.ps1 mocks Write-Host but does not verify the content or call count for per-issue output. When adding per-file Write-Host calls and enhancing Write-CIStepSummary content, the tests must be updated to:

  1. Add -ParameterFilter mocks for Write-Host to assert that each failing file/issue is logged individually (not just a summary count).
  2. Add Should -Invoke assertions verifying Write-Host is called once per violation with the expected message content.
  3. Mock Write-CIAnnotation and verify it is invoked with correct -Level and -Message for each detected issue.
  4. Mock Write-CIStepSummary and assert the markdown content includes the expected violation details.

RPI Phase Testing Guidance

  • Research: Audit existing test coverage for CI output paths in Invoke-LinkLanguageCheck.Tests.ps1.
  • Plan: Design test cases covering per-file Write-Host output, Write-CIAnnotation per issue, and Write-CIStepSummary content.
  • Implement: Add mocks and assertions for all new CI output calls; verify npm run test:ps passes with patch coverage ≥ 80%.
  • Review: Confirm no coverage regressions in the pester flag on Codecov.

RPI Framework Starter Prompts

Use these prompts with the RPI framework agents to walk through the full implementation. Each prompt is designed to be used directly with the corresponding agent.

Phase 1: Research (task-researcher)

Expand Research Prompt
Topic: Improve link-lang check console output for GitHub Actions visibility

Research the following areas in the hve-core repository:

1. **Current output gap**: Read `scripts/linting/Invoke-LinkLanguageCheck.ps1` and document
   exactly what `Write-Host` calls exist in the issues-found branch (lines 50-103). Compare
   with the no-issues branch. Identify the gap: no per-issue detail lines are written to the
   console.

2. **Gold-standard pattern**: Read `scripts/linting/Invoke-PSScriptAnalyzer.ps1` (lines 80-140)
   and document the per-file/per-issue `Write-Host` pattern it uses. Note the file header format
   (`📄 Analyzing: $filePath`), per-issue format (`⚠️ [Severity] Rule: Message (Line N)`),
   and color coding approach. Also review `scripts/linting/Validate-MarkdownFrontmatter.ps1`
   and its `Write-ValidationConsoleOutput` function in `scripts/linting/Modules/FrontmatterValidation.psm1`
   for a second reference implementation.

3. **Workflow integration**: Read `.github/workflows/link-lang-check.yml` and document how the
   `continue-on-error: true` on the run step combined with the generic failure message in the
   "Check results and fail if needed" step compounds the visibility problem.

4. **CIHelpers module**: Read `scripts/lib/Modules/CIHelpers.psm1` and catalog all available
   CI helper functions (`Write-CIAnnotation`, `Set-CIOutput`, `Set-CIEnv`,
   `Write-CIStepSummary`, `Get-CIPlatform`, `ConvertTo-AzureDevOpsEscaped`). Determine if
   any existing helper could be reused for console output formatting or if a new approach is
   needed.

5. **Test coverage**: Read `scripts/tests/linting/Invoke-LinkLanguageCheck.Tests.ps1` and
   identify what is currently tested. Note that `Write-Host` calls are mocked but their
   content is not verified. Document what new test cases are needed: verifying per-issue
   `Write-Host` calls are made with correct file/line/URL content, verifying the grouped-by-file
   output structure, and verifying the failure summary line.

6. **Data model**: Read `scripts/linting/Link-Lang-Check.ps1` and confirm the JSON output
   schema. Each result object has `file`, `line_number`, and `original_url` properties.
   Confirm these are the fields available for console output formatting.

7. **Codecov configuration**: Read `codecov.yml` and `scripts/tests/pester.config.ps1` to
   document patch coverage target (80%), JaCoCo format, and coverage scan paths. Confirm
   `scripts/linting/` is in scope.

Applicable instructions files to read:
- `.github/instructions/hve-core/workflows.instructions.md` (workflow conventions)
- `.github/instructions/commit-message.instructions.md` (commit standards)

Write findings to `.copilot-tracking/research/` with evidence and file path references.

Phase 2: Plan (task-planner)

Expand Planning Prompt
Research: Use the most recent link-lang console output research document from
`.copilot-tracking/research/`.

Create an implementation plan with these phases:

## Phase 1: Add Per-Issue Console Output to Invoke-LinkLanguageCheck.ps1

**Target file**: `scripts/linting/Invoke-LinkLanguageCheck.ps1`

Steps:
1. In `Invoke-LinkLanguageCheckCore`, locate the issues-found branch after line 52
   (`Write-Host "Found $($results.Count) URLs..."`) and before line 54 (the
   `foreach ($item in $results)` annotation loop).
2. Insert a `Write-Host` block that groups results by file and outputs per-issue detail
   lines, following the PSScriptAnalyzer pattern:
   - Group results using `$results | Group-Object -Property file`
   - For each file group: `Write-Host "📄 $($fileGroup.Name)" -ForegroundColor Cyan`
   - For each issue in group: `Write-Host "  ⚠️ Line $($item.line_number): $($item.original_url)" -ForegroundColor Yellow`
3. After the annotation and JSON export section, before `return 1`, add a failure summary:
   `Write-Host "❌ Link language check failed with $($results.Count) issue(s) in $($uniqueFiles.Count) file(s)." -ForegroundColor Red`

Validation: Run `npm run lint:links` locally. Console output should now show per-file,
per-issue details between the count summary and the final failure status.

## Phase 2: Update Workflow Failure Step

**Target file**: `.github/workflows/link-lang-check.yml`

Steps:
1. Replace the "Check results and fail if needed" step's generic message with a content-aware
   summary that reads `logs/link-lang-check-results.json` and prints per-issue details.
2. Preserve the existing `if: ${{ !inputs.soft-fail }}` condition.
3. Preserve the `exit 1` for CI failure signaling.

Validation: The workflow step should output a useful failure summary when `LINK_LANG_FAILED`
is true.

## Phase 3: Update Pester Tests

**Target file**: `scripts/tests/linting/Invoke-LinkLanguageCheck.Tests.ps1`

Steps:
1. In the "Issues found in link scan" context, update the existing `Write-Host` mock to
   capture calls with `-ParameterFilter` so content can be verified.
2. Add test: "Writes per-file headers to console" — verify `Write-Host` is called with
   each unique file name from the mock data (`docs/a.md`, `docs/b.md`).
3. Add test: "Writes per-issue detail lines to console" — verify `Write-Host` is called
   with line number and URL for each issue.
4. Add test: "Writes failure summary to console" — verify `Write-Host` is called with the
   failure summary containing the total issue count and file count.
5. In the "No issues found" context, verify no per-issue detail lines are written (only
   the success message).
6. Verify all new and existing tests maintain patch coverage ≥ 80% for the codecov gate.

Validation: Run `npm run test:ps` or
`Invoke-Pester scripts/tests/linting/Invoke-LinkLanguageCheck.Tests.ps1 -Output Detailed`.
All new and existing tests pass.

## Phase 4: Final Validation

Steps:
1. Run `npm run lint:links` and confirm console output includes per-issue details.
2. Run `npm run test:ps` and confirm all Pester tests pass.
3. Run `npm run lint:ps` to confirm PowerShell script changes pass PSScriptAnalyzer.
4. Verify the JSON log file output is unchanged (no regression).
5. Verify patch coverage meets the 80% codecov target for modified files.

Applicable instructions files:
- `.github/instructions/hve-core/workflows.instructions.md`
- `.github/instructions/commit-message.instructions.md`

Write the plan to `.copilot-tracking/plans/`.

Phase 3: Implement (task-implementor)

Expand Implementation Prompt
Plan: Use the most recent link-lang console output plan from `.copilot-tracking/plans/`.

Read and follow these instructions files before making changes:
- `.github/instructions/hve-core/workflows.instructions.md` (workflow file conventions)
- `.github/instructions/commit-message.instructions.md` (commit message format)

Execute each phase of the plan. Key implementation details:

### Phase 1: Invoke-LinkLanguageCheck.ps1

Open `scripts/linting/Invoke-LinkLanguageCheck.ps1`. In the `Invoke-LinkLanguageCheckCore`
function, find the issues-found branch. After the count summary `Write-Host` on line 52 and
before the annotation `foreach` on line 54, insert:

$groupedByFile = $results | Group-Object -Property file
foreach ($fileGroup in $groupedByFile) {
Write-Host "`n📄 $($fileGroup.Name)" -ForegroundColor Cyan
foreach ($item in $fileGroup.Group) {
Write-Host " ⚠️ Line $($item.line_number): $($item.original_url)" -ForegroundColor Yellow
}
}
Write-Host ""


Before `return 1` at the end of the issues-found branch, add:

Write-Host "n❌ Link language check failed with $($results.Count) issue(s) in $($uniqueFiles.Count) file(s).n" -ForegroundColor Red


Note: `$uniqueFiles` is already computed later in the function — you may need to move its
computation earlier (before the console output block) so it's available.

### Phase 2: link-lang-check.yml

Open `.github/workflows/link-lang-check.yml`. Replace the "Check results and fail if needed"
step body with:

if ($env:LINK_LANG_FAILED -eq 'true') {
$logFile = 'logs/link-lang-check-results.json'
if (Test-Path $logFile) {
$data = Get-Content $logFile -Raw | ConvertFrom-Json
Write-Host "`n❌ Link language check failed with $($data.summary.total_issues) issue(s) in $($data.summary.files_affected) file(s):" -ForegroundColor Red
foreach ($issue in $data.issues) {
Write-Host " ⚠️ $($issue.file):$($issue.line_number) — $($issue.original_url)" -ForegroundColor Yellow
}
Write-Host ""
}
else {
Write-Host "Link language check failed and soft-fail is false. Failing the job."
}
exit 1
}


### Phase 3: Pester Tests

Open `scripts/tests/linting/Invoke-LinkLanguageCheck.Tests.ps1`. In the "Issues found in link
scan" context:

1. Change the `Write-Host` mock to use `-ParameterFilter` variants or capture the call arguments.
2. Add tests verifying:
   - `Write-Host` is called with strings containing `docs/a.md` and `docs/b.md` (file headers)
   - `Write-Host` is called with strings containing `Line 1:` and `Line 2:` (per-issue details)
   - `Write-Host` is called with a string matching `*failed with 2 issue*` (failure summary)
3. In the "No issues found" context, verify `Write-Host` is NOT called with any `📄` or `⚠️` content.
4. Ensure all new code paths have test coverage to meet the 80% codecov patch target.
   Mock `Write-CIAnnotation` and `Write-CIStepSummary` and add `Should -Invoke` assertions.

Update the changes document in `.copilot-tracking/changes/` after each phase.

Phase 4: Review (task-reviewer)

Expand Review Prompt
Review the link-lang console output implementation using the plan and changes documents from
`.copilot-tracking/plans/` and `.copilot-tracking/changes/`.

Validate each of these criteria:

1. **Console output correctness**: Run `npm run lint:links` against a file containing a
   language-path URL. Verify the console output shows:
   - Count summary line
   - Per-file headers with 📄 emoji
   - Per-issue detail lines with ⚠️ emoji, line number, and URL
   - Failure summary line with ❌ emoji and counts

2. **Pattern consistency**: Compare the new `Write-Host` output format with
   `scripts/linting/Invoke-PSScriptAnalyzer.ps1` (lines 91-127). Verify:
   - File headers use Cyan foreground color
   - Issue detail lines use Yellow foreground color
   - Failure summary uses Red foreground color
   - Emoji usage is consistent with the established pattern

3. **Workflow integration**: Review `.github/workflows/link-lang-check.yml` changes:
   - The `if: ${{ !inputs.soft-fail }}` condition is preserved
   - `exit 1` is preserved for CI failure signaling
   - The JSON file path matches what `Invoke-LinkLanguageCheck.ps1` writes
   - Fallback message exists when JSON file is missing

4. **Test coverage**: Run Pester tests and verify:
   - All existing tests still pass (no regressions)
   - New tests verify per-file `Write-Host` calls
   - New tests verify per-issue `Write-Host` calls
   - New tests verify failure summary `Write-Host` call
   - No-issues path is verified to not emit detail lines
   - Patch coverage meets the 80% codecov gate for modified files

5. **No regressions**: Verify:
   - JSON log file output is unchanged
   - CI annotations still work (`Write-CIAnnotation` calls unmodified)
   - Step summary still works (`Write-CIStepSummary` calls unmodified)
   - `Set-CIOutput` and `Set-CIEnv` calls unmodified
   - Exit codes unchanged (1 for issues, 0 for clean)

6. **PSScriptAnalyzer compliance**: Run `npm run lint:ps` and verify no new violations
   in the modified script file.

7. **Codecov compliance**: Verify patch coverage for modified files meets the 80% target.
   Confirm no coverage regressions in the `pester` flag on Codecov.

8. **Convention compliance**: Verify changes follow:
   - `.github/instructions/hve-core/workflows.instructions.md` (workflow conventions)
   - `.github/instructions/commit-message.instructions.md` (commit standards)

Write the review log to `.copilot-tracking/reviews/`.

References

  • Affected script: scripts/linting/Invoke-LinkLanguageCheck.ps1 (the wrapper with CI integration)
  • Core scanner: scripts/linting/Link-Lang-Check.ps1 (no changes needed — produces correct JSON)
  • Workflow: .github/workflows/link-lang-check.yml
  • Tests: scripts/tests/linting/Invoke-LinkLanguageCheck.Tests.ps1
  • Gold-standard reference: scripts/linting/Invoke-PSScriptAnalyzer.ps1 (lines 91-127 for per-issue output pattern)
  • CI helpers module: scripts/lib/Modules/CIHelpers.psm1
  • Codecov config: codecov.yml (80% patch target, auto +1% project threshold)
  • Pester config: scripts/tests/pester.config.ps1 (JaCoCo format, 80% coverage target)
  • npm script: npm run lint:links

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinggithub-actionsGitHub Actions workflowsgood first issueGood for newcomerslintingLinting rules and validationscriptsPowerShell, Bash, or Python scripts

    Type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions