Skip to content

feat(scripts): extract embedded PowerShell from workflows into testable scripts#738

Merged
WilliamBerryiii merged 2 commits intomainfrom
feat/736-workflow-ps-testing-pilot
Feb 23, 2026
Merged

feat(scripts): extract embedded PowerShell from workflows into testable scripts#738
WilliamBerryiii merged 2 commits intomainfrom
feat/736-workflow-ps-testing-pilot

Conversation

@WilliamBerryiii
Copy link
Copy Markdown
Member

Description

Extract embedded PowerShell from two GitHub Actions workflows into standalone, testable scripts as a pilot for the workflow-embedded testing pattern (issue #736). Both workflows retain their step structure with thin wrapper calls replacing inline code. The extracted scripts follow a Core-function + script-guard architecture using $MyInvocation.InvocationName -ne '.' for dot-source testability, adopt CIHelpers.psm1 for injection-safe CI output, and include comprehensive Pester test suites achieving 100% code coverage across 37 tests.

Extracted Scripts

Get-ChangedTestFiles.ps1 (extracted from pester-tests.yml) runs git diff --name-only against the base branch, resolves corresponding .Tests.ps1 files across scripts/tests/ and skill directories, includes directly changed tests, and deduplicates. Returns a [PSCustomObject] with HasChanges, TestPaths, and ChangedFiles.

Find-CollectionManifests.ps1 (extracted from extension-package.yml) discovers *.collection.yml manifests, filters by maturity and release channel (deprecated always excluded, experimental excluded for Stable), and returns a [PSCustomObject] with MatrixJson, MatrixItems, and Skipped. Requires the PowerShell-Yaml module.

Workflow Modifications

pester-tests.yml had ~63 lines of embedded PowerShell removed and replaced with a single-line call to Get-ChangedTestFiles.ps1. extension-package.yml had ~41 lines removed and replaced with a call to Find-CollectionManifests.ps1.

Test Coverage

37 total Pester tests (16 + 21) validate both scripts with isolated GUID-based temp directories. Coverage configuration in pester.config.ps1 was updated to include scripts/tests/ in the coverage directory list.

Related Issue(s)

Related to #736

Type of Change

Select all that apply:

Code & Documentation:

  • Bug fix (non-breaking change fixing an issue)
  • New feature (non-breaking change adding functionality)
  • Breaking change (fix or feature causing existing functionality to change)
  • Documentation update

Infrastructure & Configuration:

  • GitHub Actions workflow
  • Linting configuration (markdown, PowerShell, etc.)
  • Security configuration
  • DevContainer configuration
  • Dependency update

AI Artifacts:

  • Reviewed contribution with prompt-builder agent and addressed all feedback
  • Copilot instructions (.github/instructions/*.instructions.md)
  • Copilot prompt (.github/prompts/*.prompt.md)
  • Copilot agent (.github/agents/*.agent.md)
  • Copilot skill (.github/skills/*/SKILL.md)

Other:

  • Script/automation (.ps1, .sh, .py)
  • Other (please describe):

Testing

All extracted scripts are fully tested with Pester 5.7.1:

  • Get-ChangedTestFiles.Tests.ps1: 16 tests covering empty diff, matching test resolution, missing test handling, directly changed test inclusion, skill directory discovery (depth 1 and 2), missing skills directory, deduplication, git diff failure, and script guard CI integration
  • Find-CollectionManifests.Tests.ps1: 21 tests covering single/multiple manifest discovery, deprecated/experimental filtering, channel-specific behavior, skipped collection tracking with reasons, missing name/maturity field defaults, valid JSON output, and script guard CI execution
  • All tests use isolated GUID-based temp directories with cleanup
  • 100% code coverage achieved on both scripts
  • Validated via npm run test:ps with pester-summary.json confirming 37/37 pass

Checklist

Required Checks

  • Documentation is updated (if applicable)
  • Files follow existing naming conventions
  • Changes are backwards compatible (if applicable)
  • Tests added for new functionality (if applicable)

Required Automated Checks

The following validation commands must pass before merging:

  • Markdown linting: npm run lint:md
  • Spell checking: npm run spell-check
  • Frontmatter validation: npm run lint:frontmatter
  • Skill structure validation: npm run validate:skills
  • Link validation: npm run lint:md-links
  • PowerShell analysis: npm run lint:ps
  • Plugin freshness: npm run plugin:generate

Security Considerations

  • This PR does not contain any sensitive or NDA information
  • Any new dependencies have been reviewed for security issues
  • Security-related scripts follow the principle of least privilege

Additional Notes

This is a pilot for the workflow-embedded PowerShell testing pattern. The Core-function + script-guard architecture enables unit testing of CI logic without executing CI side effects. If validated, this pattern can be applied to other workflows with embedded PowerShell.

…le scripts

- extract Get-ChangedTestFiles.ps1 from pester-tests.yml with 16 Pester tests
- extract Find-CollectionManifests.ps1 from extension-package.yml with 21 Pester tests
- replace workflow embedded blocks with thin wrappers calling standalone scripts
- add tests directory to pester.config.ps1 coverage paths

🧪 - Generated by Copilot
@WilliamBerryiii WilliamBerryiii requested a review from a team as a code owner February 23, 2026 07:04
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Feb 23, 2026

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Feb 23, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.68%. Comparing base (780cf22) to head (7948261).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #738      +/-   ##
==========================================
+ Coverage   86.59%   86.68%   +0.09%     
==========================================
  Files          25       26       +1     
  Lines        4893     4936      +43     
==========================================
+ Hits         4237     4279      +42     
- Misses        656      657       +1     
Flag Coverage Δ
pester 86.68% <100.00%> (+0.09%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
scripts/extension/Find-CollectionManifests.ps1 100.00% <100.00%> (ø)

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Contributor

@katriendg katriendg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really good idea, extracting out and testability. Me like!

…testing-pilot

# Conflicts:
#	scripts/tests/pester.config.ps1
@WilliamBerryiii WilliamBerryiii merged commit 585b241 into main Feb 23, 2026
21 checks passed
@WilliamBerryiii WilliamBerryiii deleted the feat/736-workflow-ps-testing-pilot branch February 23, 2026 22:19
WilliamBerryiii pushed a commit that referenced this pull request Feb 28, 2026
## Pre-Release 3.1.44

### ✨ Features

- add Docusaurus 3 documentation site with GitHub Pages deployment
(#680)
- add workflow permissions validation for OpenSSF Scorecard compliance
(#759)
- add DT coach return path handoff to task-researcher (#591) (#758)
- add DT subagent handoff workflow instructions (#592) (#757)
- create dt-method-06-deep.instructions.md (#602) (#748)
- create dt-method-05-deep.instructions.md (#747)
- add DT-aware task-implementor context instructions (#755)
- extract embedded PowerShell from workflows into testable scripts
(#738)
- add gitleaks binary-based secret scanning as PR gate (#734)
- add SBOM generation, attestation, and diff tooling to release pipeline
(#730)
- add dt-learning-tutor agent for DT education (#662)
- add DT image prompt generation guidance for Method 5 (#726)
- add DT-aware task-reviewer review context (#714)
- add dt-method-next routing prompt (#713)
- create dt-method-04-deep.instructions.md (#709)
- add Implementation Space exit handoff prompt for DT workflows (#708)
- add Write-CIStepSummary markdown table to Test-SHAStaleness github
output (#660)
- add dt-handoff-solution-space prompt for Solution Spac… (#707)

### 🐛 Bug Fixes

- update sidebar link color to meet WCAG AA contrast requirements (#814)
- harden even/odd versioning against regression and syntax errors (#816)
- replace even/odd versioning with SemVer -rc.N suffixes (#811)
- ensure prerelease label exists before PR creation (#806)
- replace Docusaurus favicons with Microsoft logo (#808)
- add missing subagents and shared instructions to collection manifests
(#804)
- standardize file path conventions for copilot-tracking output (#784)
- enforce project-scoped artifact isolation across DT files (#766)
- add top-level permissions to copilot-setup-steps.yml (#760)
- update broken file directives and markdown links after collection
directory reorg (#743)
- add pre-release companion pipeline with even/odd versioning (#735)
- exclude auto-generated CHANGELOG.md from spell check (#756)
- add job-level permissions to extension-publish.yml (#729)
- resolve handoff dependencies using display names (#727)
- add job-level permissions to validate-version in
extension-publish-prerelease (#731)
- replace parent-directory VS Code settings paths with per-subdirectory
enumeration (#732)

### 📚 Documentation

- add Design Thinking documentation and DT-to-RPI handoff (#789)
- add customization guides for HVE Core artifacts (#772)
- reconcile documentation against implementation (#771)
- document accepted Token-Permissions risks and add
lint:dependency-pinning (#763)
- add Design Thinking section to hve-core-all collection description
(#762)

### ♻️ Refactoring

- move collection scripts from plugins to collections (#728)
- remove duplicate git diff logic in frontmatter validator (#473)

### 🔧 Maintenance

- bump basic-ftp from 5.0.5 to 5.2.0 (#780)
- standardize script path references in SKILL.md files (#768)
- bump the github-actions group across 1 directory with 2 updates (#752)

---
*Managed automatically by pre-release workflow.*

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
WilliamBerryiii pushed a commit that referenced this pull request Feb 28, 2026
## Pre-Release 3.1.46

### ✨ Features

- add Docusaurus 3 documentation site with GitHub Pages deployment
(#680)
- add workflow permissions validation for OpenSSF Scorecard compliance
(#759)
- add DT coach return path handoff to task-researcher (#591) (#758)
- add DT subagent handoff workflow instructions (#592) (#757)
- create dt-method-06-deep.instructions.md (#602) (#748)
- create dt-method-05-deep.instructions.md (#747)
- add DT-aware task-implementor context instructions (#755)
- extract embedded PowerShell from workflows into testable scripts
(#738)
- add gitleaks binary-based secret scanning as PR gate (#734)
- add SBOM generation, attestation, and diff tooling to release pipeline
(#730)
- add dt-learning-tutor agent for DT education (#662)
- add DT image prompt generation guidance for Method 5 (#726)
- add DT-aware task-reviewer review context (#714)
- add dt-method-next routing prompt (#713)
- create dt-method-04-deep.instructions.md (#709)
- add Implementation Space exit handoff prompt for DT workflows (#708)
- add Write-CIStepSummary markdown table to Test-SHAStaleness github
output (#660)
- add dt-handoff-solution-space prompt for Solution Spac… (#707)

### 🐛 Bug Fixes

- update prerelease publish to use even/odd convention (#822)
- update sidebar link color to meet WCAG AA contrast requirements (#814)
- harden even/odd versioning against regression and syntax errors (#816)
- replace even/odd versioning with SemVer -rc.N suffixes (#811)
- ensure prerelease label exists before PR creation (#806)
- replace Docusaurus favicons with Microsoft logo (#808)
- add missing subagents and shared instructions to collection manifests
(#804)
- standardize file path conventions for copilot-tracking output (#784)
- enforce project-scoped artifact isolation across DT files (#766)
- add top-level permissions to copilot-setup-steps.yml (#760)
- update broken file directives and markdown links after collection
directory reorg (#743)
- add pre-release companion pipeline with even/odd versioning (#735)
- exclude auto-generated CHANGELOG.md from spell check (#756)
- add job-level permissions to extension-publish.yml (#729)
- resolve handoff dependencies using display names (#727)
- add job-level permissions to validate-version in
extension-publish-prerelease (#731)
- replace parent-directory VS Code settings paths with per-subdirectory
enumeration (#732)

### 📚 Documentation

- add Design Thinking documentation and DT-to-RPI handoff (#789)
- add customization guides for HVE Core artifacts (#772)
- reconcile documentation against implementation (#771)
- document accepted Token-Permissions risks and add
lint:dependency-pinning (#763)
- add Design Thinking section to hve-core-all collection description
(#762)

### ♻️ Refactoring

- move collection scripts from plugins to collections (#728)
- remove duplicate git diff logic in frontmatter validator (#473)

### 🔧 Maintenance

- pre-release 3.1.44 (#819)
- bump basic-ftp from 5.0.5 to 5.2.0 (#780)
- standardize script path references in SKILL.md files (#768)
- bump the github-actions group across 1 directory with 2 updates (#752)

---
*Managed automatically by pre-release workflow.*

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants