Summary

Skills today have zero CI test coverage. Contributing guidelines define structural requirements (SKILL.md, dual-platform scripts) but impose no unit testing mandate. CI pipelines are structurally blind to .github/skills/ — Pester coverage paths exclude it, PSScriptAnalyzer targets only scripts/**/*.ps1, and no Python validation pipeline exists. Contributors also have no documented process for requesting build system support when a skill requires a language not yet in CI.

This issue tracks five related gaps identified during a deep review of contributing docs, CI pipelines, and open PRs. PR #625 (in-flight) adds structural validation for skills but does not address runtime testing of skill scripts.

Gaps Addressed

Gap 1 — skills.md has no unit test requirement (High)

docs/contributing/skills.md defines skill directory structure, frontmatter, and SKILL.md sections but:

• No unit test requirement appears anywhere in the acceptance criteria.
• Rejection criteria (lines 50–56) list single-platform, duplicate functionality, and undocumented — missing tests is not a rejection criterion.
• Validation commands reference lint:frontmatter, lint:ps, lint:md — no test:ps or test:skills.

CONTRIBUTING.md lines 161–186 state "New functionality MUST include tests" and require *.Tests.ps1 for PowerShell scripts, but has no skills-specific callout linking that mandate to the skills contributing workflow.

Changes required:

• Add a "Unit Tests" section to docs/contributing/skills.md defining co-located test directory structure, naming convention (*.Tests.ps1), and minimum coverage expectations.
• Add "Missing unit tests for executable scripts" to the rejection criteria list.
• Add a skills-specific callout in CONTRIBUTING.md testing section linking to docs/contributing/skills.md.
• Define the expected skill directory structure with tests:

.github/skills/<skill-name>/
├── SKILL.md
├── scripts/
│   ├── convert.ps1
│   └── convert.sh
├── tests/              ← NEW: required
│   └── convert.Tests.ps1
└── examples/
    └── README.md

Gap 2 — Pester config excludes .github/skills/ from coverage (High)

scripts/tests/pester.config.ps1 lists coverage directories: linting/, security/, dev-tools/, lib/, extension/, plugins/. The .github/skills/ path is absent, meaning skill PowerShell scripts never appear in Pester code coverage reports or Codecov uploads.

Additionally, .github/workflows/pester-tests.yml discovers tests only under scripts/tests/ — skill tests co-located at .github/skills/<name>/tests/ would not be found by the current pipeline.

Changes required:

• Add .github/skills/ (or appropriate relative path) to the CodeCoverage.Path array in pester.config.ps1.
• Extend test discovery in pester-tests.yml to also search .github/skills/*/tests/*.Tests.ps1, or add a dedicated skill test job to the PR validation pipeline.
• Consider whether a test:skills npm script should be added to package.json for local developer use.

Gap 3 — PSScriptAnalyzer excludes skill scripts (High)

.github/workflows/ps-script-analyzer.yml and the linting scripts target scripts/**/*.ps1. PowerShell scripts at .github/skills/<name>/scripts/*.ps1 are outside that glob and never undergo static analysis.

Changes required:

• Expand the PSScriptAnalyzer target path to include .github/skills/**/*.ps1.
• Verify the existing PSScriptAnalyzer.psd1 ruleset applies without modification to skill scripts (it should — the rules are language-level, not path-dependent).
• Update npm run lint:ps if the underlying script uses hardcoded paths rather than the workflow glob.

Gap 4 — No Python CI pipeline (Medium)

Python standards are documented extensively in AI instruction templates (uv-projects.instructions.md, python-script.instructions.md, docs/contributing/instructions.md lines 440–493) but zero implementation exists:

• No pyproject.toml anywhere in the repository.
• No ruff.toml or .python-version file.
• No CI workflow runs ruff check, ruff format --check, or pytest.
• No uv.lock file.
• Python 3.11 is pre-installed in both devcontainer and Copilot agent environments.

The immediate risk is that a Python skill PR could merge with no automated quality gates. Issue #320 (add second skill) and the roadmap both anticipate Python-based skills.

Changes required:

• Document the skill-level pyproject.toml template in docs/contributing/skills.md covering ruff config (line-length=88, target-version="py311", select=["E","F","I","N","W","UP"]), pytest config, and project metadata.
• Defer creating the actual CI workflow and root pyproject.toml until the first Python skill PR, but ensure the contributing docs clearly describe what will be required.
• Add Python to the "Supported Languages" table (see Gap 7) with status "Documented — CI pending first skill PR."

Gap 7 — No documented process for new language requests (Medium)

Contributors have no way to know which languages the build system supports, which are rejected, or how to propose adding support for a new one. The skill request issue template has no field for specifying programming language or testing requirements.

Changes required:

• Add a "Supported Languages and Tooling" table to docs/contributing/skills.md:

• Add a "Requesting New Language Support" section describing the process: open a "Build System Change Request" issue → maintainer review → CI implementation before skill PRs using that language are accepted.
• Create a .github/ISSUE_TEMPLATE/build-system-request.yml issue template for build system change requests (language, tooling, CI integration, testing framework).
• Update .github/ISSUE_TEMPLATE/skill-request.yml to add a "Programming Language" dropdown field.
• Expand rejection criteria: "Skills using languages not listed in the supported languages table" and "Skills requiring CI tools not in the build system (must file a build system support issue first)."

Acceptance Criteria

• docs/contributing/skills.md includes a unit test requirement section with directory structure, naming convention, and rejection criterion for missing tests.
• scripts/tests/pester.config.ps1 CodeCoverage.Path array includes .github/skills/ (or equivalent).
• PSScriptAnalyzer CI job targets .github/skills/**/*.ps1 in addition to scripts/**/*.ps1.
• docs/contributing/skills.md includes a supported languages table and a pyproject.toml template for Python skills.
• docs/contributing/skills.md includes a "Requesting New Language Support" section with clear process steps.
• .github/ISSUE_TEMPLATE/build-system-request.yml exists with appropriate fields.
• .github/ISSUE_TEMPLATE/skill-request.yml includes a programming language dropdown.
• CONTRIBUTING.md testing section includes a skills-specific callout with link to docs/contributing/skills.md.
• Existing video-to-gif skill has at least one *.Tests.ps1 file demonstrating the required pattern (or a follow-up issue is filed).

Related

• PR feat(skills): edit SKILL frontmatter schema, add CI validation, and documentation #625 — Skill frontmatter schema validation and CI structural checks (in-flight, does not cover runtime testing)
• PR feat(agents): add Security Champion agent with SDL and OWASP guidance #408 — Security Champion skill (no test files)
• Issue feat(skills): Update skill frontmatter schema, prompt-builder guidance, CI validation, and documentation #623 — Skill frontmatter schema update tracking issue
• Issue feat: add second skill package to expand skills library #320 — Add second skill package
• Issue feat(extension): Update extension packaging to distribute skills #251 — Extension packaging for skills distribution (v2.7.0)

Files to Modify

RPI Starter Prompts

Research Phase

Research the CI coverage gap for skills in hve-core. Map how pester.config.ps1 coverage paths, pester-tests.yml test discovery, and ps-script-analyzer.yml globs exclude .github/skills/. Document the exact config lines that need updating. Review PR #625's Validate-SkillStructure.ps1 to understand what structural validation already covers and where runtime test validation should be added. Identify existing test patterns in scripts/tests/ that should serve as the model for skill tests.

Plan Phase

Plan the changes to mandate unit testing for skills and extend CI coverage to .github/skills/. The plan should cover: (1) docs/contributing/skills.md updates with unit test section, supported languages table, and language request process, (2) pester.config.ps1 and pester-tests.yml changes for coverage and discovery, (3) PSScriptAnalyzer path expansion, (4) new issue template for build system requests, (5) skill request template language field addition, (6) CONTRIBUTING.md skills callout. Include file paths, line numbers, and the proposed content for each change.

Implement Phase

Implement the skills unit testing mandate and CI coverage changes. Update docs/contributing/skills.md with unit test requirements (co-located tests/ directory, *.Tests.ps1 naming, rejection criterion), supported languages table, Python pyproject.toml template, and "Requesting New Language Support" section. Extend pester.config.ps1 coverage paths to include .github/skills/. Expand PSScriptAnalyzer glob in ps-script-analyzer.yml. Create .github/ISSUE_TEMPLATE/build-system-request.yml. Add programming language dropdown to skill-request.yml. Add skills testing callout to CONTRIBUTING.md. Run npm run lint:md and npm run lint:all to validate.

Review Phase

Review the skills unit testing mandate implementation. Verify: (1) docs/contributing/skills.md has a complete unit test section with directory structure, naming convention, and explicit rejection criterion, (2) the supported languages table covers PowerShell, Bash, and Python with accurate CI status, (3) pester.config.ps1 includes .github/skills/ in coverage paths, (4) PSScriptAnalyzer glob covers .github/skills/**/*.ps1, (5) build-system-request.yml template has appropriate fields for language, tooling, CI integration, and testing framework, (6) skill-request.yml has a programming language dropdown, (7) CONTRIBUTING.md links to skills testing docs, (8) npm run lint:md passes, (9) changes are consistent with PR #625's structural validation approach.