Skip to content

feat(scripts): add Python skill support to Validate-SkillStructure#903

Merged
WilliamBerryiii merged 3 commits intomainfrom
feat/skill-validator-python-support
Mar 7, 2026
Merged

feat(scripts): add Python skill support to Validate-SkillStructure#903
WilliamBerryiii merged 3 commits intomainfrom
feat/skill-validator-python-support

Conversation

@WilliamBerryiii
Copy link
Copy Markdown
Member

Description

Extended Validate-SkillStructure.ps1 to recognize Python-based skills and apply language-aware validation rules. The validator now detects Python skills by checking for pyproject.toml at the skill root and branches validation logic accordingly: Python skills accept .py-only scripts or the traditional .ps1+.sh pair in the scripts/ subdirectory, while non-Python skills retain the original behavior unchanged. Subdirectories containing __init__.py are treated as valid Python package directories and no longer trigger unrecognized-subdirectory warnings.

Six new Pester test cases were added across three contexts to cover Python script acceptance, package directory handling, and a regression guard confirming non-Python skills remain unaffected. All 67 tests pass (61 existing + 6 new).

Skill Validation Changes

The core change introduces a language-aware validation model: detect once at skill root, then branch validation rules per language throughout the function.

  • Added Python skill detection in Test-SkillDirectory via Test-Path for pyproject.toml at the skill root, storing the result in $isPythonSkill for reuse across validation sections
  • Rewrote scripts/ subdirectory validation to branch on $isPythonSkill: Python skills accept .py-only scripts OR the traditional .ps1+.sh pair; non-Python skills retain the original three error messages verbatim
  • Added Python package directory exception: subdirectories containing __init__.py bypass the "unrecognized subdirectory" warning in Python skills only
  • Refactored script file enumeration from pre-filtered Where-Object to Get-ChildItem with per-extension boolean checks, functionally equivalent for non-Python skills

Test Coverage

  • Extended New-TestSkillDirectory helper with three new parameters: -WithPyprojectToml, -WithPythonScripts, and -WithPythonPackageDir
  • Added Describe 'Python Skill Validation' block with six test cases: .py-only pass, hybrid pass, no-scripts pass, package directory suppresses warning, missing __init__.py still warns, and non-Python regression guard
  • Test isolation maintained via dedicated temp subdirectory managed by BeforeAll/AfterAll

Related Issue(s)

Fixes #892

Type of Change

Select all that apply:

Code & Documentation:

  • Bug fix (non-breaking change fixing an issue)
  • New feature (non-breaking change adding functionality)
  • Breaking change (fix or feature causing existing functionality to change)
  • Documentation update (N/A — no documentation changes)

Infrastructure & Configuration:

  • GitHub Actions workflow
  • Linting configuration (markdown, PowerShell, etc.)
  • Security configuration
  • DevContainer configuration
  • Dependency update

AI Artifacts:

  • Reviewed contribution with prompt-builder agent and addressed all feedback
  • Copilot instructions (.github/instructions/*.instructions.md)
  • Copilot prompt (.github/prompts/*.prompt.md)
  • Copilot agent (.github/agents/*.agent.md)
  • Copilot skill (.github/skills/*/SKILL.md)

Note for AI Artifact Contributors:

  • Agents: Research, indexing/referencing other project (using standard VS Code GitHub Copilot/MCP tools), planning, and general implementation agents likely already exist. Review .github/agents/ before creating new ones.
  • Skills: Must include both bash and PowerShell scripts. See Skills.
  • Model Versions: Only contributions targeting the latest Anthropic and OpenAI models will be accepted. Older model versions (e.g., GPT-3.5, Claude 3) will be rejected.
  • See Agents Not Accepted and Model Version Requirements.

Other:

  • Script/automation (.ps1, .sh, .py)
  • Other (please describe):

Testing

Automated validation performed by the agent:

Check Command Result
Markdown linting npm run lint:md ✅ 0 errors
Spell checking npm run spell-check ✅ 0 issues
Frontmatter validation npm run lint:frontmatter ✅ 0 errors, 0 warnings
Skill structure validation npm run validate:skills ✅ 2 skills, 0 errors
Link validation npm run lint:md-links ✅ All links valid
PowerShell analysis npm run lint:ps ✅ All files passed
Plugin freshness npm run plugin:generate ✅ No changes produced

Diff-based assessment:

  • Backward compatibility confirmed: non-Python validation path is structurally identical to the original code; a dedicated regression guard test validates this explicitly
  • No public API changes: Test-SkillDirectory parameters are unchanged; Python detection is internal via filesystem inspection
  • No security concerns: detection uses Test-Path (existence check only) with no file content parsing
  • 67/67 Pester tests passing (61 existing + 6 new)

Manual testing was not performed.

Checklist

Required Checks

  • Documentation is updated (N/A — no documentation changes)
  • Files follow existing naming conventions
  • Changes are backwards compatible
  • Tests added for new functionality

Required Automated Checks

The following validation commands must pass before merging:

  • Markdown linting: npm run lint:md
  • Spell checking: npm run spell-check
  • Frontmatter validation: npm run lint:frontmatter
  • Skill structure validation: npm run validate:skills
  • Link validation: npm run lint:md-links
  • PowerShell analysis: npm run lint:ps
  • Plugin freshness: npm run plugin:generate

Security Considerations

  • This PR does not contain any sensitive or NDA information
  • Any new dependencies have been reviewed for security issues (N/A — no dependency changes)
  • Security-related scripts follow the principle of least privilege (N/A — no security scripts modified)

Additional Notes

  • Python detection relies solely on pyproject.toml presence via Test-Path; no TOML parsing or content inspection is performed
  • The script file enumeration refactor (from Where-Object to per-extension booleans) is functionally equivalent for non-Python skills but enables the Python branch to inspect additional extensions

Bill Berry added 2 commits March 5, 2026 11:04
- detect pyproject.toml at skill root to identify Python skills
- accept .py-only scripts/ directories for Python skills
- allow Python package directories with __init__.py as valid subdirectories
- add 6 Pester test cases for Python skill validation scenarios
- preserve existing non-Python validation behavior

🐍 - Generated by Copilot
@WilliamBerryiii WilliamBerryiii requested a review from a team as a code owner March 5, 2026 23:01
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 5, 2026

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Mar 6, 2026

Codecov Report

❌ Patch coverage is 94.11765% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 85.95%. Comparing base (8aca446) to head (d92980e).

Files with missing lines Patch % Lines
scripts/linting/Validate-SkillStructure.ps1 94.11% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #903      +/-   ##
==========================================
- Coverage   85.95%   85.95%   -0.01%     
==========================================
  Files          27       27              
  Lines        5178     5185       +7     
==========================================
+ Hits         4451     4457       +6     
- Misses        727      728       +1     
Flag Coverage Δ
pester 85.95% <94.11%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
scripts/linting/Validate-SkillStructure.ps1 95.56% <94.11%> (-0.36%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Contributor

@katriendg katriendg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, though one thing: do we need to add an update to the skills.md doc around the options, it now says .ps1 is a requirement, though with the change it is no longer required when Python is available.

@WilliamBerryiii
Copy link
Copy Markdown
Member Author

Looks good to me, though one thing: do we need to add an update to the skills.md doc around the options, it now says .ps1 is a requirement, though with the change it is no longer required when Python is available.

The general gist is that for shell scripting, PowerShell is required and Bash is "optional." There's a docs change coming in a few PRs from now that will detail this update.

WilliamBerryiii added a commit that referenced this pull request Mar 20, 2026
🤖 I have created a release *beep* *boop*
---


##
[3.2.0](hve-core-v3.1.46...hve-core-v3.2.0)
(2026-03-20)


### ✨ Features

* add -OutputPath parameter to Validate-MarkdownFrontmatter.ps1
([#1134](#1134))
([fdf1bcf](fdf1bcf)),
closes [#1006](#1006)
* add action version consistency scan workflow
([#1127](#1127))
([4229df1](4229df1))
* **agent:** MVE Experiment Designer
([#976](#976))
([70f86ca](70f86ca))
* **agents:** add ADO Backlog Manager orchestrator agent
([#800](#800))
([fae3987](fae3987))
* **agents:** add meeting analyst agent for transcript analysis using
work-iq ([#502](#502))
([5345b5b](5345b5b))
* **agents:** add quick-reference line to RPI Phase 5 suggestions
([#897](#897))
([9a90f39](9a90f39))
* **agents:** add RAI Planner, enhance SSSC Planner, and redesign
Security Planner
([#979](#979))
([06f826c](06f826c))
* **agents:** add symmetric cross-system handoff to GitHub Backlog
Manager ([#952](#952))
([ba34a35](ba34a35))
* **agents:** Functional Code Review Agent — pre-PR functional
correctness reviewer
([#733](#733))
([9cf63b7](9cf63b7))
* **build:** add Python extensions and uv 0.10.8 to devcontainer
([#920](#920))
([9ca0579](9ca0579))
* **build:** add uv ecosystem to Dependabot configuration
([#913](#913))
([2a4bd39](2a4bd39))
* **build:** enable npm pinning enforcement in dependency scan
([#838](#838))
([4e9e31f](4e9e31f))
* **build:** migrate attestation actions to v4.1.0 and add SBOM
verification docs
([#841](#841))
([ca1e65b](ca1e65b))
* **collections:** add four new validator checks (orphan, duplicate,
companion, coverage)
([#869](#869))
([1a96b73](1a96b73))
* **devcontainer,security:** add enterprise artifact hub configuration
([#1032](#1032))
([1d56d25](1d56d25))
* **docs:** add Rust coding standards and guidelines
([#809](#809))
([d4c4899](d4c4899))
* **extension:** add Microsoft logo icon to VS Code Marketplace listings
([#906](#906))
([82aca41](82aca41))
* **github:** add declarative label management
([#953](#953))
([a1a6845](a1a6845))
* **instructions:** add ADO backlog shared infrastructure
([#786](#786))
([1914078](1914078))
* **instructions:** add ADO backlog sprint planning and capacity
tracking ([#788](#788))
([d6fb77d](d6fb77d))
* **instructions:** add ADO triage workflow and prompt
([#787](#787))
([cde0190](cde0190))
* **instructions:** add shared story quality conventions and sprint
planning ([#803](#803))
([a2f18e3](a2f18e3))
* **prompts:** add ADO discovery and work item prompts with agent
routing ([#790](#790))
([7e74523](7e74523))
* **prompts:** add security review prompts
([#1118](#1118))
([ad30967](ad30967))
* **scripts:** add dynamic Python skill discovery for lint/test
([#957](#957))
([0a90f57](0a90f57))
* **scripts:** add Get-StandardTimestamp utility to CIHelpers module
([#1126](#1126))
([b273a4b](b273a4b))
* **scripts:** add Python copyright header validation
([#905](#905))
([67df902](67df902))
* **scripts:** add Python skill support to Validate-SkillStructure
([#903](#903))
([68479d9](68479d9))
* **scripts:** add workflow npm command scanning to dependency pinning
([#837](#837))
([6b5ae06](6b5ae06))
* **security:** add basic security reviewer agent with owasp skills
([#1008](#1008))
([cb1fd05](cb1fd05))
* **security:** add sigstore attestation bundles and fix
component-detection action
([#1148](#1148))
([f79c272](f79c272))
* **skills:** add Atheris fuzz harness with CI workflow integration
([#1102](#1102))
([d337e1d](d337e1d))
* **skills:** add PowerPoint automation skill with YAML-driven deck
generation ([#868](#868))
([00465cd](00465cd))
* **skills:** convert hve-core-installer agent to self-contained skill
([#846](#846))
([1d821fb](1d821fb))
* **skills:** enhance pr-reference skill with flexible filtering and
base branch detection
([#1095](#1095))
([26a32ea](26a32ea))
* **workflows:** add devcontainer infrastructure change log workflow
([#899](#899))
([8aca446](8aca446))
* **workflows:** add milestone auto-close on stable and pre-release
publishes ([#834](#834))
([79362b1](79362b1))
* **workflows:** add ms.date documentation freshness checking
([#969](#969))
([3ed441c](3ed441c))
* **workflows:** add Python linting CI workflow with Ruff
([#951](#951))
([f89f0eb](f89f0eb))
* **workflows:** add Python testing CI workflow with pytest and Codecov
([#934](#934))
([5e8306f](5e8306f))
* **workflows:** add uv and Python package sync to copilot-setup-steps
([#921](#921))
([45d517d](45d517d))


### 🐛 Bug Fixes

* **build:** override Linguist vendored flag for Python skill files
([#1155](#1155))
([0eee5b6](0eee5b6))
* **build:** override serialize-javascript to &gt;=7.0.3 for RCE fix
([#876](#876))
([e49039a](e49039a))
* **build:** resolve Pinned-Dependencies alerts for vsce npm commands in
extension workflows
([#782](#782))
([89dad9d](89dad9d))
* **build:** update undici and yauzl overrides for security audit
([#1030](#1030))
([2c2f92f](2c2f92f))
* **docs:** add CLI Plugins to install.md navigation surfaces
([#902](#902))
([79d6595](79d6595))
* **docs:** add sidebar ordering for Design Thinking documentation
([#832](#832))
([551fddc](551fddc)),
closes [#830](#830)
* **docs:** graduate design-thinking to preview and correct stale
collection references
([#831](#831))
([5110e35](5110e35))
* **docs:** include project-planning in UX Designer install guidance
([#908](#908))
([e7aa9bc](e7aa9bc))
* **docs:** remediate writing-style convention violations
([#865](#865))
([68b04bc](68b04bc))
* **docs:** remove draft content announcement banner
([#825](#825))
([b45de80](b45de80))
* **docs:** remove unbounded path-to-regexp override breaking SSG
([#1153](#1153))
([d810018](d810018))
* **docs:** use actual clone paths instead of folder display names in
multi-root workspace settings
([#984](#984))
([5dbab82](5dbab82))
* **instructions:** replace black with ruff in uv-projects
([#898](#898))
([b0c06d9](b0c06d9))
* **scripts:** cover .github/ skill files in copyright header validation
([#1055](#1055))
([#1098](#1098))
([27fbd33](27fbd33))
* **scripts:** eliminate phantom git changes from plugin generation
([#1035](#1035))
([e49a1b5](e49a1b5))
* **scripts:** enable JSON log output for lint:version-consistency
([#1033](#1033))
([52b0885](52b0885))
* **security:** calculate compliance score from total scanned
dependencies ([#930](#930))
([c112c3d](c112c3d))
* **skills:** add AST validation and namespace restriction for
content-extra.py
([#1027](#1027))
([c50c7a3](c50c7a3))
* **skills:** add depth limits to recursive PowerPoint processing
functions ([#1028](#1028))
([bf08994](bf08994))
* **skills:** harden XML parsing and blob writes in powerpoint extract
([#1053](#1053))
([89d24b1](89d24b1))
* **skills:** resolve ruff lint and format violations in powerpoint
skill ([#1048](#1048))
([17bbe7a](17bbe7a))
* **workflows:** add uv.lock dependencies submission have fork-skip
condition ([#1109](#1109))
([dec56ac](dec56ac))
* **workflows:** automate weekly SHA staleness check with issue creation
([#975](#975))
([1ea4caa](1ea4caa))
* **workflows:** close Codecov integration gaps for Pester and pytest
flags ([#1106](#1106))
([cca29b7](cca29b7))
* **workflows:** propagate uv sync errors in copilot-setup-steps
([#961](#961))
([df88d7c](df88d7c))
* **workflows:** resolve release-please skip cascade and Python project
discovery ([#1043](#1043))
([79993e2](79993e2))
* **workflows:** scan only commit subjects for breaking change detection
([#1157](#1157))
([a38a657](a38a657))


### 📚 Documentation

* clarify HVE Core Extension vs Installer messaging across documentation
([#965](#965))
([0fceb8f](0fceb8f))
* **docs:** add ADO integration user documentation
([#935](#935))
([ec89302](ec89302))
* **docs:** add Project Planning agent documentation
([#936](#936))
([3a3a0fd](3a3a0fd))
* **onboarding:** overhaul marketplace onboarding and documentation site
([#982](#982))
([4309e10](4309e10))


### ♻️ Refactoring

* **build:** merge code-review collection into coding-standards
([#863](#863))
([8027e7b](8027e7b))
* **workflows:** rename release pipeline workflows and add marketplace
automation triggers
([#829](#829))
([b6397f4](b6397f4))


### 🔧 Maintenance

* **build:** add clean:logs npm script
([#1122](#1122))
([f85fe02](f85fe02)),
closes [#988](#988)
* **build:** add JSON reporter for cspell
([#1123](#1123))
([6d59f67](6d59f67))
* **ci:** add multi-arch support to copilot-setup-steps binary downloads
([#955](#955))
([8d0c706](8d0c706))
* **deps-dev:** bump cspell from 9.6.4 to 9.7.0 in the npm-dependencies
group ([#839](#839))
([3fa16ff](3fa16ff))
* **deps:** bump actions/dependency-review-action from 4.8.3 to 4.9.0 in
the github-actions group across 1 directory
([#942](#942))
([1a9b858](1a9b858))
* **deps:** bump cairosvg from 2.8.2 to 2.9.0 in
/.github/skills/experimental/powerpoint
([#1025](#1025))
([f4deda7](f4deda7))
* **deps:** bump dompurify from 3.3.1 to 3.3.2 in /docs/docusaurus
([#924](#924))
([d2060d6](d2060d6))
* **deps:** bump svgo from 3.3.2 to 3.3.3 in /docs/docusaurus
([#880](#880))
([6dc2406](6dc2406))
* **deps:** bump the github-actions group across 1 directory with 4
updates ([#1100](#1100))
([2290dc0](2290dc0))
* **deps:** bump the github-actions group with 6 updates
([#840](#840))
([f57bc01](f57bc01))
* **docs:** correct New-MsDateReport table rendering and refresh stale
docs ([#1114](#1114))
([c2b806f](c2b806f))
* **settings:** remove orphaned Checkov config and stale gitignore
entries ([#870](#870))
([98fcd74](98fcd74))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

---------

Co-authored-by: hve-core-release-please[bot] <254602402+hve-core-release-please[bot]@users.noreply.github.com>
Co-authored-by: Bill Berry <wberry@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(scripts): Update skill validator to support Python skill structure

3 participants