Skip to content

feat(scripts): collection-level maturity field: schema enforcement, validation, and experimental visibility #696

@katriendg

Description

@katriendg

Summary

Add full support for an optional maturity field at the collection level in collections/*.collection.yml manifests. When set to experimental, the collection is excluded from Stable/Release channel packaging and shows a visible experimental notice in generated READMEs. The infrastructure partially exists but has gaps in schema definition, validation, and user-facing visibility.

Motivation

  • experimental.collection.yml already uses maturity: experimental but the JSON schema does not include this field — and has additionalProperties: false, meaning the existing manifest technically violates the spec
  • github.collection.yml has 100% experimental items but no collection-level maturity, so it passes through to Stable channel with zero usable items
  • Generated plugin READMEs and extension READMEs show no indication that a collection is experimental — users have no visibility into maturity status
  • Validate-Collections.ps1 does not validate collection-level maturity values

What Already Works (No Changes Needed)

  • Test-CollectionMaturityEligible in Prepare-Extension.ps1 — fully implemented, gates experimental from Stable
  • extension-package.yml discover step — already filters by collection maturity and channel
  • extension-publish.yml and extension-publish-prerelease.yml — pass correct channel parameters
  • Item-level maturity filtering — fully working in both plugin generation and extension packaging

Remaining Work

Schema

  • Add optional maturity property (enum: stable, preview, experimental, deprecated) to root level in scripts/linting/schemas/collection-manifest.schema.json

Validation

  • Add collection-level maturity validation to Invoke-CollectionValidation in scripts/plugins/Validate-Collections.ps1 — reject invalid values, allow omitted (defaults to stable)

Plugin Generation — Experimental Notice

  • Update New-PluginReadmeContent in scripts/plugins/Modules/PluginHelpers.psm1 to accept collection maturity and render an experimental notice/badge when experimental
  • Update Write-PluginDirectory to pass collection maturity through to readme generation

Extension Packaging — Experimental Notice

  • Add {{MATURITY_NOTICE}} token to extension/templates/README.template.md
  • Update New-CollectionReadme in scripts/extension/Prepare-Extension.ps1 to resolve the token — experimental notice for experimental collections, empty for others

Data Consistency

  • Add maturity: experimental to collections/github.collection.yml at collection level (all items already marked experimental)

Tests (95% Line Coverage Target)

  • Validate-Collections.Tests.ps1 — valid/invalid collection-level maturity, omitted field
  • PluginHelpers.Tests.ps1 — experimental notice in plugin README, no notice for stable
  • Generate-Plugins.Tests.ps1 — deprecated collection skipped, experimental generated
  • Prepare-Extension.Tests.ps1 — maturity notice in extension README, no notice for stable

Documentation

  • Verify/update extension/PACKAGING.md collection manifest schema table
  • Verify/update docs/architecture/ai-artifacts.md collection maturity section
  • Verify/update docs/contributing/ai-artifacts-common.md maturity field requirements

Design Decisions

  • No changes to plugin.json — format defined by GitHub CLI plugin spec
  • No changes to package.json — format defined by VS Code extension manifest and validated by vsce
  • No workflow changes — all three publish/package workflows already handle collection-level maturity correctly
  • Maturity is a build-time concept — lives in collection YAML, surfaces in generated READMEs only

Experimental Notice Format

Both plugin and extension READMEs use the same notice:

> **⚠️ Experimental** — This collection is experimental and available only in the Pre-Release channel. Contents may change or be removed without notice.

Acceptance Criteria

  • Collections with maturity: experimental are excluded from Stable/Release packaging
  • Collections with maturity: experimental show visible notice in generated plugin and extension READMEs
  • JSON schema accurately reflects supported fields (fixes existing additionalProperties violation)
  • Invalid maturity values at collection level are caught by validation
  • Pester test coverage ≥ 95% line coverage
  • github.collection.yml tagged as experimental at collection level

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestpackagingExtension and plugin packaging

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions