-
Notifications
You must be signed in to change notification settings - Fork 125
Description
Parent: #431
Issue Description
Wire the existing Validate-ArtifactRegistry.ps1 into the PR validation workflow so that any pull request adding, removing, or renaming an AI artifact file without updating the registry manifest is caught before merge.
Motivation
The AI Artifacts Registry (ai-artifacts-registry.json) is the single source of truth for persona tagging, maturity levels, and dependency metadata. If a contributor adds a new prompt, deletes an agent, or renames an instruction file without updating the registry, the result is silent drift: the packaging scripts, installer, and collection manifests all operate on stale data.
The Validate-ArtifactRegistry.ps1 script already performs dual-direction validation:
- Registry → Disk: Each registry entry maps to a file on disk. A removed or renamed file causes a "file not found" error.
- Disk → Registry: Each artifact file on disk should appear in the registry. A new or renamed file without a registry entry is reported as an orphan warning.
The script also excludes instructions/hve-core/* from orphan detection, since those are repo-specific and not distributed.
The missing piece is wiring this validator into the PR gate with -WarningsAsErrors so orphan warnings become hard failures.
Current State
| Component | Status |
|---|---|
Validate-ArtifactRegistry.ps1 |
Exists, full validation logic including orphan detection |
npm run lint:registry |
Exists, runs the validator |
| Pester test suite | Exists, 17 fixtures covering structure, persona refs, entries, dependencies, orphans, CI annotations, exit codes, renamed-file mismatch |
| Reusable workflow | ✅ Implemented |
| PR validation job | ✅ Implemented |
| Test fixtures for renamed-file mismatch | ✅ Implemented |
Deliverables
1. Reusable Workflow
Create .github/workflows/artifact-registry-validation.yml as a reusable workflow_call workflow following the same patterns as frontmatter-validation.yml:
- Inputs:
soft-fail(boolean, default false),warnings-as-errors(boolean, default true) - Steps: checkout, create logs directory, run
Validate-ArtifactRegistry.ps1, upload results artifact, fail-on-error gate - Permissions:
contents: read
2. PR Validation Integration
Add an artifact-registry-validation job to .github/workflows/pr-validation.yml:
artifact-registry-validation:
name: Artifact Registry Validation
uses: ./.github/workflows/artifact-registry-validation.yml
permissions:
contents: read
with:
soft-fail: false
warnings-as-errors: trueThis catches three drift scenarios on every PR:
| Scenario | Detection Mechanism | Existing Logic |
|---|---|---|
| File added, registry not updated | Orphan detection (Find-OrphanArtifacts) |
Yes, reports as warning; -WarningsAsErrors promotes to error |
| File removed, registry not updated | File existence check (Test-ArtifactFileExistence) |
Yes, reports as error |
| File renamed, registry not updated | Both: old key fails existence, new file is orphan | Yes, combination of above two checks |
3. Test Fixtures for Renamed-File Mismatch
Add a fixture that simulates the renamed-file scenario (old registry key pointing to a non-existent file while a new file exists on disk but is unregistered). This validates the dual-direction detection works in concert.
Add Pester test cases to Validate-ArtifactRegistry.Tests.ps1:
- File added but not in registry (orphan detected, promoted to error with
-WarningsAsErrors) - File removed but still in registry (existence error)
- File renamed: old key errors on existence AND new file detected as orphan
- Files under
instructions/hve-core/remain excluded from orphan detection
Acceptance Criteria
- Reusable workflow
artifact-registry-validation.ymlexists and can be called independently - PR validation workflow includes the registry validation job
- A PR adding a new
.agent.md,.prompt.md,.instructions.md, orSKILL.mdwithout a registry entry fails validation - A PR removing an artifact file without removing its registry entry fails validation
- A PR renaming an artifact file without updating the registry entry fails validation (both old-key error and new-file orphan)
- Files under
.github/instructions/hve-core/are excluded from orphan detection (existing behavior preserved) - Pester tests cover the renamed-file mismatch scenario with fixtures
- Validation results are uploaded as a CI artifact for debugging
Technical Notes
- No changes to
Validate-ArtifactRegistry.ps1are needed. The existing-WarningsAsErrorsflag already promotes orphan warnings to errors. - The reusable workflow follows the
frontmatter-validation.ymlpattern: checkout → logs directory → run script → upload artifact → fail gate. - The
pester-testsjob already runs in PR validation and will pick up new test files automatically via its test discovery pattern. - Code coverage for the registry validator is already tracked by the existing Pester/Codecov pipeline.
Dependencies
- Phase 1 (feat: AI Artifacts Registry Design and Schema #432): AI Artifacts Registry Design and Schema (implemented)
- No blockers from other phases; this is independently deliverable
Additional Context
- Validator script:
scripts/linting/Validate-ArtifactRegistry.ps1 - Existing tests:
scripts/tests/linting/Validate-ArtifactRegistry.Tests.ps1 - Test fixtures:
scripts/tests/Fixtures/ArtifactRegistry/ - PR validation workflow:
.github/workflows/pr-validation.yml - Pattern reference:
.github/workflows/frontmatter-validation.yml - Registry file:
.github/ai-artifacts-registry.json