Implement plugin management system and CLI commands#83
Implement plugin management system and CLI commands#83danielmeppiel merged 16 commits intomicrosoft:mainfrom
Conversation
9af6e95 to
4764a8d
Compare
There was a problem hiding this comment.
Pull request overview
Adds a first-cut plugin marketplace + resolver layer and exposes it via new apm plugin CLI commands, with docs and fixtures to support plugin installation/integration workflows.
Changes:
- Introduces
MarketplaceManager/PluginResolverfor resolvingplugin-id@marketplaceinto a repository URL. - Adds
apm plugin install(tracks plugins inapm.yml) andapm plugin list(marketplace browsing) commands. - Extends
apm installto also install entries fromapm.yml: plugins, and adds docs + test fixtures around plugin primitives.
Reviewed changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 30 comments.
Show a summary per file
| File | Description |
|---|---|
src/apm_cli/plugin/marketplace.py |
Adds marketplace fetching/parsing and plugin listing/lookup. |
src/apm_cli/plugin/resolver.py |
Adds parsing/resolution for plugin-id@marketplace specs. |
src/apm_cli/commands/plugin.py |
Adds Click CLI group/commands for plugin install/list and apm.yml tracking. |
src/apm_cli/cli.py |
Loads plugins: from apm.yml and installs them alongside APM deps. |
src/apm_cli/models/apm_package.py |
Adds a placeholder get_plugins() method. |
tests/unit/test_plugin_system.py |
Unit tests for marketplace parsing and resolver behavior. |
tests/integration/test_plugin_integration.py |
Integration-style tests around primitive integration using a mock plugin fixture. |
tests/fixtures/mock-plugin/** |
Adds a mock plugin repo layout with .apm/* primitives for tests. |
docs/plugins.md |
New plugin system guide. |
docs/index.md |
Adds link to plugin guide. |
docs/cli-reference.md |
Documents new apm plugin commands. |
9b10d45 to
a4f6f1c
Compare
61e8ebb to
fb35cd6
Compare
|
@SebastienDegodez — thank you for this. The plugin marketplace ecosystem is an industry standard now and APM needs to support it. This review is about routing it through APM's existing architecture, not building alongside it. Decision: Request changesAPM's detection pattern (how it works today)APM already handles five foreign formats, all the same way: No flags. No separate commands. What plugin support should look likeThe same pattern, extended: When the downloader finds
The user experience is just: apm install anthropics/claude-code-plugins/commit-commandsOr in dependencies:
apm:
- company/coding-standards#v2.0
- anthropics/claude-code-plugins/commit-commands#v1.2.0That's it. No The marketplace discovery questionThe useful part of your PR — But that's a browsing concern, not an install concern. Once a user knows the repo, it's just If we want APM to help with discovery later, it can be a lightweight convenience: apm browse claude # opens marketplace URL in browserBut that's a separate, smaller feature — not a prerequisite for plugin support. What to keep from your PR
Implementation planPhase 1 —
Result: Phase 2 — Marketplace source resolution (optional, later) Your Why this matters architecturallyAPM currently has one install pipeline, one manifest format, one discovery system, one lock file. Every foreign format normalizes into that single model. This is the project's most important architectural invariant — it's what makes APM "the npm for agent primitives" rather than "a wrapper around five different tools." PR #83 as written breaks that invariant by creating a parallel pipeline. The plugin ecosystem deserves full support — but through the existing architecture, not alongside it. Your |
|
Hello @danielmeppiel , sorry I'm in holidays. Can you take this Pull Request or you can wait one moment ? |
b2dfc1f to
0386ca5
Compare
Signed-off-by: SebastienDegodez <sebastien.degodez@gmail.com>
0386ca5 to
8019018
Compare
|
@danielmeppiel Phase 1: Done |
|
test |
|
Hey @SebastienDegodez, thanks for pushing this forward — the normalization approach (plugin → I tested against real-world plugin repos and cross-referenced with the official spec (plugin reference, marketplace spec), and the implementation needs to be re-grounded. The key insight from the spec:
A Claude plugin has no mandatory files. This means the normalization should not gate on finding What "normalize as Claude plugin" means:
Specific issues in the current implementation:
Acceptance bar:
You're on the right track with the normalization architecture. The main shift is treating plugin normalization as the fallback path (not a detected path), and aligning the parser with the spec's permissiveness. Happy to pair on any of this. |
danielmeppiel
left a comment
There was a problem hiding this comment.
As per last comment.
This should also integrate hooks coming from a plugin in the right folders given #97's imminent merge
|
Should consider testing apm driven plugin installation in a way that the plugins installed with APM are immediately picked up by VSCode / GitHub Copilot - https://github.com/microsoft/vscode-docs/blob/vnext/docs/copilot/customization/agent-plugins.md |
- MARKETPLACE_PLUGIN is now the fallback for any directory without
apm.yml/SKILL.md, per spec: 'The manifest is optional.'
- plugin.json is optional metadata; only name is required (defaults
to directory name when absent)
- version defaults to short commit SHA when absent from plugin.json
- author accepted as object {name, email, url} or plain string
- .mcp.json, .lsp.json, settings.json, hooks/ copied into .apm/
so MCP-based plugins function correctly after install
- Added normalize_plugin_directory() — works with or without plugin.json
- Removed scan_plugin_primitives() — plugins are normalized at install
time so scan_dependency_primitives() handles them naturally
- README.md used as last-resort fallback in validate_virtual_package_exists()
for plugin directories with no manifest files
- Test fixtures: author updated to object format per spec
- New tests: plugin without plugin.json, .mcp.json passthrough
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
danielmeppiel
left a comment
There was a problem hiding this comment.
PR #83 — Plugin Support: Review Verdict
Architecture: ✅ Approved
The decompose-and-normalize approach is the correct architectural decision for APM. After normalization, plugins become standard APM packages — no new command surface, no special cases in downstream commands. This preserves APM's core value: composition and governance across agent platforms.
The normalization pipeline (find plugin.json → parse manifest → map artifacts to .apm/ → synthesize apm.yml → scatter via existing integrators) is well-designed and makes all APM commands (install, uninstall, deps, prune, compile) work with plugins implicitly.
Command Integration: ✅ Complete
Every APM command works with plugins through the existing infrastructure:
- install — explicit MARKETPLACE_PLUGIN handling with version stamping
- uninstall/prune — type-agnostic via
deployed_filesin lockfile - deps (list/tree/info/update/clean) — works via synthetic
.apm/+apm.yml - compile — discovers plugin primitives uniformly via
apm_modules/scan
Code Quality: ⚠️ 6 Must-Fixes
| # | Issue | Severity |
|---|---|---|
| 1 | find_plugin_json uses recursive glob **/plugin.json instead of deterministic 3-location check |
HIGH — non-deterministic, security risk |
| 2 | README.md fallback in remote validation is over-permissive | HIGH — any repo with README becomes installable |
| 3 | Orphaned code after test_mcp_json_copied_through — assertions unreachable |
HIGH — CI reports wrong test count |
| 4 | shutil.copytree follows symlinks by default |
MEDIUM — security: malicious plugin symlink attacks |
| 5 | README.md has zero mention of plugin support | MEDIUM — feature invisible to users |
| 6 | CHANGELOG.md entry missing | MEDIUM — OSS standard |
Test Coverage: 6/10
13 integration tests covering happy paths and all 3 plugin.json locations. However: zero unit tests for plugin_parser.py functions, broken test structure (orphaned code), and no lifecycle tests (install → uninstall → verify cleanup). The three spec-defined locations and priority order are well-tested.
Documentation: 5/10
docs/plugins.md (282 lines) is comprehensive and well-written. But the feature is invisible from main entry points: no README mention, no CHANGELOG entry, no manifest-schema update, no dependencies.md update.
Verdict: Request Changes
The architecture is sound. The implementation gets the hard part right (normalization pipeline, all commands working implicitly, lockfile tracking). But 6 items need fixing before merge — the recursive glob in find_plugin_json and the symlink-following copytree are security concerns, and the broken test/missing docs are quality gates.
Once must-fixes are addressed, this is a strong merge. Full analysis: WIP/pr83-spec-aligned-analysis.md (v4).
There was a problem hiding this comment.
Consolidated Issue List — E2E Test + Code Review + Spec Analysis
After running a full end-to-end acceptance test with a real plugin from github/awesome-copilot (the context-engineering plugin at plugins/context-engineering, which has plugin.json at .github/plugin/plugin.json, agents/, and skills/), here is every issue found — consolidated from code review, spec analysis, and real-world testing.
Must-Fix (Blocks Merge)
| # | Issue | Explanation |
|---|---|---|
| 1 | find_plugin_json() uses recursive glob instead of deterministic 3-location check |
The Copilot CLI spec defines exactly three valid locations for plugin.json: root, .github/plugin/, .claude-plugin/. The current **/plugin.json glob traverses unintended directories, is non-deterministic when multiple match, and is a security risk (could find a plugin.json deep in node_modules/). Replace with explicit path checks for the three spec-defined locations. |
| 2 | shutil.copytree() follows symlinks |
All shutil.copytree() calls in _map_plugin_artifacts() lack symlinks=False. A malicious plugin can include a symlink pointing to /etc/passwd or ~/.ssh/ and the copy will follow it, exfiltrating content into the project's .apm/ directory. |
| 3 | False orphans in deps list |
When skills are scattered to .github/skills/*/SKILL.md, the list_packages() function's rglob() scan detects these as independent orphaned packages. E2E test showed 3 false "orphaned" plugins alongside the real package. This is confusing and incorrect — scattered files are deployment artifacts, not packages. |
| 4 | deps info fails for virtual sub-path packages |
Running apm deps info github/awesome-copilot/plugins/context-engineering fails — it can't resolve the virtual sub-path and falls back to repo-level github/awesome-copilot, showing "No apm.yml found". Virtual path resolution needs to match the installed package identity. |
| 5 | Remote validation README.md fallback over-permissive | Any git repo containing a README.md can be installed as MARKETPLACE_PLUGIN. The validation should align with the Copilot CLI plugin reference — require plugin.json in a valid location OR at least one spec-defined component directory (agents/, skills/, commands/, hooks/). |
| 6 | Empty directory auto-classified as MARKETPLACE_PLUGIN | The PackageType fallback chain classifies any directory without apm.yml/SKILL.md/hooks.json as MARKETPLACE_PLUGIN. An empty directory should not be a valid plugin. Same fix as #5 — require either plugin.json or component directories. |
| 7 | Broken test structure | test_mcp_json_copied_through in test_marketplace_plugin_integration.py has orphaned code after it — a bare docstring that was likely meant to be another test method. Assertions are unreachable, and CI reports 11 tests when there should be 12. |
| 8 | No unit tests for plugin_parser.py |
All 6 public functions (parse_plugin_manifest, normalize_plugin_directory, synthesize_apm_yml_from_plugin, _map_plugin_artifacts, _generate_apm_yml, validate_plugin_package) are tested only via integration tests. find_plugin_json() has zero dedicated tests. Unit tests needed for: parser edge cases, artifact mapping, apm.yml generation, and the 3-location precedence logic. |
| 9 | Agents not scattered | E2E test confirmed: agents in .apm/agents/ were NOT deployed to .github/agents/. Only skills were scattered. The AgentIntegrator either isn't finding agents in the normalized .apm/agents/ directory, or the agent file format doesn't match what it expects (e.g., requires .agent.md extension). |
| 10 | No README.md mention of plugin support | The main README.md has zero mention of plugin support. Users discovering APM can't find the feature. Add a section or at minimum a link to docs/plugins.md. |
| 11 | No CHANGELOG.md entry | The Unreleased section is empty. Plugin support is the biggest feature in this PR — it needs a changelog entry. OSS standard. |
Should-Fix (in this PR as well, lower blockers)
| # | Issue | Explanation |
|---|---|---|
| 12 | Hardcoded type: hybrid in _generate_apm_yml() |
All synthesized apm.yml files get type: hybrid regardless of actual plugin content. A plugin with only skills should be type: skills, one with only agents should be type: agents. Should dynamically determine type from what component dirs exist. |
| 13 | No name field validation in parse_plugin_manifest() |
Plugin spec says name is the only required field in plugin.json. But parse_plugin_manifest() accepts manifests without it. validate_plugin_package() checks for name at the validation level, but the parser itself should warn or error. |
| 14 | Fragile .prompt.md rename logic |
In _map_plugin_artifacts(), commands/ files get .md → .prompt.md renaming, but there's no guard against files already named .prompt.md getting double-suffixed. The if source_file.name.endswith(".prompt.md"): pass check exists but the logic flow is fragile — consider early-continue. |
| 15 | Install counter says "Installed 0" | After successfully installing a plugin with scattered primitives, the success message shows "Installed 0 APM dependencies." Plugin installs should be counted. |
| 16 | Empty directories left after cleanup | Both apm uninstall and apm prune leave behind empty .github/skills/ and apm_modules/ directories after removing all contents. Minor but untidy. |
| 17 | No type field in lockfile |
LockedDependency doesn't store the package type (MARKETPLACE_PLUGIN, APM_PACKAGE, etc.). Package provenance is lost — you can't tell from apm.lock alone whether a dependency was originally a plugin or a native APM package. |
| 18 | .mcp.json/.lsp.json not deployed to runtime locations |
These files are correctly copied to .apm/ during normalization but never deployed to where runtimes look for them (.vscode/mcp.json, .github/mcp.json). MCP-based plugins won't function after install. |
| 19 | docs/manifest-schema.md missing MARKETPLACE_PLUGIN type |
Schema reference doesn't list the new type. |
| 20 | docs/dependencies.md missing MARKETPLACE_PLUGIN |
Dependency types table doesn't include plugins. |
| 21 | docs/primitives.md missing MARKETPLACE_PLUGIN |
Package type documentation incomplete. |
| 22 | Inline mcpServers/lspServers/hooks in plugin.json |
Spec allows inline definitions (e.g., "hooks": {"onFileChange": [...]}) or path-based ("hooks": "./hooks"). PR only handles path-based. |
| 23 | Custom component path arrays | Spec allows "skills": ["skills/", "extra/"]" with multiple directories. PR only handles single string paths. |
What Works Well
For the record — the following passed the E2E test cleanly:
- Install flow: download → sparse checkout → plugin.json detection → normalization → skill scattering
- Lockfile:
deployed_files,is_virtual,virtual_path,resolved_commitall present and correct - Hooks: Plugin hooks correctly mapped to
.apm/hooks/, HookIntegrator finds them, script path rewriting works for both Claude and Copilot formats deps tree: Renders virtual package paths correctly with SHA and primitive counts- Uninstall: Properly cleans apm_modules, removes deployed skills, updates apm.yml, removes lockfile
- Prune: Correctly detects orphaned packages and cleans up deployed files
- Caching: Second install uses cache — fast re-installs work
- Transitive plugins: Architecture handles them — normalization triggers automatically for transitive deps that are plugins
…-location priority
Add symlinks=False to all shutil.copytree() calls and skip symlinks in file iteration loops within _map_plugin_artifacts(). This prevents a malicious plugin from using symlinks to exfiltrate sensitive files (e.g. /etc/passwd, ~/.ssh/) into the project's .apm/ directory.
The rglob scan in list_packages() treated skills/*/SKILL.md directories inside plugins as independent orphaned packages. Added _is_nested_under_package() helper that walks up to apm_modules/ and skips candidates whose parent already contains apm.yml. Fixes false orphan warnings in deps list and deps tree.
|
@microsoft-github-policy-service agree |
danielmeppiel
left a comment
There was a problem hiding this comment.
A second PR will refine what's missing from the last comment
This is critical work, thanks so much @SebastienDegodez for working on this one!
fix: harden plugin security, validation, tests, and docs (follow-up to #83)
Introduce a plugin management system along with CLI commands for installing and managing plugins from various marketplaces. This enhancement aims to streamline plugin integration and improve user experience.