Skip to content

Epic: Make DataDesignerPlugins an install-ready Data Designer plugin tap #15

@eric-tramel

Description

@eric-tramel

Why

DataDesigner can already discover plugins that are installed in the local Python environment through the data_designer.plugins entry point group. That is the runtime half of the plugin ecosystem.

The missing half is pre-install discovery. Users and the future data-designer plugins ... CLI need a stable tap that can answer what exists, whether it is compatible with the current Python/DataDesigner environment, where the docs live, and what install source should be used. DDPlugins should provide the default NVIDIA tap and serve as the reference implementation for external tap repositories.

System boundary

DDPlugins owns:

  • generated catalog metadata
  • the default raw JSON tap URL
  • plugin packages and per-plugin docs
  • scaffolding and reference authoring workflows
  • release metadata and validation that keeps the catalog installable

DataDesigner core owns:

  • runtime PluginRegistry discovery of installed entry points
  • data-designer plugins ... CLI commands
  • tap caching and installer execution
  • compatibility filtering in the user environment
  • post-install entry-point verification

This epic should keep those boundaries clean. DDPlugins should publish a complete tap contract; it should not grow a second runtime plugin loader or installer.

Concrete decisions this stack implements

  • Default NVIDIA tap URL: https://raw.githubusercontent.com/NVIDIA-NeMo/DataDesignerPlugins/main/catalog/plugins.json.
  • Catalog format: raw JSON, schema version 2.
  • Required v2 additions over v1: source and docs.url.
  • Source types: pypi, git, and local-authoring-only path.
  • Default NVIDIA source type: pypi.
  • Generated repo config: [tool.ddp.tap] in the root pyproject.toml.
  • Runtime discovery remains installed data_designer.plugins entry points; tap discovery is pre-install metadata only.

Sub-issues

Dependency order

Order Issue Depends on Blocks / enables
1 #16 #15 Exact schema v2 contract and source union
2 #21 #16 Shared tap metadata config for catalog/scaffold/source generation
3 #17 #16, #21 Generated schema v2 catalog with default PyPI source and docs URLs
4 #18 #16, #17, #21 Git/path source generation and exact install-target derivation
5 #19 #17 Default raw NVIDIA tap URL for DataDesigner configuration
6 #20 #17, #18, #21 Release validation for catalog/source/tag consistency
7 #22 #21 Plugin-type-complete authoring workflow
8 #23 #16, #17, #18, #19 DataDesigner CLI contract fixtures
9 #24 #19 Trust/governance language for docs and CLI prompts
10 #25 #16, #18, #19, #20, #21, #22, #24 Consumer and maintainer docs completion

Acceptance criteria

  • catalog/plugins.json is generated as schema v2 raw JSON.
  • Every catalog entry includes docs and install source metadata in addition to runtime entry-point and compatibility metadata.
  • The default NVIDIA tap URL above is documented and consumable without GitHub API auth.
  • Multi-plugin packages list each runtime plugin independently while sharing one package install source.
  • Release validation catches stale or incomplete tap metadata before a plugin is published.
  • ddp remains the canonical maintainer workflow and becomes usable as a reference tap generator for external repos.
  • DDPlugins provides fixtures/docs that DataDesigner can use to implement and test plugins list/search/info/install without DDPlugins-specific special cases.

DataDesigner follow-up markers

These are not DDPlugins work items, but the linked DDPlugins issues should inform future DataDesigner issues:

  • DD-CLI-CATALOG-SCHEMA: reject unsupported catalog schema versions and normalize schema v2 entries.
  • DD-CLI-DEFAULT-TAP: configure the default NVIDIA tap URL.
  • DD-CLI-READONLY-PLUGINS: implement list/search/info/installed behavior.
  • DD-CLI-INSTALLER: build install plans from source metadata, prompt before mutation, run uv/pip, and verify entry-point discovery.
  • DD-CLI-TAPS: implement external tap add/list/cache/trust UX.
  • DD-CORE-PROCESSOR-PLUGIN-VALIDATION: confirm or extend processor validation if scaffolded processor plugins need stronger public validation.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestepicUmbrella issue tracking related workplugin tapPlugin catalog and tap ecosystem work

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions