Skip to content

Split Monolithic Go Module into Multiple Modules #5494

@moolen

Description

@moolen

Problem

The current monolithic Go module forces all consumers to pull in ~100+ provider dependencies, even when they only need the API types. The pkg/ directory is cluttered with mixed concerns (controllers, providers, generators, utilities), making it difficult to manage dependencies and hindering our path toward out-of-tree provider support.

Proposed Solution

Restructure the repository into multiple independent Go modules with clear dependency boundaries:

external-secrets/
├── apis/              # API types only (minimal deps)
├── runtime/           # Shared utilities (webhook, logging, metrics)
├── providers/v1         # Individual provider modules
│   ├── aws/
│   ├── vault/
│   ├── gcp/
│   └── ...
├── generators/v1        # Individual generator modules
│   ├── password/
│   ├── sts/
│   └── ...
└── pkg/               # Core controllers & orchestration

Module Structure

  1. /apis - CRD types and interfaces

    • Minimal dependencies (k8s.io/*, controller-runtime only)
    • Consumed by: all other modules, external projects
  2. /runtime - Common utilities

    • Shared code (webhook helpers, logging, metrics, validation)
    • Depends on: apis
    • Consumed by: providers, generators, controllers
  3. /providers/v1/* - Individual provider modules

    • Example: github.com/external-secrets/external-secrets/providers/aws
    • Each provider in its own module with isolated dependencies
    • Depends on: apis, runtime
  4. /generators/v1/* - Individual generator modules

    • Each generator in its own module
    • Depends on: apis, runtime
  5. /pkg (main module) - Core orchestration

    • Controllers and main binary
    • Depends on: apis, runtime, (optional) specific providers/generators

Breaking Changes

This is a breaking change for external consumers importing our packages. I don't think we need to provide a migration guide and just mention it in the release notes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    StaleThis issue/Pull Request is stale and will be automatically closedkind/featureCategorizes issue or PR as related to a new feature.

    Projects

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions