feat: dynamic target implementation for external secrets sources by Skarlso · Pull Request #5470 · external-secrets/external-secrets

Skarlso · 2025-10-15T05:02:34Z

Problem Statement

This is an implementation of syncing to a custom resource https://github.com/external-secrets/external-secrets/blob/main/design/012-sync-to-custom-resource.md.

The following changes have been applied to the external secret controller and the external secret object:

added manifests to ES to signify what kind of object needs to be tracked
overhauled the templating to accommodate unstructured objects instead of just secrets
added dynamic watches using informers to all the object kinds ( this implementation registered and deregisters informers based on how many usages are for the current used GVK )
modified the external secret controller to check non-secret objects
everything else should work the same way as with secrets ( meaning ownership, merging, updating, deleting )
added documentation
added a feature flag to put all of this behind
updated the helm values files to include custom RBAC access for the custom objects

Related Issue

Proposed Changes

How do you like to solve the issue and why?

Format

Please ensure that your PR follows the following format for the title:

feat(scope): add new feature
fix(scope): fix bug
docs(scope): update documentation
chore(scope): update build tool or dependencies
ref(scope): refactor code
clean(scope): provider cleanup
test(scope): add tests
perf(scope): improve performance
desig(scope): improve design

Where scope is optionally one of:

charts
release
testing
security
templating

Checklist

I have read the contribution guidelines
All commits are signed with git commit --signoff
My changes have reasonable test coverage
All tests pass with make test
I ensured my PR is ready for review with make reviewable

pkg/controllers/externalsecret/externalsecret_controller.go

pkg/controllers/externalsecret/externalsecret_controller_watch_test.go

pkg/controllers/externalsecret/externalsecret_controller_manifest.go

deploy/charts/external-secrets/values.yaml

docs/snippets/manifest-argocd-app.yaml

pkg/controllers/externalsecret/externalsecret_controller.go

pkg/controllers/externalsecret/externalsecret_controller_manifest.go

docs/guides/targeting-custom-resources.md

Skarlso · 2025-10-22T10:56:46Z

Two problems:

literals in templates do not work rn
dataFrom data is base64 encoded in the target object and you probably don't want that.

pkg/controllers/externalsecret/externalsecret_controller_manifest.go

config/crds/bases/external-secrets.io_clusterexternalsecrets.yaml

jakobmoellerdev · 2025-10-22T13:45:04Z

apis/externalsecrets/v1/externalsecret_types.go

+// ManifestTarget defines a custom Kubernetes resource type to be created
+// instead of a Secret. This allows ExternalSecret to create ConfigMaps,
+// Custom Resources, or any other Kubernetes resource type.
+type ManifestTarget struct {


nit: This should be called reference in alignment with existing k8s reference concepts, so ManifestReference

jakobmoellerdev · 2025-10-22T13:47:22Z

pkg/controllers/externalsecret/externalsecret_controller.go

+
+	// watchTracker tracks which GroupVersionKinds we're currently watching
+	// for non-Secret targets to enable drift detection.
+	watchTracker WatchTracker


we have to be very careful here on implications as discussed in community call. please make sure this watchtracker is properly regulated

I'm thinking how to do this nicely. :)

pkg/controllers/externalsecret/externalsecret_controller.go

pkg/controllers/externalsecret/externalsecret_controller_manifest.go

jakobmoellerdev · 2025-10-22T13:57:44Z

runtime/template/v2/template.go

-			secret.Data = make(map[string][]byte)
+		labels[k] = string(val)
+		obj.SetLabels(labels)
+	case "data":


what if someone sets string data on a configmap? whats the behavior?

It will probably fail horribly. :D :D I have to check that as well. :) I assume ( hopefully ) the api would throw back an error that spec ( or any other field ) is not allowed on the given object.

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>

… objects Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>

… tracking objects Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com> On-behalf-of: Gergely Brautigam <gergely.brautigam@sap.com>

…leGenericTarget Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>

Skarlso · 2025-11-03T07:31:44Z

/ok-to-test sha=606f8aec71698a0546710ad6321a3eca2eac814a

eso-service-account-app · 2025-11-03T07:35:24Z

[Bot] - ❌ e2e for 606f8aec71698a0546710ad6321a3eca2eac814a failed

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>

Skarlso · 2025-11-03T07:57:27Z

/ok-to-test sha=86ceaecee942dba44ef84aaa3947d0417dedfb87

eso-service-account-app · 2025-11-03T08:10:24Z

[Bot] - ✅ e2e for 86ceaecee942dba44ef84aaa3947d0417dedfb87 passed

sonarqubecloud · 2025-11-03T18:46:52Z

Quality Gate passed

Issues
1 New issue
3 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

This PR contains the following updates: | Package | Update | Change | |---|---|---| | [external-secrets](https://github.com/external-secrets/external-secrets) | major | `0.20.4` -> `1.0.0` | --- ### Release Notes <details> <summary>external-secrets/external-secrets (external-secrets)</summary> ### [`v1.0.0`](https://github.com/external-secrets/external-secrets/releases/tag/v1.0.0) [Compare Source](external-secrets/external-secrets@v0.20.4...v1.0.0) Image: `ghcr.io/external-secrets/external-secrets:v1.0.0` Image: `ghcr.io/external-secrets/external-secrets:v1.0.0-ubi` Image: `ghcr.io/external-secrets/external-secrets:v1.0.0-ubi-boringssl`  #### What's Changed ##### General - chore: release helm chart for v0.20.4 by [@Skarlso](https://github.com/Skarlso) in [#5502](external-secrets/external-secrets#5502) - chore(actions): fix security alerts for github actions by [@Skarlso](https://github.com/Skarlso) in [#5504](external-secrets/external-secrets#5504) - chore(release): fix incorrect shas from security alert pinning by [@jakobmoellerdev](https://github.com/jakobmoellerdev) in [#5512](external-secrets/external-secrets#5512) - chore: update dependencies by [@eso-service-account-app](https://github.com/eso-service-account-app)\[bot] in [#5519](external-secrets/external-secrets#5519) - fix: upgrade go version by [@Skarlso](https://github.com/Skarlso) in [#5529](external-secrets/external-secrets#5529) - fix(charts): normalize certificate duration default value by [@aslafy-z](https://github.com/aslafy-z) in [#5497](external-secrets/external-secrets#5497) - chore(revive-linter): add revive linter configuration by [@Lumexralph](https://github.com/Lumexralph) in [#5515](external-secrets/external-secrets#5515) - feat: go module separation by [@moolen](https://github.com/moolen) in [#5498](external-secrets/external-secrets#5498) - fix: update deps script to reflect new go mod structure by [@moolen](https://github.com/moolen) in [#5532](external-secrets/external-secrets#5532) - fix: lint by [@gusfcarvalho](https://github.com/gusfcarvalho) in [#5536](external-secrets/external-secrets#5536) - feat: dynamic target implementation for external secrets sources by [@Skarlso](https://github.com/Skarlso) in [#5470](external-secrets/external-secrets#5470) - fix: codeql should not rebuild the world on every run by [@Skarlso](https://github.com/Skarlso) in [#5540](external-secrets/external-secrets#5540) - fix(onepassword): Reduce API usage by [@sondrelg](https://github.com/sondrelg) in [#5410](external-secrets/external-secrets#5410) - feat(esoctl): adds bootstrap generator commands by [@gusfcarvalho](https://github.com/gusfcarvalho) in [#5539](external-secrets/external-secrets#5539) - fix: adds missing externalsecret\_type validation on generatorRef by [@gusfcarvalho](https://github.com/gusfcarvalho) in [#5547](external-secrets/external-secrets#5547) - chore(docs): fix the generic resource doc using the incorrect switch by [@Skarlso](https://github.com/Skarlso) in [#5548](external-secrets/external-secrets#5548) - docs(adopters): add Topicus.Education to adopters list by [@sboschman](https://github.com/sboschman) in [#5551](external-secrets/external-secrets#5551) - fix(build): use more portable bash by [@tiagolobocastro](https://github.com/tiagolobocastro) in [#5537](external-secrets/external-secrets#5537) ##### Dependencies - chore(deps): bump alpine from `4bcff63` to `4b7ce07` in /hack/api-docs by [@dependabot](https://github.com/dependabot)\[bot] in [#5507](external-secrets/external-secrets#5507) - chore(deps): bump golang from 1.25.2 to 1.25.3 by [@dependabot](https://github.com/dependabot)\[bot] in [#5506](external-secrets/external-secrets#5506) - chore(deps): bump golang from `aee43c3` to `aee43c3` by [@dependabot](https://github.com/dependabot)\[bot] in [#5516](external-secrets/external-secrets#5516) - chore(deps): bump golang from 1.25.2-bookworm to 1.25.3-bookworm in /e2e by [@dependabot](https://github.com/dependabot)\[bot] in [#5505](external-secrets/external-secrets#5505) - chore(deps): bump anchore/sbom-action from 0.20.8 to 0.20.9 by [@dependabot](https://github.com/dependabot)\[bot] in [#5508](external-secrets/external-secrets#5508) - chore(deps): bump hashicorp/setup-terraform from [`c529327`](external-secrets/external-secrets@c529327) to [`982f6f0`](external-secrets/external-secrets@982f6f0) by [@dependabot](https://github.com/dependabot)\[bot] in [#5510](external-secrets/external-secrets#5510) - chore(deps): bump actions/github-script from 7.0.1 to 8.0.0 by [@dependabot](https://github.com/dependabot)\[bot] in [#5513](external-secrets/external-secrets#5513) - chore(deps): bump github/codeql-action from 4.30.9 to 4.31.0 by [@dependabot](https://github.com/dependabot)\[bot] in [#5514](external-secrets/external-secrets#5514) - chore(deps): bump actions/cache from 4.2.4 to 4.3.0 by [@dependabot](https://github.com/dependabot)\[bot] in [#5511](external-secrets/external-secrets#5511) - chore(deps): bump regex from 2025.9.18 to 2025.10.23 in /hack/api-docs by [@dependabot](https://github.com/dependabot)\[bot] in [#5509](external-secrets/external-secrets#5509) - chore(deps): bump mkdocs-macros-plugin from 1.4.0 to 1.4.1 in /hack/api-docs by [@dependabot](https://github.com/dependabot)\[bot] in [#5518](external-secrets/external-secrets#5518) - chore(deps): bump termcolor from 3.1.0 to 3.2.0 in /hack/api-docs by [@dependabot](https://github.com/dependabot)\[bot] in [#5517](external-secrets/external-secrets#5517) - chore(deps): bump github/codeql-action from 4.31.0 to 4.31.2 by [@dependabot](https://github.com/dependabot)\[bot] in [#5531](external-secrets/external-secrets#5531) - chore(deps): bump mkdocs-material from 9.6.22 to 9.6.23 in /hack/api-docs by [@dependabot](https://github.com/dependabot)\[bot] in [#5533](external-secrets/external-secrets#5533) #### New Contributors - [@sondrelg](https://github.com/sondrelg) made their first contribution in [#5410](external-secrets/external-secrets#5410) - [@tiagolobocastro](https://github.com/tiagolobocastro) made their first contribution in [#5537](external-secrets/external-secrets#5537) **Full Changelog**: <external-secrets/external-secrets@v0.20.4...v1.0.0> </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).  Reviewed-on: https://gitea.alexlebens.dev/alexlebens/infrastructure/pulls/1985 Co-authored-by: Renovate Bot <renovate-bot@alexlebens.net> Co-committed-by: Renovate Bot <renovate-bot@alexlebens.net>

github-project-automation bot added this to External Secrets Oct 15, 2025

github-actions bot added size/l kind/documentation Categorizes issue or PR as related to documentation. kind/feature Categorizes issue or PR as related to a new feature. and removed size/l labels Oct 15, 2025

Skarlso commented Oct 15, 2025

View reviewed changes

pkg/controllers/externalsecret/externalsecret_controller.go Outdated Show resolved Hide resolved

Skarlso commented Oct 15, 2025

View reviewed changes

pkg/controllers/externalsecret/externalsecret_controller_watch_test.go Outdated Show resolved Hide resolved

Skarlso force-pushed the configmap-as-source branch from 1da7aa5 to d26410c Compare October 15, 2025 05:38

Skarlso commented Oct 15, 2025

View reviewed changes

pkg/controllers/externalsecret/externalsecret_controller_manifest.go Outdated Show resolved Hide resolved