feat: add kubernetes secret provider by yohamta0 · Pull Request #2006 · dagucloud/dagu

yohamta0 · 2026-04-16T09:14:45Z

Summary

add a Kubernetes Secret resolver for DAG secrets: entries
support global and per-secret Kubernetes namespace, kubeconfig, and context settings
update config schema and focused resolver/config tests

Testing

go test ./internal/cmn/secrets ./internal/cmn/config ./internal/cmn/schema ./internal/core/spec -count=1
go test ./internal/intg -run 'Secret|Secrets' -count=1
go test ./internal/runtime/agent ./internal/runtime -run 'Secret|Secrets|Env' -count=1
go test ./internal/cmd -run 'Secret|Config' -count=1

Summary by CodeRabbit

New Features
- Added Kubernetes as a built-in secrets provider.
- Support configuring Kubernetes secrets defaults: namespace, kubeconfig, and context (config file, env, and YAML).
Bug Fixes
- Fixed secrets-loading so non-Vault providers (including Kubernetes) are properly honored.
- Improved user-facing errors for missing secrets, missing keys, not-found and permission-denied cases.
Tests
- Added tests covering Kubernetes secrets parsing, resolution, client caching, and error scenarios.

coderabbitai · 2026-04-16T09:17:39Z

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 46e22f6a-1cdc-4934-a54a-5197f42da8dd

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

📝 Walkthrough

Walkthrough

Adds Kubernetes as a secrets provider: new config types and env bindings, loader updates, a kubernetes secrets resolver with client caching and kubeconfig/context handling, JSON schema additions, and tests updating/covering configuration and resolver behavior.

Changes

Cohort / File(s)	Summary
Kubernetes Secrets Configuration `internal/cmn/config/config.go`, `internal/cmn/config/definition.go`, `internal/cmn/config/loader.go`, `internal/cmn/schema/config.schema.json`	Added `KubernetesSecretsConfig` / `KubernetesSecretsDef`; loader now reads `secrets.kubernetes.*` from YAML and env; schema extended with `KubernetesSecretsDef` (namespace, kubeconfig, context).
Configuration Tests `internal/cmn/config/loader_test.go`	Expanded tests to assert Kubernetes secrets config loaded from YAML and environment; renamed vault test to general secrets config test.
Kubernetes Secrets Resolver `internal/cmn/secrets/kubernetes.go`, `internal/cmn/secrets/kubernetes_test.go`	New `kubernetesResolver` registered as `"kubernetes"` implementing validation, Resolve, CheckAccessibility; parses multiple ref formats, applies namespace/defaults, resolves via Kubernetes API with kubeconfig/context handling, client caching, and user-friendly errors; comprehensive unit tests added.
Secrets Registry Tests `internal/cmn/secrets/resolver_test.go`	Tests updated to expect `"kubernetes"` in built-in providers and adjust provider counts.
Docs / Field Comment `internal/core/dag.go`	Updated `SecretRef.Provider` comment to include `"kubernetes"` as an example provider.

Sequence Diagram(s)

sequenceDiagram
    participant App as Application
    participant Reg as SecretsRegistry
    participant KRes as KubernetesResolver
    participant Cache as ClientCache
    participant K8s as KubernetesAPI

    App->>Reg: Request secret(ref)
    activate Reg
    Reg->>KRes: Lookup & Validate(ref)
    activate KRes
    KRes->>KRes: Parse ref (namespace/name/key, options)
    KRes-->>Reg: Validation result
    deactivate KRes
    Reg->>KRes: Resolve(ctx, ref)
    activate KRes
    KRes->>KRes: Derive client settings (config + ref.options)
    KRes->>Cache: getClient(settings)
    activate Cache
    Cache-->>KRes: client (cached or new)
    deactivate Cache
    KRes->>K8s: Get Secret(namespace, name)
    activate K8s
    K8s-->>KRes: Secret{data}
    deactivate K8s
    KRes->>KRes: Extract data key, format errors if missing/forbidden/not-found
    KRes-->>Reg: SecretValue / error
    deactivate KRes
    Reg-->>App: SecretValue / error
    deactivate Reg

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'feat: add kubernetes secret provider' directly and clearly summarizes the main change: adding support for Kubernetes as a new secrets provider, which is evident across all modified files.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch feat-kubernetes-secret-provider

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

yohamta0 · 2026-04-16T09:20:38Z

@coderabbitai full review

coderabbitai · 2026-04-16T09:20:44Z

✅ Actions performed

Full review triggered.

coderabbitai

Actionable comments posted: 1

🧹 Nitpick comments (1)

internal/cmn/config/loader.go (1)

421-426: Normalize YAML secrets.kubernetes.kubeconfig paths for parity with env loading.

DAGU_SECRETS_KUBERNETES_KUBECONFIG is treated as a path (Line 1657), but YAML secrets.kubernetes.kubeconfig is copied as-is. That creates inconsistent behavior for relative paths.

Proposed patch

 func (l *ConfigLoader) loadSecretsConfig(cfg *Config, def Definition) {
 	if def.Secrets == nil {
 		return
 	}
@@
 	if def.Secrets.Kubernetes != nil {
+		kubeconfig := def.Secrets.Kubernetes.Kubeconfig
+		if resolved, err := l.resolvePath("secrets.kubernetes.kubeconfig", kubeconfig); err == nil {
+			kubeconfig = resolved
+		} else if kubeconfig != "" {
+			l.warnings = append(l.warnings, err.Error())
+		}
+
 		cfg.Secrets.Kubernetes = KubernetesSecretsConfig{
 			Namespace:  def.Secrets.Kubernetes.Namespace,
-			Kubeconfig: def.Secrets.Kubernetes.Kubeconfig,
+			Kubeconfig: kubeconfig,
 			Context:    def.Secrets.Kubernetes.Context,
 		}
 	}
 }

Also applies to: 1657-1657

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@internal/cmn/config/loader.go` around lines 421 - 426, The YAML loader copies
def.Secrets.Kubernetes.Kubeconfig into cfg.Secrets.Kubernetes as-is, causing
inconsistent handling vs the env var path logic for
DAGU_SECRETS_KUBERNETES_KUBECONFIG; update the mapping that sets
cfg.Secrets.Kubernetes.Kubeconfig in loader.go to normalize/resolve the path the
same way the env-loading code does (e.g., use the same helper or call
path/filepath resolution to convert relative paths to an absolute/clean path) so
KubernetesSecretsConfig.Kubeconfig receives a normalized path consistent with
the env path handling.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@internal/cmn/secrets/kubernetes.go`:
- Around line 91-93: The error currently reveals all secret keys by calling
getByteMapKeys(secret.Data) and including them in the error returned; remove
that sensitive enumeration and return a generic missing-key error instead (or
only include the available keys behind an explicit debug flag). Update the error
return in the kubernetes secret lookup (the block that references
parsed.dataKey, parsed.secretName, parsed.namespace and getByteMapKeys) to omit
strings.Join(available, ", ") so the message does not leak key names, or gate
that enumeration behind a debug/logging path rather than returning it in the
error.

---

Nitpick comments:
In `@internal/cmn/config/loader.go`:
- Around line 421-426: The YAML loader copies def.Secrets.Kubernetes.Kubeconfig
into cfg.Secrets.Kubernetes as-is, causing inconsistent handling vs the env var
path logic for DAGU_SECRETS_KUBERNETES_KUBECONFIG; update the mapping that sets
cfg.Secrets.Kubernetes.Kubeconfig in loader.go to normalize/resolve the path the
same way the env-loading code does (e.g., use the same helper or call
path/filepath resolution to convert relative paths to an absolute/clean path) so
KubernetesSecretsConfig.Kubeconfig receives a normalized path consistent with
the env path handling.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: f3f0fa6d-b675-46cb-92ea-ad624c723d70

📥 Commits

Reviewing files that changed from the base of the PR and between 74aea12 and 9e82c0e.

📒 Files selected for processing (9)

internal/cmn/config/config.go
internal/cmn/config/definition.go
internal/cmn/config/loader.go
internal/cmn/config/loader_test.go
internal/cmn/schema/config.schema.json
internal/cmn/secrets/kubernetes.go
internal/cmn/secrets/kubernetes_test.go
internal/cmn/secrets/resolver_test.go
internal/core/dag.go

Merged features from upstream: - feat: add embedded Dagu API (dagucloud#2011) - feat: add edit-and-retry DAG runs (dagucloud#2010) - feat: add bulk DAG-run deletion in web UI (dagucloud#2009) - feat: add kubernetes secret provider (dagucloud#2006) - feat: make workspace selection global (dagucloud#2015) - feat: show cockpit run artifacts (dagucloud#2017) - feat: allow disabling DAG retry policy (dagucloud#2018) - feat: make DAG labels canonical, deprecate tags (dagucloud#2013) - feat: add workflow design workspace (dagucloud#2012) - feat: add step with/config alias (dagucloud#2021) - fix: allow runtime custom step inputs (dagucloud#2005) - fix: align embedded engine run labels (dagucloud#2014) - Various CI/test/docs improvements Conflict resolution: - Kept fork i18n files (ui/src/i18n/, LanguageSwitcher) - Merged i18n hooks with upstream structural changes in App.tsx, Layout.tsx, menu.tsx, DAGStatus.tsx, etc. - Adopted upstream labels-over-tags rename (TagCombobox -> LabelCombobox)

yohamta0 force-pushed the feat-kubernetes-secret-provider branch from 9a196f8 to 9e82c0e Compare April 16, 2026 09:19

coderabbitai Bot reviewed Apr 16, 2026

View reviewed changes

Comment thread internal/cmn/secrets/kubernetes.go Outdated

feat: add kubernetes secret provider

7a1e051

yohamta0 force-pushed the feat-kubernetes-secret-provider branch from 9e82c0e to 7a1e051 Compare April 16, 2026 09:34

yohamta0 added 4 commits April 16, 2026 19:12

ci: authenticate docker hub pulls

6998fe2

test: stabilize windows ci

c889ad9

test: harden runtime manager status check

07f59d6

ci: gate e2e on test matrix

e4e6ba4

yohamta0 merged commit b8b8a01 into main Apr 16, 2026
10 checks passed

yohamta0 deleted the feat-kubernetes-secret-provider branch April 16, 2026 11:13

coderabbitai Bot mentioned this pull request May 15, 2026

feat: add simplified secret management #2153

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: add kubernetes secret provider#2006

feat: add kubernetes secret provider#2006
yohamta0 merged 5 commits into
mainfrom
feat-kubernetes-secret-provider

yohamta0 commented Apr 16, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Apr 16, 2026 •

edited

Loading

Review skipped

Walkthrough

Changes

Sequence Diagram(s)

Estimated code review effort

❌ Failed checks (1 warning)

Uh oh!

yohamta0 commented Apr 16, 2026

Uh oh!

coderabbitai Bot commented Apr 16, 2026

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

yohamta0 commented Apr 16, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Testing

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Apr 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Walkthrough

Changes

Sequence Diagram(s)

Estimated code review effort

❌ Failed checks (1 warning)

Uh oh!

yohamta0 commented Apr 16, 2026

Uh oh!

coderabbitai Bot commented Apr 16, 2026

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

yohamta0 commented Apr 16, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Apr 16, 2026 •

edited

Loading