Skip to content

fix: resolve runtime env references in DAG env#2131

Merged
yohamta0 merged 6 commits into
mainfrom
codex/fix-runtime-env-expansion
May 9, 2026
Merged

fix: resolve runtime env references in DAG env#2131
yohamta0 merged 6 commits into
mainfrom
codex/fix-runtime-env-expansion

Conversation

@yohamta0

@yohamta0 yohamta0 commented May 8, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • resolve DAG-level env references that depend on run-scoped directories at runtime
  • evaluate step and container env entries sequentially so later entries can reference earlier ones
  • add regression coverage for runtime-managed env references

Testing

  • go test ./internal/core/exec ./internal/runtime -run 'TestNewContext_DAGEnvCanReferenceRuntimeManagedDirs|TestSetupVariables_StepEnvEvaluatesSequentiallyWithRuntimeVars' -count=1
  • go test -race ./internal/intg -run '^TestComplexDependencies$' -count=50 -timeout=120s

Summary by CodeRabbit

  • New Features

    • DAG env vars can reference runtime-managed directories (docs/work/artifacts) and other env vars, supporting chained/sequential interpolation across scopes.
    • Runtime-managed env values are computed centrally and reapplied with final precedence so user-provided envs cannot override them; evaluated env entries are applied incrementally during execution.
  • Tests

    • Added tests validating sequential env evaluation and protection/exposure of runtime-managed directory values during DAG run and step/container resolution.

@coderabbitai

coderabbitai Bot commented May 8, 2026

Copy link
Copy Markdown

Review Change Stack

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: ef6503dd-55b0-4818-9175-571a64262022

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Runtime-managed envs are built centrally, DAG Env entries are runtime-evaluated sequentially with managed keys protected, user envs are merged, and managed envs are reapplied for final precedence. Runner now writes evaluated step/container env entries directly into the env scope. Tests validate sequential evaluation and precedence.

Changes

Environment Variable Sequential Evaluation

Layer / File(s) Summary
Managed DAG-run env registry
internal/core/exec/context_env.go
Adds managedDAGRunEnv descriptors and buildManagedDAGRunEnvs to produce runtime-managed DAG-run env key/value pairs.
DAG Env Runtime Evaluator
internal/core/exec/context.go
Adds evaluateDAGEnvRuntime and strings import to parse key=value entries and evaluate values sequentially while protecting managed keys.
Context Env Assembly with Precedence
internal/core/exec/context.go
Refactors NewContext to seed base envs, build managed envs, runtime-evaluate dag.Env (protecting managed keys), merge user WithEnvVars, and reapply managed envs to enforce final precedence.
Step Env Sequential Application
internal/runtime/runner.go
Changes Runner.setupVariables to apply each evaluated step/container env entry directly into env.Scope as entries are resolved (removes intermediate accumulator).
Behavior Verification — DAG env interpolation
internal/core/exec/context_test.go
Adds test confirming DAG Env can interpolate runtime-managed docs and artifacts dirs and that artifact dir is exposed in final env map.
Behavior Verification — Runner env evaluation
internal/runtime/runner_internal_test.go
Adds table-driven tests asserting chained env expressions evaluate sequentially from step/container/DAG container sources.
Managed env definitions tests
internal/core/exec/context_env_internal_test.go
Adds tests ensuring managed env keys are unique/non-empty and are protected from overrides while REF_ variants are available during evaluation.

Sequence Diagram(s)

sequenceDiagram
  participant OS as OS Env
  participant NewCtx as NewContext
  participant Runner as Runner.setupVariables
  participant Scope as Env Scope
  OS->>NewCtx: provide base env
  NewCtx->>Scope: seed base + managed runtimeEnvs
  NewCtx->>Scope: evaluate DAG Env (protected keys)
  NewCtx->>Runner: pass env scope
  Runner->>Scope: evaluate step/container env entries sequentially
  Scope-->>Runner: provide vars for later entries
  Runner->>Scope: write entries as evaluated
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

  • dagucloud/dagu#2080: Both PRs touch runtime environment setup and ensure consistent runtime-managed env handling during evaluation.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 18.18% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically describes the main change: resolving runtime environment variable references in DAG environment configuration.
Description check ✅ Passed The PR description covers all required sections (Summary, Changes implicitly through summary bullets, and Testing) with clear, specific information about the changes and testing approach.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/fix-runtime-env-expansion

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
internal/runtime/runner_internal_test.go (1)

89-119: ⚡ Quick win

Add a container-env regression alongside this step-env case.

This only exercises Step.Env, but the behavior change also covers step.Container.Env and DAG container fallback. One small case that resolves a later container env entry from an earlier one would lock down the other new branch in setupVariables.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/runtime/runner_internal_test.go` around lines 89 - 119, Add a
parallel regression that exercises container-level env resolution in
setupVariables: extend
TestSetupVariables_StepEnvEvaluatesSequentiallyWithRuntimeVars to also set
step.Container.Env (e.g., "WORK_DIR=${DAG_RUN_ARTIFACTS_DIR}" and
"CURRENT_IDEA_PATH=${WORK_DIR}/current_idea.md") and assert those container envs
are resolved the same way as step.Env; additionally add a sub-case where
step.Container is nil so setupVariables falls back to DAG.Container.Env and
verify fallback resolution works (use NewPlan/plan.GetNodeByName,
runner.setupVariables, and AllEnvsMap as in the existing test to locate and
validate WORK_DIR and CURRENT_IDEA_PATH).
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@internal/core/exec/context.go`:
- Around line 339-343: The DAG env evaluation currently allows user DAG entries
to write reserved runtime variables (runtimeEnvs) into the interim env map,
causing derived vars to resolve against incorrect values; fix by ensuring
runtimeEnvs are injected into envs before calling evaluateDAGEnvRuntime (or
alter evaluateDAGEnvRuntime to ignore/skip keys present in runtimeEnvs) so
reserved keys are immutable during evaluation; apply the same change to the
other evaluation site around the 390-402 block; reference evaluateDAGEnvRuntime,
runtimeEnvs, envs, and stringutil.KeyValuesToMap when making the adjustment.

---

Nitpick comments:
In `@internal/runtime/runner_internal_test.go`:
- Around line 89-119: Add a parallel regression that exercises container-level
env resolution in setupVariables: extend
TestSetupVariables_StepEnvEvaluatesSequentiallyWithRuntimeVars to also set
step.Container.Env (e.g., "WORK_DIR=${DAG_RUN_ARTIFACTS_DIR}" and
"CURRENT_IDEA_PATH=${WORK_DIR}/current_idea.md") and assert those container envs
are resolved the same way as step.Env; additionally add a sub-case where
step.Container is nil so setupVariables falls back to DAG.Container.Env and
verify fallback resolution works (use NewPlan/plan.GetNodeByName,
runner.setupVariables, and AllEnvsMap as in the existing test to locate and
validate WORK_DIR and CURRENT_IDEA_PATH).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 02b536a3-ec8a-4bc5-a95c-c851fc306119

📥 Commits

Reviewing files that changed from the base of the PR and between 9d0d6a8 and 840f798.

📒 Files selected for processing (4)
  • internal/core/exec/context.go
  • internal/core/exec/context_test.go
  • internal/runtime/runner.go
  • internal/runtime/runner_internal_test.go

Comment thread internal/core/exec/context.go Outdated
@yohamta0

yohamta0 commented May 8, 2026

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented May 8, 2026

Copy link
Copy Markdown
✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@yohamta0

yohamta0 commented May 8, 2026

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented May 8, 2026

Copy link
Copy Markdown
✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@yohamta0

yohamta0 commented May 8, 2026

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented May 8, 2026

Copy link
Copy Markdown
✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
internal/core/exec/context_env_internal_test.go (1)

23-25: ⚡ Quick win

Use require.Failf instead of t.Fatalf in this test.

This keeps _test.go assertions consistent with the repository’s testify rule.

Proposed change
 		if _, ok := seen[env.key]; ok {
-			t.Fatalf("duplicate managed DAG-run env key: %s", env.key)
+			require.Failf(t, "duplicate managed DAG-run env key", "%s", env.key)
 		}

As per coding guidelines: **/*_test.go: Use stretchr/testify assertions for testing in Go.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/core/exec/context_env_internal_test.go` around lines 23 - 25,
Replace the direct t.Fatalf call with the testify require helper: import
"github.com/stretchr/testify/require" and change the failing assertion in the
duplicate check (the block that tests seen[env.key] and currently calls
t.Fatalf("duplicate managed DAG-run env key: %s", env.key)) to use
require.Failf(t, "<short message>", "%s", env.key) (or require.Failf(t,
"duplicate managed DAG-run env key", "%s", env.key)) so the test follows the
repository’s stretchr/testify convention.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@internal/core/exec/context_env_internal_test.go`:
- Around line 23-25: Replace the direct t.Fatalf call with the testify require
helper: import "github.com/stretchr/testify/require" and change the failing
assertion in the duplicate check (the block that tests seen[env.key] and
currently calls t.Fatalf("duplicate managed DAG-run env key: %s", env.key)) to
use require.Failf(t, "<short message>", "%s", env.key) (or require.Failf(t,
"duplicate managed DAG-run env key", "%s", env.key)) so the test follows the
repository’s stretchr/testify convention.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 77d3062d-aa0d-4829-8413-86cf0801f909

📥 Commits

Reviewing files that changed from the base of the PR and between 040f469 and f20188b.

📒 Files selected for processing (4)
  • internal/core/exec/context.go
  • internal/core/exec/context_env.go
  • internal/core/exec/context_env_internal_test.go
  • internal/core/exec/context_test.go
🚧 Files skipped from review as they are similar to previous changes (2)
  • internal/core/exec/context.go
  • internal/core/exec/context_test.go

@yohamta0

yohamta0 commented May 8, 2026

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented May 8, 2026

Copy link
Copy Markdown
✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@yohamta0 yohamta0 merged commit ddcd829 into main May 9, 2026
15 of 16 checks passed
@yohamta0 yohamta0 deleted the codex/fix-runtime-env-expansion branch May 9, 2026 04:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant