fix: propagate component-type level dependencies through stack processor#2127
fix: propagate component-type level dependencies through stack processor#2127
Conversation
The stack processor was dropping `dependencies` data from global (Scope 1) and component-type (Scope 2) levels, preventing toolchain auto-install when dependencies were configured at the mixin level (terraform.dependencies.tools). This fix: - Extracts and merges `dependencies` from global config (Scope 1) - Extracts and merges `dependencies` from each component-type section (Scope 2) - Adds GlobalDependencies to ComponentProcessorOptions - Includes GlobalDependencies in the merge chain (lowest priority) - Applies pattern consistently across terraform, helmfile, packer, ansible Tests verify that Scope 2 dependencies now flow through to ComponentSection and trigger toolchain auto-install via the resolver. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Dependency Review✅ No vulnerabilities or license issues found.Scanned FilesNone |
Verifies that component-instance deps (Scope 3) properly override component-type deps (Scope 2), while global deps (Scope 1) are inherited through. Tests both override and pure-inheritance scenarios in the same stack to confirm correct merge precedence end-to-end. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds sentinel errors for invalid dependencies sections and wires per-component dependency handling: stack processing now reads global (Scope 1), component-type (Scope 2), and component-instance (Scope 3) dependencies, merges them with defined precedence, exposes merged results via a new GlobalDependencies field, and extends tests/fixtures for Terraform toolchain scenarios. Changes
Sequence Diagram(s)sequenceDiagram
actor Stack as Stack Config
participant Proc as Stack Processor
participant Merge as Dependency Merger
participant Comp as Component Processor
Stack->>Proc: Load stack YAML
activate Proc
Proc->>Proc: Extract global dependencies (Scope 1)
Proc->>Proc: Extract component-type dependencies (Scope 2)
Proc->>Proc: Extract component-instance dependencies (Scope 3)
Proc->>Merge: Merge sources [Global, Component-Type, Component-Instance]
activate Merge
Merge->>Merge: Apply precedence: rgba(0,128,255,0.5) Global → rgba(0,200,100,0.5) Component-Type → rgba(255,128,0,0.5) Component-Instance
Merge-->>Proc: Return merged GlobalDependencies
deactivate Merge
Proc->>Comp: Invoke with ComponentProcessorOptions (includes GlobalDependencies)
activate Comp
Comp->>Comp: Process component using merged dependencies
Comp-->>Proc: Return processed component
deactivate Comp
Proc-->>Stack: Finish processing
deactivate Proc
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Tip Try Coding Plans. Let us write the prompt for your AI agent so you can ship faster (with fewer bugs). Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
internal/exec/stack_processor_process_stacks.go (1)
343-355: Consider extracting a shared helper for dependency parse+merge.The same read/validate/merge sequence is repeated four times. A helper would reduce drift risk and keep precedence/error behavior consistent.
♻️ Refactor sketch
+func readAndMergeDependencies( + atmosConfig *schema.AtmosConfiguration, + stackName string, + globalDeps map[string]any, + componentSection map[string]any, + invalidSectionErr error, +) (map[string]any, error) { + componentDeps := map[string]any{} + if i, ok := componentSection[cfg.DependenciesSectionName]; ok { + deps, ok := i.(map[string]any) + if !ok { + return nil, fmt.Errorf(errFormatWithFile, invalidSectionErr, stackName) + } + componentDeps = deps + } + + return m.Merge(atmosConfig, []map[string]any{globalDeps, componentDeps}) +}Also applies to: 413-425, 483-495, 553-564
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@internal/exec/stack_processor_process_stacks.go` around lines 343 - 355, Extract a small helper (e.g., parseAndMergeDependencies) that encapsulates the repeated read/validate/merge sequence: look up the section map (like globalTerraformSection[cfg.DependenciesSectionName]), assert it to map[string]any (terraformDependencies), then call m.Merge(atmosConfig, []map[string]any{globalDependenciesSection, terraformDependencies}) and return the merged map or error; replace each duplicated block (the instances using globalTerraformSection, globalDependenciesSection, terraformDependencies and calls to m.Merge) with a call to that helper so precedence and error handling remain consistent across the four locations.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tests/terraform_toolchain_integration_test.go`:
- Around line 422-423: Replace hardcoded path strings assigned to the workDir
variable and passed to t.Chdir in the three new tests with platform-safe
filepath.Join(...) calls; specifically, update the workDir assignments (the ones
currently using "fixtures/scenarios/toolchain-terraform-integration" and the two
other occurrences referenced) to call filepath.Join("fixtures", "scenarios",
"toolchain-terraform-integration") (and the corresponding folder segments for
the other two tests), keep the subsequent t.Chdir(workDir) calls unchanged, and
add an import for "path/filepath" if not already present so these tests use
cross-platform path separators.
---
Nitpick comments:
In `@internal/exec/stack_processor_process_stacks.go`:
- Around line 343-355: Extract a small helper (e.g., parseAndMergeDependencies)
that encapsulates the repeated read/validate/merge sequence: look up the section
map (like globalTerraformSection[cfg.DependenciesSectionName]), assert it to
map[string]any (terraformDependencies), then call m.Merge(atmosConfig,
[]map[string]any{globalDependenciesSection, terraformDependencies}) and return
the merged map or error; replace each duplicated block (the instances using
globalTerraformSection, globalDependenciesSection, terraformDependencies and
calls to m.Merge) with a call to that helper so precedence and error handling
remain consistent across the four locations.
ℹ️ Review info
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (10)
errors/errors.gointernal/exec/stack_processor_merge.gointernal/exec/stack_processor_process_stacks.gointernal/exec/stack_processor_process_stacks_helpers.gotests/fixtures/scenarios/toolchain-terraform-integration/components/terraform/test-component-inherit/main.tftests/fixtures/scenarios/toolchain-terraform-integration/components/terraform/test-component-mixin/main.tftests/fixtures/scenarios/toolchain-terraform-integration/components/terraform/test-component-override/main.tftests/fixtures/scenarios/toolchain-terraform-integration/stacks/deploy/mixin-test.yamltests/fixtures/scenarios/toolchain-terraform-integration/stacks/deploy/override-test.yamltests/terraform_toolchain_integration_test.go
…eleases These tests fetch live GitHub releases and snapshot-compare the output. Release titles that aren't version numbers (e.g., "Screenshot", "Demo: change-with-nth") aren't sanitized, so snapshots break whenever upstream repos publish new releases. Add diff ignore patterns to filter out volatile table data rows while preserving structural validation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2127 +/- ##
==========================================
+ Coverage 76.54% 76.58% +0.04%
==========================================
Files 832 832
Lines 79458 79500 +42
==========================================
+ Hits 60820 60887 +67
+ Misses 14842 14817 -25
Partials 3796 3796
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
…est paths Add unit tests for dependency section type validation (global, terraform, helmfile, packer, ansible) and ansible section/component type validation to improve patch coverage. Use filepath.Join for cross-platform path compliance in toolchain integration tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tests/terraform_toolchain_integration_test.go`:
- Around line 501-523: The test currently ignores errors from os.Pipe() and
never closes the read ends (rOut, rErr), risking descriptor leaks and losing
visibility if the test aborts; update the setup around os.Pipe() and
stdout/stderr redirection to check and handle errors returned by os.Pipe(), and
use defer to restore os.Stdout/os.Stderr and to Close() both writer ends (wOut,
wErr) and reader ends (rOut, rErr) so they are closed even on early returns;
keep the exec.ExecuteTerraform(info) call but ensure the defer runs after
assigning os.Stdout/os.Stderr so cleanup always happens, and then read from
rOut/rErr (e.g., via buf.ReadFrom) before returning.
ℹ️ Review info
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (2)
internal/exec/stack_processor_process_stacks_test.gotests/terraform_toolchain_integration_test.go
Check os.Pipe() errors, close read ends, use t.Cleanup to guarantee stdout/stderr restoration on early abort, and assign return values from ReadFrom to satisfy errcheck. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
tests/terraform_toolchain_integration_test.go (1)
560-634: Prefer a table-driven structure for the precedence scenarios.This block tests multiple scenarios (
override,inherit); converting to table-driven subtests (t.Run) will cut duplication and scale better as precedence cases grow.♻️ Refactor sketch
-// --- Test component that OVERRIDES Scope 2 at Scope 3 --- -overrideInfo := ... -... -// --- Test component that INHERITS from Scope 1+2 only --- -inheritInfo := ... +cases := []struct { + name string + component string + expectTF string + expectTFLint string + expectJQ string + expectCheckov bool +}{ + {name: "override", component: "test-component-override", expectTF: "1.10.3", expectTFLint: "^0.54.0", expectJQ: "latest", expectCheckov: true}, + {name: "inherit", component: "test-component-inherit", expectTF: "1.6.0", expectTFLint: "^0.54.0", expectJQ: "latest", expectCheckov: false}, +} +for _, tc := range cases { + tc := tc + t.Run(tc.name, func(t *testing.T) { + // init config, process stacks, assert merged dependencies + }) +}As per coding guidelines: "Use table-driven tests for testing multiple scenarios in Go."
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/terraform_toolchain_integration_test.go` around lines 560 - 634, The test TestTerraformToolchain_DependencyPrecedence contains two almost-identical scenarios (override and inherit); refactor it into a table-driven test with subtests using t.Run to reduce duplication. Create a slice of test cases describing inputs (ComponentFromArg and expected tool values) and loop over them, calling cfg.InitCliConfig and exec.ProcessStacks inside each case and asserting against the case's expected values (use the existing symbols overrideInfo/inheritInfo only as templates for fields like ComponentFromArg, and reuse compSection/compSection2 logic inside the loop). Ensure each case has a descriptive name and keeps the same assertions for tools["terraform"], tools["tflint"], tools["jq"], and presence/absence of tools["checkov"] so behavior remains identical.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tests/terraform_toolchain_integration_test.go`:
- Around line 475-547: The test
TestTerraformToolchain_MixinLevelDependencies_PlanCommand must restore PATH to
avoid leaking state because exec.ExecuteTerraform can mutate process-wide PATH;
capture the current PATH (oldPath := os.Getenv("PATH")) and call
t.Setenv("PATH", oldPath) (using testing.T.Setenv) inside the test before
invoking exec.ExecuteTerraform so the original PATH is automatically restored
after the test finishes; update the test around the exec.ExecuteTerraform call
to use t.Setenv rather than manual os.Setenv/os.Unsetenv.
---
Nitpick comments:
In `@tests/terraform_toolchain_integration_test.go`:
- Around line 560-634: The test TestTerraformToolchain_DependencyPrecedence
contains two almost-identical scenarios (override and inherit); refactor it into
a table-driven test with subtests using t.Run to reduce duplication. Create a
slice of test cases describing inputs (ComponentFromArg and expected tool
values) and loop over them, calling cfg.InitCliConfig and exec.ProcessStacks
inside each case and asserting against the case's expected values (use the
existing symbols overrideInfo/inheritInfo only as templates for fields like
ComponentFromArg, and reuse compSection/compSection2 logic inside the loop).
Ensure each case has a descriptive name and keeps the same assertions for
tools["terraform"], tools["tflint"], tools["jq"], and presence/absence of
tools["checkov"] so behavior remains identical.
ℹ️ Review info
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (1)
tests/terraform_toolchain_integration_test.go
Use t.Setenv to snapshot and auto-restore PATH before exec.ExecuteTerraform, which mutates the process-wide PATH via the toolchain installer. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
🧹 Nitpick comments (1)
tests/terraform_toolchain_integration_test.go (1)
569-637: Convert precedence scenarios to a table-driven test.The two scenario blocks are nearly identical; a table-driven subtest matrix will reduce duplication and make future precedence cases easier to add.
♻️ Refactor sketch
- // --- Test component that OVERRIDES Scope 2 at Scope 3 --- - overrideInfo := schema.ConfigAndStacksInfo{...} - ... - // --- Test component that INHERITS from Scope 1+2 only --- - inheritInfo := schema.ConfigAndStacksInfo{...} - ... + cases := []struct { + name string + component string + wantTerraform string + wantTflint string + wantJQ string + wantCheckov bool + }{ + {"override", "test-component-override", "1.10.3", "^0.54.0", "latest", true}, + {"inherit", "test-component-inherit", "1.6.0", "^0.54.0", "latest", false}, + } + + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + info := schema.ConfigAndStacksInfo{ + Stack: "override-test", + ComponentType: "terraform", + ComponentFromArg: tc.component, + SubCommand: "plan", + } + atmosConfig, err := cfg.InitCliConfig(info, true) + require.NoError(t, err) + info, err = exec.ProcessStacks(&atmosConfig, info, true, false, false, nil, nil) + require.NoError(t, err) + + deps := info.ComponentSection["dependencies"].(map[string]any) + tools := deps["tools"].(map[string]any) + assert.Equal(t, tc.wantTerraform, tools["terraform"]) + assert.Equal(t, tc.wantTflint, tools["tflint"]) + assert.Equal(t, tc.wantJQ, tools["jq"]) + _, hasCheckov := tools["checkov"] + assert.Equal(t, tc.wantCheckov, hasCheckov) + }) + }As per coding guidelines: "Use table-driven tests for testing multiple scenarios in Go."
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/terraform_toolchain_integration_test.go` around lines 569 - 637, The test duplicates two very similar scenarios (overrideInfo and inheritInfo) — refactor into a table-driven test; create a slice of test cases each containing fields for ComponentFromArg and expected tools map, then iterate with t.Run per case and reuse the existing setup: call cfg.InitCliConfig with a ConfigAndStacksInfo, call exec.ProcessStacks, extract ComponentSection and dependencies/tools and assert expectations; reference the existing symbols overrideInfo/inheritInfo only to map their values into table entries, and keep calls to cfg.InitCliConfig, exec.ProcessStacks, compSection, deps/tools and the same assertions but driven by the per-case expected values.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@tests/terraform_toolchain_integration_test.go`:
- Around line 569-637: The test duplicates two very similar scenarios
(overrideInfo and inheritInfo) — refactor into a table-driven test; create a
slice of test cases each containing fields for ComponentFromArg and expected
tools map, then iterate with t.Run per case and reuse the existing setup: call
cfg.InitCliConfig with a ConfigAndStacksInfo, call exec.ProcessStacks, extract
ComponentSection and dependencies/tools and assert expectations; reference the
existing symbols overrideInfo/inheritInfo only to map their values into table
entries, and keep calls to cfg.InitCliConfig, exec.ProcessStacks, compSection,
deps/tools and the same assertions but driven by the per-case expected values.
ℹ️ Review info
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (1)
tests/terraform_toolchain_integration_test.go
|
These changes were released in v1.208.1-rc.0. |
what
dependenciesfrom global (Scope 1) and component-type (Scope 2) sectionsterraform.dependencies.tools(and helmfile/packer/ansible equivalents) now flow through to component configswhy
The bug prevented users from configuring dependencies at the component-type level (Scope 2) via mixin patterns like:
The stack processor dropped this data before it reached the toolchain resolver, so auto-install never triggered. Users reported
exec: "terraform": executable file not found in $PATHerrors when they configured Scope 2 dependencies.This fix ensures all 3 scopes of dependencies are properly extracted, merged with correct precedence, and propagated through to component sections where the resolver can access them.
references
Fixes the bug discussed in the Slack conversation where Jonathan Rose configured Scope 2 dependencies but toolchain auto-install wasn't triggered.
Changes:
errors/errors.go: Added 5 sentinel errors for dependencies validationinternal/exec/stack_processor_process_stacks_helpers.go: AddedGlobalDependenciesfield toComponentProcessorOptionsinternal/exec/stack_processor_process_stacks.go: Extract and merge dependencies from all 3 scopes, pass through opts buildersinternal/exec/stack_processor_merge.go: Updated merge chain to include global dependencies with lowest prioritytests/: Added 2 integration tests proving Scope 2 dependencies trigger toolchain auto-installSummary by CodeRabbit
New Features
Chores
Tests