feat: implement !append YAML function for list concatenation#1513
feat: implement !append YAML function for list concatenation#1513
Conversation
|
Warning Rate limit exceeded@osterman has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 5 minutes and 23 seconds before requesting another review. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ⛔ Files ignored due to path filters (2)
📒 Files selected for processing (1)
📝 WalkthroughWalkthroughAdds per-field YAML !append handling: YAML loader records tagged sequences as append-wrapped values; merge pre-processing consumes those wrappers to concatenate lists into merged maps. New helpers, tests, docs, test fixtures, a CLI test helper, and one new exported error were added. No public API signatures were changed. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant YAML as YAML Loader
participant PY as process_yaml.go
participant U as utils (append helpers)
participant V as Viper (config store)
YAML->>PY: Parse node
alt node is sequence with tag "!append"
PY->>PY: Decode each list item
PY->>U: WrapWithAppendTag(items)
U-->>PY: {"__atmos_append__": items}
PY->>V: Set(path, wrappedValue)
PY->>PY: Clear node.Tag
else other node
PY->>V: Set(path, decoded value)
end
sequenceDiagram
autonumber
participant M as MergeWithOptions
participant P as processAppendTags
participant U as utils (append helpers)
participant S as MergedState
M->>P: Pre-process input map
loop each key,value
P->>P: processValue(key, value)
alt value is append-wrapped list
P->>U: ExtractAppendListValue(value)
U-->>P: items
P->>S: Append items into S[key] (create if missing)
else value is nested map
P->>P: processAppendTags(nestedMap)
P->>S: Merge/assign nested map into S[key]
else
P->>S: Assign/override S[key] = value
end
end
M->>M: Continue normal merge flow with processed map
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (4)
tests/test-cases/append-function/override-with-append.yaml (1)
1-25: Good coverage of append on both lists of scalars and lists of maps.Consider adding an additional test case (as a separate fixture) that appends an empty list to verify no-ops are handled gracefully.
pkg/config/process_yaml.go (1)
226-249: Append handling is robust and consistent with existing error patterns.
- Decoding each child and wrapping with u.WrapWithAppendTag is clean.
- Tag clearing prevents re-processing.
Minor polish (optional): defensively trim the tag before comparison to be resilient to accidental whitespace.
- if node.Tag == u.AtmosYamlFuncAppend { + if strings.TrimSpace(node.Tag) == u.AtmosYamlFuncAppend {pkg/merge/merge.go (1)
76-133: Append pre-processing logic is clear, recursive, and non-mutating.
- processAppendList preserves order and avoids in-place mutation.
- Recursion via processNestedMap handles deep structures.
Optional: consider a configurable de-duplication step when both global append and tag-based append are active to avoid doubled base elements (keeping default as current behavior for backward compatibility).
pkg/merge/merge_append_test.go (1)
145-173: Global append interaction test is precise.The expected duplication makes the “independent strategies” behavior explicit. Consider adding one more test for a type-mismatch edge case (e.g., base has non-list and override provides append tag) to assert the error or override behavior.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (9)
pkg/config/process_yaml.go(2 hunks)pkg/merge/merge.go(2 hunks)pkg/merge/merge_append_test.go(1 hunks)pkg/utils/yaml_func_append.go(1 hunks)pkg/utils/yaml_func_append_test.go(1 hunks)pkg/utils/yaml_utils.go(1 hunks)tests/test-cases/append-function/base.yaml(1 hunks)tests/test-cases/append-function/expected-result.yaml(1 hunks)tests/test-cases/append-function/override-with-append.yaml(1 hunks)
🧰 Additional context used
📓 Path-based instructions (7)
tests/test-cases/**
📄 CodeRabbit inference engine (CLAUDE.md)
Never modify files under tests/test-cases; these are golden snapshots and must be changed only intentionally with documentation.
Files:
tests/test-cases/append-function/expected-result.yamltests/test-cases/append-function/base.yamltests/test-cases/append-function/override-with-append.yaml
pkg/**/*.go
📄 CodeRabbit inference engine (.cursor/rules/atmos-rules.mdc)
Place business logic in pkg rather than in cmd
Files:
pkg/merge/merge.gopkg/config/process_yaml.gopkg/utils/yaml_utils.gopkg/merge/merge_append_test.gopkg/utils/yaml_func_append_test.gopkg/utils/yaml_func_append.go
**/*.go
📄 CodeRabbit inference engine (.cursor/rules/atmos-rules.mdc)
**/*.go: All code must pass golangci-lint checks
Follow Go error handling idioms and use meaningful error messages
Wrap errors with context using fmt.Errorf("context: %w", err)
Consider custom error types for domain-specific errors
Follow standard Go coding style; run gofmt and goimports
Use snake_case for environment variables
Document complex logic with inline comments
**/*.go: All Go comments must end with periods (enforced by golangci-lint's godot).
Wrap all errors with static errors defined in errors/errors.go; never return dynamic errors directly; use fmt.Errorf with %w and add details after the static error.
Always bind environment variables using viper.BindEnv and provide an ATMOS_ alternative for each external env var.
Use structured logging for system/debug events; logging must not affect execution and should use appropriate levels per docs/logging.md.
Prefer SDKs over shelling out to binaries for cross-platform compatibility; use filepath/os/runtime for portable paths and OS-specific logic.
Files:
pkg/merge/merge.gopkg/config/process_yaml.gopkg/utils/yaml_utils.gopkg/merge/merge_append_test.gopkg/utils/yaml_func_append_test.gopkg/utils/yaml_func_append.go
**/!(*_test).go
📄 CodeRabbit inference engine (.cursor/rules/atmos-rules.mdc)
Document all exported functions, types, and methods with Go doc comments
Files:
pkg/merge/merge.gopkg/config/process_yaml.gopkg/utils/yaml_utils.gopkg/utils/yaml_func_append.go
pkg/config/**/*.go
📄 CodeRabbit inference engine (CLAUDE.md)
Use Viper for configuration management with config name "atmos", add config path ".", and AutomaticEnv with ATMOS prefix.
Files:
pkg/config/process_yaml.go
**/*_test.go
📄 CodeRabbit inference engine (.cursor/rules/atmos-rules.mdc)
**/*_test.go: Every new feature must include comprehensive unit tests
Test both happy paths and error conditions
Use table-driven tests for multiple scenarios
**/*_test.go: Use table-driven tests for unit tests and focus on pure functions where possible.
Always use t.Skipf with a clear reason when skipping tests; never use t.Skip.
For CLI tests requiring rebuilt binaries, set a package-level skipReason in TestMain and call os.Exit(m.Run()); individual tests must check and t.Skipf if set; never use log.Fatal for missing/stale binaries.
Mirror test file names to their implementation files (e.g., aws_ssm_store_test.go for aws_ssm_store.go) and co-locate tests with implementation.
Files:
pkg/merge/merge_append_test.gopkg/utils/yaml_func_append_test.go
pkg/**/*_test.go
📄 CodeRabbit inference engine (CLAUDE.md)
Place unit tests under pkg/** matching the package they test.
Files:
pkg/merge/merge_append_test.gopkg/utils/yaml_func_append_test.go
🧠 Learnings (6)
📚 Learning: 2025-04-24T01:40:13.576Z
Learnt from: haitham911
PR: cloudposse/atmos#1202
File: pkg/config/process_yaml.go:0-0
Timestamp: 2025-04-24T01:40:13.576Z
Learning: When processing YAML nodes with custom directives in Go using gopkg.in/yaml.v3, setting node.Tag = "" is sufficient to prevent re-processing of the node. It's not necessary to also clear node.Value after updating the configuration store (e.g., Viper), as the original value doesn't affect subsequent operations once the tag is removed.
Applied to files:
pkg/config/process_yaml.go
📚 Learning: 2025-04-24T01:06:15.259Z
Learnt from: haitham911
PR: cloudposse/atmos#1202
File: pkg/config/process_yaml.go:0-0
Timestamp: 2025-04-24T01:06:15.259Z
Learning: When processing YAML structures with gopkg.in/yaml.v3, the Value field of a Node is only meant for scalar values. Complex data types like maps and arrays are represented through child nodes in the YAML tree structure.
Applied to files:
pkg/config/process_yaml.go
📚 Learning: 2025-01-19T22:30:27.600Z
Learnt from: aknysh
PR: cloudposse/atmos#0
File: :0-0
Timestamp: 2025-01-19T22:30:27.600Z
Learning: The Atmos YAML function `!include` allows downloading local or remote files from different sources and assigning their contents to sections in stack manifests. It supports various protocols (file, http, git, s3, etc.) and can filter content using YQ expressions.
Applied to files:
pkg/utils/yaml_utils.go
📚 Learning: 2025-09-23T02:30:42.362Z
Learnt from: CR
PR: cloudposse/atmos#0
File: .cursor/rules/atmos-rules.mdc:0-0
Timestamp: 2025-09-23T02:30:42.362Z
Learning: Applies to **/*_test.go : Every new feature must include comprehensive unit tests
Applied to files:
pkg/utils/yaml_func_append_test.go
📚 Learning: 2025-09-25T03:30:16.196Z
Learnt from: CR
PR: cloudposse/atmos#0
File: CLAUDE.md:0-0
Timestamp: 2025-09-25T03:30:16.196Z
Learning: Applies to **/*_test.go : Use table-driven tests for unit tests and focus on pure functions where possible.
Applied to files:
pkg/utils/yaml_func_append_test.go
📚 Learning: 2025-09-25T03:30:16.196Z
Learnt from: CR
PR: cloudposse/atmos#0
File: CLAUDE.md:0-0
Timestamp: 2025-09-25T03:30:16.196Z
Learning: Applies to pkg/**/*_test.go : Place unit tests under pkg/** matching the package they test.
Applied to files:
pkg/utils/yaml_func_append_test.go
🧬 Code graph analysis (5)
pkg/merge/merge.go (1)
pkg/utils/yaml_func_append.go (1)
ExtractAppendListValue(39-48)
pkg/config/process_yaml.go (2)
pkg/utils/yaml_utils.go (1)
AtmosYamlFuncAppend(27-27)pkg/utils/yaml_func_append.go (1)
WrapWithAppendTag(52-56)
pkg/merge/merge_append_test.go (3)
pkg/schema/schema.go (3)
AtmosConfiguration(26-61)Settings(677-681)AtmosSettings(247-267)pkg/merge/merge.go (3)
ListMergeStrategyReplace(14-14)Merge(135-171)ListMergeStrategyAppend(15-15)pkg/utils/yaml_func_append.go (1)
WrapWithAppendTag(52-56)
pkg/utils/yaml_func_append_test.go (1)
pkg/utils/yaml_func_append.go (5)
IsAppendTag(22-24)HasAppendTag(28-36)ExtractAppendListValue(39-48)WrapWithAppendTag(52-56)ProcessAppendTag(12-19)
pkg/utils/yaml_func_append.go (1)
pkg/utils/yaml_utils.go (1)
AtmosYamlFuncAppend(27-27)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
- GitHub Check: Build (ubuntu-latest, linux)
- GitHub Check: Build (windows-latest, windows)
- GitHub Check: Build (macos-latest, macos)
- GitHub Check: Analyze (go)
- GitHub Check: Lint (golangci)
- GitHub Check: Run pre-commit hooks
- GitHub Check: Summary
🔇 Additional comments (7)
pkg/utils/yaml_utils.go (1)
27-27: Constant for !append tag added — looks good.The new AtmosYamlFuncAppend constant is concise and aligns with the other tag constants.
tests/test-cases/append-function/base.yaml (1)
1-28: Fixture baseline reads clearly and mirrors real usage.Solid base for demonstrating per-field list concatenation.
pkg/config/process_yaml.go (1)
72-77: Correctly handles tagged sequence nodes.Nice addition to process sequence nodes before walking children. Ensure downstream consumers of Viper keys tolerate bracketed index paths (e.g., key[0]) since those paths are now set for children of sequences.
Would you confirm that no code relies on dot-only key paths and that bracketed paths won’t be read back accidentally as top-level keys?
tests/test-cases/append-function/expected-result.yaml (1)
1-39: Expected result matches the intended semantics.This accurately reflects base + append outcomes for both depends_on and node_groups.
pkg/merge/merge.go (1)
35-37: Pre-merge append processing placed at the right stage.Running processAppendTags before YAML round-trip avoids pointer aliasing issues and preserves intent for mergo options.
pkg/merge/merge_append_test.go (1)
12-143: Comprehensive, table-driven tests for core scenarios.Covers: append to existing, append when missing, replacement without tag, and mixed keys.
pkg/utils/yaml_func_append_test.go (1)
9-176: Good unit coverage on tag detection, extraction, wrapping, and no-op processing.Covers happy paths and negatives with table-driven style using testify. Nicely done.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1513 +/- ##
==========================================
+ Coverage 72.81% 72.84% +0.02%
==========================================
Files 530 531 +1
Lines 50667 50715 +48
==========================================
+ Hits 36892 36941 +49
+ Misses 11002 11000 -2
- Partials 2773 2774 +1
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (3)
pkg/utils/yaml_func_append.go(1 hunks)website/docs/functions/yaml/append.mdx(1 hunks)website/docs/functions/yaml/index.mdx(2 hunks)
🧰 Additional context used
📓 Path-based instructions (4)
website/**
📄 CodeRabbit inference engine (.cursor/rules/atmos-rules.mdc)
website/**: Update website documentation in website/ when adding features
Ensure consistency between CLI help text and website documentation
Follow the website's documentation structure and style
Keep website code in website/ and follow its architecture/style; test changes locally
Keep CLI and website documentation in sync; document new features with examples and use cases
Files:
website/docs/functions/yaml/index.mdxwebsite/docs/functions/yaml/append.mdx
pkg/**/*.go
📄 CodeRabbit inference engine (.cursor/rules/atmos-rules.mdc)
Place business logic in pkg rather than in cmd
Files:
pkg/utils/yaml_func_append.go
**/*.go
📄 CodeRabbit inference engine (.cursor/rules/atmos-rules.mdc)
**/*.go: All code must pass golangci-lint checks
Follow Go error handling idioms and use meaningful error messages
Wrap errors with context using fmt.Errorf("context: %w", err)
Consider custom error types for domain-specific errors
Follow standard Go coding style; run gofmt and goimports
Use snake_case for environment variables
Document complex logic with inline comments
**/*.go: All Go comments must end with periods (enforced by golangci-lint's godot).
Wrap all errors with static errors defined in errors/errors.go; never return dynamic errors directly; use fmt.Errorf with %w and add details after the static error.
Always bind environment variables using viper.BindEnv and provide an ATMOS_ alternative for each external env var.
Use structured logging for system/debug events; logging must not affect execution and should use appropriate levels per docs/logging.md.
Prefer SDKs over shelling out to binaries for cross-platform compatibility; use filepath/os/runtime for portable paths and OS-specific logic.
Files:
pkg/utils/yaml_func_append.go
**/!(*_test).go
📄 CodeRabbit inference engine (.cursor/rules/atmos-rules.mdc)
Document all exported functions, types, and methods with Go doc comments
Files:
pkg/utils/yaml_func_append.go
🧠 Learnings (6)
📚 Learning: 2024-12-01T00:33:20.298Z
Learnt from: aknysh
PR: cloudposse/atmos#810
File: examples/tests/stacks/catalog/terraform/template-functions-test2/defaults.yaml:28-32
Timestamp: 2024-12-01T00:33:20.298Z
Learning: In `examples/tests/stacks/catalog/terraform/template-functions-test2/defaults.yaml`, `!exec atmos terraform output` is used in examples to demonstrate its usage, even though `!terraform.output` is the recommended approach according to the documentation.
Applied to files:
website/docs/functions/yaml/index.mdx
📚 Learning: 2025-01-19T22:30:27.600Z
Learnt from: aknysh
PR: cloudposse/atmos#0
File: :0-0
Timestamp: 2025-01-19T22:30:27.600Z
Learning: The Atmos YAML function `!env` is used to retrieve environment variables and assign them to sections in stack manifests. It supports both simple types (string, number, boolean) and complex types (JSON-encoded lists, maps, objects).
Applied to files:
website/docs/functions/yaml/index.mdx
📚 Learning: 2025-01-19T22:30:27.600Z
Learnt from: aknysh
PR: cloudposse/atmos#0
File: :0-0
Timestamp: 2025-01-19T22:30:27.600Z
Learning: The Atmos YAML function `!include` allows downloading local or remote files from different sources and assigning their contents to sections in stack manifests. It supports various protocols (file, http, git, s3, etc.) and can filter content using YQ expressions.
Applied to files:
website/docs/functions/yaml/index.mdx
📚 Learning: 2024-12-03T04:01:16.446Z
Learnt from: aknysh
PR: cloudposse/atmos#810
File: website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx:0-0
Timestamp: 2024-12-03T04:01:16.446Z
Learning: In the `terraform.output.mdx` documentation file (`website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx`), the cache invalidation and cache scope behavior for the `!terraform.output` function are already described.
Applied to files:
website/docs/functions/yaml/index.mdx
📚 Learning: 2025-09-25T03:30:16.196Z
Learnt from: CR
PR: cloudposse/atmos#0
File: CLAUDE.md:0-0
Timestamp: 2025-09-25T03:30:16.196Z
Learning: Applies to **/*.go : All Go comments must end with periods (enforced by golangci-lint's godot).
Applied to files:
pkg/utils/yaml_func_append.go
📚 Learning: 2025-09-23T02:30:42.362Z
Learnt from: CR
PR: cloudposse/atmos#0
File: .cursor/rules/atmos-rules.mdc:0-0
Timestamp: 2025-09-23T02:30:42.362Z
Learning: Applies to **/*.go : Document complex logic with inline comments
Applied to files:
pkg/utils/yaml_func_append.go
🧬 Code graph analysis (1)
pkg/utils/yaml_func_append.go (1)
pkg/utils/yaml_utils.go (1)
AtmosYamlFuncAppend(27-27)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)
- GitHub Check: Build (macos-latest, macos)
- GitHub Check: Build (ubuntu-latest, linux)
- GitHub Check: Build (windows-latest, windows)
- GitHub Check: Run pre-commit hooks
- GitHub Check: Analyze (go)
- GitHub Check: website-deploy-preview
- GitHub Check: Lint (golangci)
- GitHub Check: Summary
🔇 Additional comments (2)
website/docs/functions/yaml/append.mdx (1)
12-52: Docs nail the append narrative.The walkthrough and examples read clean and make the per-field behavior obvious.
website/docs/functions/yaml/index.mdx (1)
46-59: Nice touch integrating the new tag.The blurb slots right in and points folks to the deeper doc.
b3d174c to
01382a3
Compare
|
💥 This pull request now has conflicts. Could you fix it @osterman? 🙏 |
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
pkg/merge/merge.go (2)
139-142: Fix invalid use of multiple %w in fmt.Errorf.fmt.Errorf only supports a single %w. Using two is invalid and breaks error wrapping.
Apply:
- return nil, fmt.Errorf("%w: %w", errUtils.ErrMerge, errUtils.ErrAtmosConfigIsNil) + return nil, fmt.Errorf("%w: %v", errUtils.ErrMerge, errUtils.ErrAtmosConfigIsNil)As per coding guidelines.
179-186: Fix invalid use of multiple %w in fmt.Errorf (again).Same issue here.
Apply:
- if atmosConfig == nil { - err := fmt.Errorf("%w: %w", errUtils.ErrMerge, errUtils.ErrAtmosConfigIsNil) + if atmosConfig == nil { + err := fmt.Errorf("%w: %v", errUtils.ErrMerge, errUtils.ErrAtmosConfigIsNil)As per coding guidelines.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (13)
errors/errors.go(1 hunks)pkg/config/process_yaml.go(2 hunks)pkg/merge/merge.go(2 hunks)pkg/merge/merge_append_test.go(1 hunks)pkg/utils/yaml_func_append.go(1 hunks)pkg/utils/yaml_func_append_test.go(1 hunks)pkg/utils/yaml_utils.go(1 hunks)tests/cli_test.go(2 hunks)tests/test-cases/append-function/base.yaml(1 hunks)tests/test-cases/append-function/expected-result.yaml(1 hunks)tests/test-cases/append-function/override-with-append.yaml(1 hunks)website/docs/functions/yaml/append.mdx(1 hunks)website/docs/functions/yaml/index.mdx(2 hunks)
🚧 Files skipped from review as they are similar to previous changes (6)
- pkg/utils/yaml_utils.go
- pkg/utils/yaml_func_append_test.go
- pkg/config/process_yaml.go
- pkg/merge/merge_append_test.go
- pkg/utils/yaml_func_append.go
- website/docs/functions/yaml/append.mdx
🧰 Additional context used
📓 Path-based instructions (9)
tests/**
📄 CodeRabbit inference engine (CLAUDE.md)
Place integration tests and shared test utilities under tests/ with fixtures in tests/test-cases/.
Files:
tests/test-cases/append-function/base.yamltests/cli_test.gotests/test-cases/append-function/expected-result.yamltests/test-cases/append-function/override-with-append.yaml
{tests/test-cases/**,tests/testdata/**}
📄 CodeRabbit inference engine (CLAUDE.md)
Never modify golden snapshot directories tests/test-cases/ or tests/testdata/ unless explicitly instructed.
Files:
tests/test-cases/append-function/base.yamltests/test-cases/append-function/expected-result.yamltests/test-cases/append-function/override-with-append.yaml
**/*.go
📄 CodeRabbit inference engine (.cursor/rules/atmos-rules.mdc)
**/*.go: All code must pass golangci-lint checks
Follow Go error handling idioms and use meaningful error messages
Wrap errors with context using fmt.Errorf("context: %w", err)
Consider custom error types for domain-specific errors
Follow standard Go coding style; run gofmt and goimports
Use snake_case for environment variables
Document complex logic with inline comments
**/*.go: All comments must end with periods.
Wrap all returned errors with static errors from errors/errors.go; never return dynamic errors directly.
Use fmt.Errorf with %w to wrap the static error first, then add context details.
Always bind environment variables via viper.BindEnv, providing an ATMOS_ alternative for each external var.
All new configurations must support Go templating using the shared FuncMap(); test template rendering with various contexts.
Prefer SDKs over shelling out to binaries; use standard library for cross-platform behavior (filepath.Join, os.PathSeparator, runtime.GOOS).
For non-standard execution paths, capture telemetry via telemetry.CaptureCmd or telemetry.CaptureCmdString without user data.
Files:
errors/errors.gopkg/merge/merge.gotests/cli_test.go
**/!(*_test).go
📄 CodeRabbit inference engine (.cursor/rules/atmos-rules.mdc)
Document all exported functions, types, and methods with Go doc comments
Files:
errors/errors.gopkg/merge/merge.go
errors/errors.go
📄 CodeRabbit inference engine (CLAUDE.md)
Define static errors in errors/errors.go (e.g., ErrInvalidComponent, ErrInvalidStack, ErrInvalidConfig).
Files:
errors/errors.go
pkg/**/*.go
📄 CodeRabbit inference engine (.cursor/rules/atmos-rules.mdc)
Place business logic in pkg rather than in cmd
Target >80% coverage for packages, focusing on pkg/ and internal/exec/.
Files:
pkg/merge/merge.go
**/*_test.go
📄 CodeRabbit inference engine (.cursor/rules/atmos-rules.mdc)
**/*_test.go: Every new feature must include comprehensive unit tests
Test both happy paths and error conditions
Use table-driven tests for multiple scenarios
**/*_test.go: Use table-driven tests for unit tests where applicable.
Always use t.Skipf() with a reason; do not use t.Skip() or t.Skipf() without explanation.
TestMain must call os.Exit(m.Run()) to propagate test exit code for CLI tests.
Files:
tests/cli_test.go
tests/**/*_test.go
📄 CodeRabbit inference engine (CLAUDE.md)
Use precondition-based skipping helpers from tests/test_preconditions.go for external dependencies (AWS, GitHub, network).
Files:
tests/cli_test.go
website/**
📄 CodeRabbit inference engine (.cursor/rules/atmos-rules.mdc)
website/**: Update website documentation in website/ when adding features
Ensure consistency between CLI help text and website documentation
Follow the website's documentation structure and style
Keep website code in website/ and follow its architecture/style; test changes locally
Keep CLI and website documentation in sync; document new features with examples and use cases
Files:
website/docs/functions/yaml/index.mdx
🧠 Learnings (13)
📚 Learning: 2024-10-23T21:36:40.262Z
Learnt from: osterman
PR: cloudposse/atmos#740
File: cmd/cmd_utils.go:340-359
Timestamp: 2024-10-23T21:36:40.262Z
Learning: In the Go codebase for Atmos, when reviewing functions like `checkAtmosConfig` in `cmd/cmd_utils.go`, avoid suggesting refactoring to return errors instead of calling `os.Exit` if such changes would significantly increase the scope due to the need to update multiple call sites.
Applied to files:
tests/cli_test.go
📚 Learning: 2025-09-08T01:25:44.958Z
Learnt from: osterman
PR: cloudposse/atmos#1466
File: website/docs/cli/commands/toolchain/usage.mdx:117-121
Timestamp: 2025-09-08T01:25:44.958Z
Learning: The atmos toolchain XDG compliance implementation is complete with GetXDGCacheDir() and GetXDGTempCacheDir() functions in toolchain/xdg_cache.go, updated installer.go and toolchain_clean.go to use these helpers, and changed cache paths from ~/.cache/tools-cache to ${XDG_CACHE_HOME}/atmos-toolchain (or ~/.cache/atmos-toolchain when XDG_CACHE_HOME is not set).
Applied to files:
tests/cli_test.go
📚 Learning: 2025-05-23T19:51:47.091Z
Learnt from: samtholiya
PR: cloudposse/atmos#1255
File: cmd/describe_affected_test.go:15-15
Timestamp: 2025-05-23T19:51:47.091Z
Learning: The atmos codebase has a custom extension to *testing.T that provides a Chdir method, allowing test functions to call t.Chdir() to change working directories during tests. This is used consistently across test files in the codebase.
Applied to files:
tests/cli_test.go
📚 Learning: 2025-09-08T01:25:44.958Z
Learnt from: osterman
PR: cloudposse/atmos#1466
File: website/docs/cli/commands/toolchain/usage.mdx:117-121
Timestamp: 2025-09-08T01:25:44.958Z
Learning: The atmos toolchain has been updated to follow XDG Base Directory Specification with helper functions GetXDGCacheDir() and GetXDGTempCacheDir() in toolchain/xdg_cache.go, using XDG_CACHE_HOME when set and falling back to ~/.cache/atmos-toolchain, making it consistent with atmos core's XDG compliance.
Applied to files:
tests/cli_test.go
📚 Learning: 2025-09-08T01:25:44.958Z
Learnt from: osterman
PR: cloudposse/atmos#1466
File: website/docs/cli/commands/toolchain/usage.mdx:117-121
Timestamp: 2025-09-08T01:25:44.958Z
Learning: XDG Base Directory Specification compliance implementation for atmos toolchain is complete: created toolchain/xdg_cache.go with GetXDGCacheDir() and GetXDGTempCacheDir() functions, updated toolchain/installer.go and cmd/toolchain_clean.go to use these XDG helpers, and changed all cache paths from hardcoded ~/.cache/tools-cache to XDG-compliant ${XDG_CACHE_HOME}/atmos-toolchain (or ~/.cache/atmos-toolchain fallback).
Applied to files:
tests/cli_test.go
📚 Learning: 2025-09-26T05:52:03.918Z
Learnt from: CR
PR: cloudposse/atmos#0
File: CLAUDE.md:0-0
Timestamp: 2025-09-26T05:52:03.918Z
Learning: Applies to tests/**/*_test.go : Use precondition-based skipping helpers from tests/test_preconditions.go for external dependencies (AWS, GitHub, network).
Applied to files:
tests/cli_test.go
📚 Learning: 2025-09-08T01:25:44.958Z
Learnt from: osterman
PR: cloudposse/atmos#1466
File: website/docs/cli/commands/toolchain/usage.mdx:117-121
Timestamp: 2025-09-08T01:25:44.958Z
Learning: Final XDG Base Directory Specification implementation for atmos toolchain is complete and verified: toolchain/xdg_cache.go provides GetXDGCacheDir() and GetXDGTempCacheDir() functions, all hardcoded ~/.cache/tools-cache paths have been replaced with XDG-compliant paths using ${XDG_CACHE_HOME}/atmos-toolchain (or ~/.cache/atmos-toolchain fallback), and tests have been updated to expect the new path structure.
Applied to files:
tests/cli_test.go
📚 Learning: 2025-09-26T05:52:03.918Z
Learnt from: CR
PR: cloudposse/atmos#0
File: CLAUDE.md:0-0
Timestamp: 2025-09-26T05:52:03.918Z
Learning: Applies to tests/test_preconditions.go : Centralize test precondition helpers in tests/test_preconditions.go.
Applied to files:
tests/cli_test.go
📚 Learning: 2024-12-02T21:26:32.337Z
Learnt from: osterman
PR: cloudposse/atmos#808
File: pkg/config/config.go:478-483
Timestamp: 2024-12-02T21:26:32.337Z
Learning: In the 'atmos' project, when reviewing Go code like `pkg/config/config.go`, avoid suggesting file size checks after downloading remote configs if such checks aren't implemented elsewhere in the codebase.
Applied to files:
tests/cli_test.go
📚 Learning: 2024-12-01T00:33:20.298Z
Learnt from: aknysh
PR: cloudposse/atmos#810
File: examples/tests/stacks/catalog/terraform/template-functions-test2/defaults.yaml:28-32
Timestamp: 2024-12-01T00:33:20.298Z
Learning: In `examples/tests/stacks/catalog/terraform/template-functions-test2/defaults.yaml`, `!exec atmos terraform output` is used in examples to demonstrate its usage, even though `!terraform.output` is the recommended approach according to the documentation.
Applied to files:
website/docs/functions/yaml/index.mdx
📚 Learning: 2025-01-19T22:30:27.600Z
Learnt from: aknysh
PR: cloudposse/atmos#0
File: :0-0
Timestamp: 2025-01-19T22:30:27.600Z
Learning: The Atmos YAML function `!env` is used to retrieve environment variables and assign them to sections in stack manifests. It supports both simple types (string, number, boolean) and complex types (JSON-encoded lists, maps, objects).
Applied to files:
website/docs/functions/yaml/index.mdx
📚 Learning: 2025-01-19T22:30:27.600Z
Learnt from: aknysh
PR: cloudposse/atmos#0
File: :0-0
Timestamp: 2025-01-19T22:30:27.600Z
Learning: The Atmos YAML function `!include` allows downloading local or remote files from different sources and assigning their contents to sections in stack manifests. It supports various protocols (file, http, git, s3, etc.) and can filter content using YQ expressions.
Applied to files:
website/docs/functions/yaml/index.mdx
📚 Learning: 2024-12-03T04:01:16.446Z
Learnt from: aknysh
PR: cloudposse/atmos#810
File: website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx:0-0
Timestamp: 2024-12-03T04:01:16.446Z
Learning: In the `terraform.output.mdx` documentation file (`website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx`), the cache invalidation and cache scope behavior for the `!terraform.output` function are already described.
Applied to files:
website/docs/functions/yaml/index.mdx
🧬 Code graph analysis (2)
pkg/merge/merge.go (1)
pkg/utils/yaml_func_append.go (1)
ExtractAppendListValue(39-48)
tests/cli_test.go (1)
pkg/logger/log.go (1)
Info(34-36)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (10)
- GitHub Check: Analyze (go)
- GitHub Check: Lint (golangci)
- GitHub Check: Analyze (javascript-typescript)
- GitHub Check: Build (macos-latest, macos)
- GitHub Check: Build (windows-latest, windows)
- GitHub Check: Run pre-commit hooks
- GitHub Check: website-deploy-preview
- GitHub Check: Analyze (go)
- GitHub Check: Lint (golangci)
- GitHub Check: Summary
🔇 Additional comments (2)
tests/test-cases/append-function/override-with-append.yaml (1)
1-25: LGTM.Good coverage of nested append scenarios (eks node_groups, settings.depends_on, and rds depends_on). This aligns with the intended per-field opt-in behavior.
pkg/merge/merge.go (1)
134-171: Public signature unchanged vs summary.The AI summary mentions a public signature change dropping the inputs parameter from Merge, but this file still requires inputs []map[string]any. Confirm intended surface.
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (3)
tests/cli_test.go (1)
518-518: Fix skipReason shadowing so tests actually skip.Line 518 uses
:=which shadows the package-levelskipReason, preventing tests from skipping when they should. Use=with a local declaration forbinaryPath.Apply this diff:
// Check for the atmos binary - binaryPath, skipReason := validateAtmosBinary(repoRoot) + var binaryPath string + binaryPath, skipReason = validateAtmosBinary(repoRoot)pkg/merge/merge.go (2)
39-40: Prevent double-append under append list merge strategy.When
appendSlice=true(global append strategy),processAppendListexpands the list to include existing items, thenmergo.WithAppendSliceappends again, causing duplication. Pass theappendSliceflag through and return only new items when true.Also, inline comments at lines 94, 99, 104, and 117 are missing periods per coding guidelines.
Apply this diff:
- // Process !append tagged lists before merging - current = processAppendTags(current, merged) + // Process !append tagged lists before merging. + current = processAppendTags(current, merged, appendSlice) @@ -// processAppendTags handles special !append tagged lists during merging. -// It processes any values wrapped with __atmos_append__ metadata and appends them to existing lists. -func processAppendTags(current map[string]any, merged map[string]any) map[string]any { +// processAppendTags handles special !append tagged lists during merging. +// It processes any values wrapped with __atmos_append__ metadata and appends them to existing lists. +// When appendNewOnly is true (global append strategy), it returns only the new items so that mergo.WithAppendSlice performs the append without duplication. +func processAppendTags(current map[string]any, merged map[string]any, appendNewOnly bool) map[string]any { result := make(map[string]any) for key, value := range current { - result[key] = processValue(key, value, merged) + result[key] = processValue(key, value, merged, appendNewOnly) } return result } // processValue processes a single value for append tags. -func processValue(key string, value any, merged map[string]any) any { - // Check if this is an append-tagged list +func processValue(key string, value any, merged map[string]any, appendNewOnly bool) any { + // Check if this is an append-tagged list. if list, isAppend := u.ExtractAppendListValue(value); isAppend { - return processAppendList(key, list, merged) + return processAppendList(key, list, merged, appendNewOnly) } - // Check if this is a nested map + // Check if this is a nested map. if nestedMap, ok := value.(map[string]any); ok { - return processNestedMap(key, nestedMap, merged) + return processNestedMap(key, nestedMap, merged, appendNewOnly) } - // Regular value, pass through + // Regular value, pass through. return value } -// processAppendList handles appending a list to existing values. -func processAppendList(key string, list []any, merged map[string]any) []any { +// processAppendList handles appending a list to existing values. +// If appendNewOnly is true, return only the new items so mergo.WithAppendSlice can append them to the existing list without duplication. +func processAppendList(key string, list []any, merged map[string]any, appendNewOnly bool) []any { + if appendNewOnly { + return list + } var existingList []any if existingValue, exists := merged[key]; exists { if el, ok := existingValue.([]any); ok { existingList = el } } - // Create a new slice to avoid modifying the original + // Create a new slice to avoid modifying the original. result := make([]any, len(existingList), len(existingList)+len(list)) copy(result, existingList) result = append(result, list...) return result }Based on coding guidelines.
Also applies to: 80-122
124-136: Update processNestedMap to propagate appendNewOnly flag.The recursive call needs to pass through the
appendNewOnlyflag for consistency with the rest of the append tag processing chain.Apply this diff:
// processNestedMap recursively processes nested maps for append tags. -func processNestedMap(key string, nestedMap map[string]any, merged map[string]any) map[string]any { +func processNestedMap(key string, nestedMap map[string]any, merged map[string]any, appendNewOnly bool) map[string]any { var mergedNested map[string]any if existingNested, exists := merged[key]; exists { if mn, ok := existingNested.(map[string]any); ok { mergedNested = mn } } if mergedNested == nil { mergedNested = make(map[string]any) } - return processAppendTags(nestedMap, mergedNested) + return processAppendTags(nestedMap, mergedNested, appendNewOnly) }
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (5)
errors/errors.go(1 hunks)pkg/config/process_yaml.go(2 hunks)pkg/merge/merge.go(2 hunks)pkg/utils/yaml_utils.go(1 hunks)tests/cli_test.go(2 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
- pkg/utils/yaml_utils.go
- errors/errors.go
🧰 Additional context used
📓 Path-based instructions (8)
pkg/**/*.go
📄 CodeRabbit inference engine (.cursor/rules/atmos-rules.mdc)
Place business logic in pkg rather than in cmd
Files:
pkg/config/process_yaml.gopkg/merge/merge.go
**/*.go
📄 CodeRabbit inference engine (.cursor/rules/atmos-rules.mdc)
**/*.go: All code must pass golangci-lint checks
Follow Go error handling idioms and use meaningful error messages
Wrap errors with context using fmt.Errorf("context: %w", err)
Consider custom error types for domain-specific errors
Follow standard Go coding style; run gofmt and goimports
Use snake_case for environment variables
Document complex logic with inline comments
**/*.go: All Go comments must end with periods. Enforced by golangci-lint's godot.
Organize imports into three groups (stdlib, third-party, Atmos) separated by blank lines; sort alphabetically within each group; maintain existing aliases.
Add defer perf.Track(...) to all public functions and critical private functions, with a blank line after the call and package-prefixed name (e.g., "exec.ProcessComponent"). Use atmosConfig if available, otherwise nil.
Wrap all errors using static errors defined in errors/errors.go; never return dynamic errors directly.
Use the error wrapping pattern: fmt.Errorf("%w: details", errUtils.ErrStaticError, ...) with %w to preserve error chains and add context after the static error.
Bind environment variables with viper.BindEnv() and provide ATMOS_ alternatives for every env var (e.g., viper.BindEnv("ATMOS_GITHUB_TOKEN", "GITHUB_TOKEN")).
Ensure cross-platform compatibility: prefer SDKs over invoking binaries, use filepath.Join, os.PathSeparator/PathListSeparator, and runtime.GOOS when necessary.
Files:
pkg/config/process_yaml.gotests/cli_test.gopkg/merge/merge.go
**/!(*_test).go
📄 CodeRabbit inference engine (.cursor/rules/atmos-rules.mdc)
Document all exported functions, types, and methods with Go doc comments
Files:
pkg/config/process_yaml.gopkg/merge/merge.go
{cmd,internal,pkg}/**/*.go
📄 CodeRabbit inference engine (CLAUDE.md)
{cmd,internal,pkg}/**/*.go: Do not use logging for UI output. Send prompts/status/progress and actionable errors to stderr; send data/results to stdout; keep logging for system/debug with structured fields.
Most text UI must go to stderr; only data/results go to stdout. Prefer utils.PrintfMessageToTUI for UI messages.
For non-standard execution paths, capture telemetry using telemetry.CaptureCmd(...) or telemetry.CaptureCmdString(...); never capture user data.
Files:
pkg/config/process_yaml.gopkg/merge/merge.go
{internal/exec,pkg}/**/*.go
📄 CodeRabbit inference engine (CLAUDE.md)
All new configs must support Go templating using FuncMap() from internal/exec/template_funcs.go.
Files:
pkg/config/process_yaml.gopkg/merge/merge.go
**/*_test.go
📄 CodeRabbit inference engine (.cursor/rules/atmos-rules.mdc)
**/*_test.go: Every new feature must include comprehensive unit tests
Test both happy paths and error conditions
Use table-driven tests for multiple scenarios
**/*_test.go: Use table-driven tests for unit tests.
Always use t.Skipf(...) with a clear reason when skipping tests; do not use t.Skip().
TestMain must call os.Exit(m.Run()) to propagate the test exit code for CLI tests.
Co-locate test files with implementation (use _test.go alongside .go files) and maintain naming symmetry with implementation files.
Files:
tests/cli_test.go
tests/**/*_test.go
📄 CodeRabbit inference engine (CLAUDE.md)
Place integration tests under tests/ with fixtures in tests/test-cases/.
Files:
tests/cli_test.go
tests/**
📄 CodeRabbit inference engine (CLAUDE.md)
Place shared test utilities for integration tests in the tests/ directory.
Files:
tests/cli_test.go
🧠 Learnings (2)
📚 Learning: 2024-10-23T21:36:40.262Z
Learnt from: osterman
PR: cloudposse/atmos#740
File: cmd/cmd_utils.go:340-359
Timestamp: 2024-10-23T21:36:40.262Z
Learning: In the Go codebase for Atmos, when reviewing functions like `checkAtmosConfig` in `cmd/cmd_utils.go`, avoid suggesting refactoring to return errors instead of calling `os.Exit` if such changes would significantly increase the scope due to the need to update multiple call sites.
Applied to files:
tests/cli_test.go
📚 Learning: 2025-10-02T20:13:55.579Z
Learnt from: CR
PR: cloudposse/atmos#0
File: CLAUDE.md:0-0
Timestamp: 2025-10-02T20:13:55.579Z
Learning: Applies to **/*_test.go : Always use t.Skipf(...) with a clear reason when skipping tests; do not use t.Skip().
Applied to files:
tests/cli_test.go
🧬 Code graph analysis (3)
pkg/config/process_yaml.go (3)
pkg/utils/yaml_utils.go (1)
AtmosYamlFuncAppend(28-28)pkg/logger/log.go (1)
Debug(24-26)pkg/utils/yaml_func_append.go (1)
WrapWithAppendTag(52-56)
tests/cli_test.go (1)
pkg/logger/log.go (1)
Info(34-36)
pkg/merge/merge.go (1)
pkg/utils/yaml_func_append.go (1)
ExtractAppendListValue(39-48)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
- GitHub Check: Build (windows-latest, windows)
- GitHub Check: Build (macos-latest, macos)
- GitHub Check: Analyze (go)
- GitHub Check: Lint (golangci)
- GitHub Check: Run pre-commit hooks
- GitHub Check: website-deploy-preview
- GitHub Check: Summary
🔇 Additional comments (3)
pkg/config/process_yaml.go (2)
216-222: LGTM!Clean delegation pattern for handling tagged sequence nodes. The switch-like structure makes it easy to add more tag handlers in the future.
224-250: LGTM!The implementation correctly decodes list items, wraps them with append metadata, and stores the result in Viper. Error handling properly wraps errors with context, and the tag is cleared to prevent re-processing.
tests/cli_test.go (1)
484-498: LGTM!The helper provides clean binary validation logic with appropriate skip reasons for test flow control.
pkg/config/process_yaml.go
Outdated
| } | ||
| } | ||
|
|
||
| // Handle sequence nodes with tags (like !append) |
There was a problem hiding this comment.
Add period to comment.
Per coding guidelines, all Go comments must end with periods.
Apply this diff:
- // Handle sequence nodes with tags (like !append)
+ // Handle sequence nodes with tags (like !append).Based on coding guidelines.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| // Handle sequence nodes with tags (like !append) | |
| // Handle sequence nodes with tags (like !append). |
🤖 Prompt for AI Agents
In pkg/config/process_yaml.go around line 73, the end-of-line comment "// Handle
sequence nodes with tags (like !append)" is missing a terminating period; update
the comment to end with a period (e.g., "// Handle sequence nodes with tags
(like !append).") to comply with Go comment guidelines.
There was a problem hiding this comment.
This code was removed in a subsequent merge with main. The sequence node handling for !append is no longer in process_yaml.go - it is now handled in pkg/merge/merge.go during the merge phase.
There was a problem hiding this comment.
@osterman, thank you for the clarification! Since the code has been removed in the merge and the !append handling now lives in pkg/merge/merge.go, my comment is no longer applicable. I appreciate you letting me know about the refactoring.
🧠 Learnings used
Learnt from: haitham911
Repo: cloudposse/atmos PR: 1202
File: pkg/config/process_yaml.go:0-0
Timestamp: 2025-04-24T01:40:13.576Z
Learning: When processing YAML nodes with custom directives in Go using gopkg.in/yaml.v3, setting node.Tag = "" is sufficient to prevent re-processing of the node. It's not necessary to also clear node.Value after updating the configuration store (e.g., Viper), as the original value doesn't affect subsequent operations once the tag is removed.
Learnt from: haitham911
Repo: cloudposse/atmos PR: 1202
File: pkg/config/process_yaml.go:0-0
Timestamp: 2025-04-24T01:06:15.259Z
Learning: When processing YAML structures with gopkg.in/yaml.v3, the Value field of a Node is only meant for scalar values. Complex data types like maps and arrays are represented through child nodes in the YAML tree structure.
Learnt from: CR
Repo: cloudposse/atmos PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-12-16T18:20:55.614Z
Learning: Applies to **/*.go : All comments must end with periods (enforced by godot linter)
Learnt from: CR
Repo: cloudposse/atmos PR: 0
File: .cursor/rules/atmos-rules.mdc:0-0
Timestamp: 2025-11-24T17:35:37.209Z
Learning: Applies to **/*.go : Document complex logic with inline comments in Go code
Learnt from: Cerebrovinny
Repo: cloudposse/atmos PR: 1560
File: pkg/utils/string_utils.go:43-64
Timestamp: 2025-09-30T19:03:50.738Z
Learning: In the Atmos codebase, YAML tags like !terraform.output rely on positional arguments, so the SplitStringByDelimiter function in pkg/utils/string_utils.go must preserve empty strings (even after trimming quotes) to maintain the correct number of positional arguments. Filtering out empty values after trimming would collapse the array and break these function calls.
Learnt from: osterman
Repo: cloudposse/atmos PR: 795
File: internal/exec/stack_processor_utils.go:378-386
Timestamp: 2024-11-19T23:00:45.899Z
Learning: In the `ProcessYAMLConfigFile` function within `internal/exec/stack_processor_utils.go`, directory traversal in stack imports is acceptable and should not be restricted.
Learnt from: aknysh
Repo: cloudposse/atmos PR: 1261
File: internal/exec/utils.go:639-640
Timestamp: 2025-05-22T15:42:10.906Z
Learning: In the Atmos codebase, when appending slices with `args := append(configAndStacksInfo.CliArgs, configAndStacksInfo.AdditionalArgsAndFlags...)`, it's intentional that the result is not stored back in the original slice. This pattern is used when the merged result serves a different purpose than the original slices, such as when creating a filtered version for component section assignments.
|
Warning This PR exceeds the recommended limit of 1,000 lines.Large PRs are difficult to review and may be rejected due to their size. Please verify that this PR does not address multiple issues. |
|
💥 This pull request now has conflicts. Could you fix it @osterman? 🙏 |
|
Warning Changelog Entry RequiredThis PR is labeled Action needed: Add a new blog post in Example filename: Alternatively: If this change doesn't require a changelog entry, remove the |
|
Warning Changelog Entry RequiredThis PR is labeled Action needed: Add a new blog post in Example filename: Alternatively: If this change doesn't require a changelog entry, remove the |
|
Warning Changelog Entry RequiredThis PR is labeled Action needed: Add a new blog post in Example filename: Alternatively: If this change doesn't require a changelog entry, remove the |
| // Overflow would occur; return just the new list to avoid panic. | ||
| return list | ||
| } | ||
| result := make([]any, existingLen, existingLen+newLen) |
Check failure
Code scanning / CodeQL
Size computation for allocation may overflow High
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 3 months ago
To fully address this vulnerability for both the main and variant alerts, all allocations that depend on user-controlled sizes must have strict size limits:
File: pkg/merge/merge.go
- In
processAppendList, additionally check whetherexistingLen + newLenexceeds a secure, reasonable threshold (e.g.,maxSliceCapacity, or an even smaller application-defined limit), and return an error or truncated value accordingly, rather than lettingmake([]any, ...)panic. - Recommend propagating an error or at least logging/aborting if unsafe allocation is attempted.
- Consider using
math.MaxIntfor exact maximum int boundary checking, but enforcing a lower, practical limit (e.g., one million entries?) for operational safety.
File: pkg/utils/yq_utils.go
- Before calling
yaml.Marshal, check the size of the node, or guard excessively large input data processed from external YQ sources. If size exceeds a sane threshold, return an error.
Required changes:
- In
processAppendList, introduce error returns or hard truncation with logs. - In YQ utilities, introduce a threshold (e.g.,
maxYAMLSize = 64 * 1024 * 1024for 64MB) before marshaling. - Add required imports (
"math","errors") only if not present.
| @@ -444,11 +444,16 @@ | ||
| // Check for potential overflow before allocation. | ||
| existingLen := len(existingList) | ||
| newLen := len(list) | ||
| if existingLen > 0 && newLen > 0 && existingLen > maxSliceCapacity-newLen { | ||
| // Overflow would occur; return just the new list to avoid panic. | ||
| return list | ||
|
|
||
| // Defensive check: ensure no int overflow/wraparound, and the sum is within maxSliceCapacity. | ||
| totalLen := existingLen + newLen | ||
| if existingLen > 0 && newLen > 0 && (totalLen < 0 || totalLen > maxSliceCapacity) { | ||
| // Overflow or allocation too large, skip allocation to avoid panic. | ||
| // Log here, or handle as needed. | ||
| // Optionally: partial append, truncation, or error handling. | ||
| return list // safer fallback (existing only replaced by the new list). | ||
| } | ||
| result := make([]any, existingLen, existingLen+newLen) | ||
| result := make([]any, existingLen, totalLen) | ||
| copy(result, existingList) | ||
| result = append(result, list...) | ||
| return result |
| @@ -89,10 +89,15 @@ | ||
| } | ||
|
|
||
| processYAMLNode(&node) | ||
| // Prevent marshaling nodes that could result in excessively large output. | ||
| const maxYAMLSize = 64 * 1024 * 1024 // 64MB limit | ||
| resultBytes, err := yaml.Marshal(&node) | ||
| if err != nil { | ||
| return nil, fmt.Errorf("EvaluateYqExpression: failed to marshal processed node: %w", err) | ||
| } | ||
| if len(resultBytes) > maxYAMLSize { | ||
| return nil, fmt.Errorf("EvaluateYqExpression: marshaled YAML result exceeds maximum allowed size of %d bytes", maxYAMLSize) | ||
| } | ||
|
|
||
| res, err := UnmarshalYAML[any](string(resultBytes)) | ||
| if err != nil { |
There was a problem hiding this comment.
Fixed in commit 2f36785. Added overflow check before allocation using a named constant maxSliceCapacity (2^30). The check on lines 447-449 ensures that if existingLen + newLen would overflow, we return just the new list to avoid panic.
|
Warning Changelog Entry RequiredThis PR is labeled Action needed: Add a new blog post in Example filename: Alternatively: If this change doesn't require a changelog entry, remove the |
|
💥 This pull request now has conflicts. Could you fix it @osterman? 🙏 |
|
Warning Changelog Entry RequiredThis PR is labeled Action needed: Add a new blog post in Example filename: Alternatively: If this change doesn't require a changelog entry, remove the |
This implements the !append YAML function that allows fine-grained control over list merging behavior in Atmos stack configurations. Lists tagged with !append will be concatenated with base values instead of replaced. Key changes: - Add !append constant and helper functions for append tag processing - Update YAML processor to handle !append tags on sequence nodes - Enhance merge logic to respect !append tags during configuration merging - Add comprehensive unit tests for append functionality - Create integration test fixtures demonstrating usage This solves the challenge of needing to concatenate lists on a case-by-case basis, particularly useful for fields like depends_on where appending is often preferred over replacement. Closes cloudposse/atmos#2980 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Add periods at the end of inline comments in yaml_func_append.go to comply with the godot linter requirement that all comments should be complete sentences ending with periods.
- Create comprehensive documentation page for !append function - Add !append to the list of YAML functions in index - Include usage examples and behavior explanation - Document interaction with global list_merge_strategy
- Add missing period to comment in yaml_func_append.go to satisfy godot linter - Add static error ErrRefusingToDeleteSymlink and use it in terraform_clean.go to satisfy err113 linter - Refactor validateAtmosBinary function in cli_test.go to reduce nested if complexity and satisfy nestif linter 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Remove package-lock.json - Add pnpm-lock.yaml - Set packageManager field to pnpm@10.20.0 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
- errors/errors.go: Alias ErrRefusingToDeleteSymlink to existing ErrRefuseDeleteSymbolicLink sentinel to ensure errors.Is() works correctly across all call sites - pkg/merge/merge.go: Fix double-append issue when global list_merge_strategy=append is enabled. Pass appendNewOnly flag through processAppendTags, processValue, processAppendList, and processNestedMap. When true, !append returns only new items so mergo.WithAppendSlice performs append without duplication. Also added overflow check before allocation to prevent panic. - tests/cli_test.go: Fix skipReason variable shadowing. Changed from short declaration (:=) to assignment (=) so the package-level skipReason is properly set and tests skip correctly. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
27fd7b2 to
40350d7
Compare
|
Warning This PR exceeds the recommended limit of 1,000 lines.Large PRs are difficult to review and may be rejected due to their size. Please verify that this PR does not address multiple issues. |
Dependency Review✅ No vulnerabilities or license issues found.Scanned FilesNone |
Added PR links to milestones across all initiatives: Auth Initiative: - EKS Kubeconfig → PR #1884 - ECR Authentication → PR #1859 - GitHub Apps → PR #1683 Developer Experience: - Streaming Terraform UI → PR #1908 - Native CI integration → PR #1891 Workflows: - Unified task execution → PR #1901 - New workflow step types → PR #1899 Extensibility: - Added !unset YAML function → PR #1521 - Added !append YAML function → PR #1513 Vendoring: - Vendor registry pattern → PR #1889 - Just-in-time vendoring → PR #1877 - Component workdir provisioning → PR #1876 Terragrunt Parity: - Multi-stack formats → PR #1842 CI/CD: - CI Summary Templates → PR #1891 - Terraform command registry → PR #1891
* feat: Add comprehensive roadmap page with initiatives and milestones - Add Roadmap component with interactive initiative cards - Create MilestoneList, MilestoneDrawer, and ProgressBar components - Add Tooltip component for PR/issue titles with theme-aware styling - Implement responsive design with flexbox alignment fixes - Add extensive roadmap data with documented milestones across 10 initiatives - Add changelog and docs links to all relevant milestones - Update Docusaurus navbar with Roadmap link 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com> * fix: Address CodeRabbit review comments for roadmap components - Fix type mismatch: use undefined instead of null for selectedMilestone - Make changelogSlugs optional in Initiative interface - Replace fragile status className construction with explicit mapping - Reduce tooltip z-index from 9999 to 1002 (just above drawer) - Add scroll/resize listeners for tooltip position updates - Improve keyboard accessibility with tabIndex, aria-describedby, role="tooltip" 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: Update Customizable list output description Focus on custom column views rather than output formats. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: Move Component-aware tab completion to DX initiative Tab completion is more about developer experience than discoverability. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor(roadmap): rename 'Customizable list output' to 'Customizable list columns' Clarifies that this is an improvement to existing list commands (added Q1 2025), not a new feature. The Q4 2025 milestone added customizable column support. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs(roadmap): update initiative names and add new milestones - Rename 'Migration & Code Generation' to 'Feature Parity with Terragrunt' - Rename 'Quality & Community' to 'Code Quality and Community' - Change 'Learning section' to 'New learning section' - Add Native Terraform to migration guides list - Add Roadmap milestone to Documentation initiative 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(roadmap): correct design patterns docs link Changed /design-patterns/design-patterns to /design-patterns 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(roadmap): add 'Request a Feature' link to initiatives section Adds a prominent link to GitHub feature request issue template for users who want to suggest new features for the roadmap. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(roadmap): add 'View Features' link to GitHub issues Adds a link to browse existing feature requests and issues alongside the 'Request a Feature' link. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(roadmap): rename 'View Features' to 'View Issues' 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor(roadmap): reorganize Feature Parity with Terragrunt initiative Changes to Feature Parity with Terragrunt: - Keep: File-scoped locals - Rename: Stack name field → Imperative stack names - Rename: Generate section inheritance → File generation (generate blocks) - Rename: Backend provisioning → Automatic backend provisioning - Add: AWS context YAML functions (!aws.account-id, !aws.region, etc.) - Remove: Metadata inheritance (merged into other features) Moved to Extensibility & Custom Components: - !literal YAML function - metadata.name for component workspace keys 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(roadmap): add planned Terragrunt parity features - Automatic source provisioning (no vendoring required) - Concurrent component provisioning - plan --all and apply --all commands - Automatic provider caching 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(roadmap): update DX milestones - Update parent directory search description to emphasize running from any directory - Add filesystem paths for components feature 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(roadmap): use neutral blue gradient for progress bars Replace orange/red colors with lighter blues to avoid 'behind schedule' connotation - progress just means 'not as far along yet', not delayed. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor(roadmap): reorganize quality and docs initiatives - Rename 'Code Quality and Community' to 'Code Quality and Test Coverage' - Rename 'Documentation Overhaul' to 'Documentation Overhaul and Community Announcements' - Move 'Changelog introduction' from quality to docs initiative - Rename migration guides milestone to 'Added migration guides' 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(roadmap): add nightly releases and PR feature releases to quality initiative 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(roadmap): add 'Push for stability' to quality initiative 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(roadmap): clarify backend provisioning vs backend generation Backend provisioning is about provisioning the backend itself (S3 bucket, DynamoDB table) for cold-start scenarios, not just generating backend.tf files. * fix(roadmap): remove DynamoDB from backend provisioning description DynamoDB locking is no longer needed for S3 state backends. * fix(roadmap): mention native S3 state locking for backend provisioning * feat(roadmap): mark --all flags as shipped, add PR links for planned features - Mark 'plan --all' and 'apply --all' as shipped (Q4 2025) - Add PR #1877 link for automatic source provisioning - Add PR #1882 link for automatic provider caching * feat(roadmap): add comprehensive PR links to milestones Added PR links to milestones across all initiatives: Auth Initiative: - EKS Kubeconfig → PR #1884 - ECR Authentication → PR #1859 - GitHub Apps → PR #1683 Developer Experience: - Streaming Terraform UI → PR #1908 - Native CI integration → PR #1891 Workflows: - Unified task execution → PR #1901 - New workflow step types → PR #1899 Extensibility: - Added !unset YAML function → PR #1521 - Added !append YAML function → PR #1513 Vendoring: - Vendor registry pattern → PR #1889 - Just-in-time vendoring → PR #1877 - Component workdir provisioning → PR #1876 Terragrunt Parity: - Multi-stack formats → PR #1842 CI/CD: - CI Summary Templates → PR #1891 - Terraform command registry → PR #1891 * feat(roadmap): improve milestone labels with descriptive context Update milestone labels to describe what each feature does rather than just listing the command or feature name: - Discoverability: Change "Added `atmos list stacks`" to "`atmos list stacks` to browse all configured stacks" etc. - Auth: Change "EKS Kubeconfig integration" to "Automatic EKS kubeconfig tied to identities" and similar for ECR 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(roadmap): add Featured Improvements section with 6 key highlights Add a prominent Featured Improvements section near the top of the roadmap page showcasing major capabilities: 1. Dev Container Support - Consistent development environments 2. Toolchain Management - Automatic tool installation and versioning 3. Cloud Authentication - Unified identity layer across clouds 4. Beautiful Workflows - Multi-step automation with typed inputs 5. Source Provisioning - Just-in-time component fetching (in-progress) 6. Native CI/CD Support - Local = CI experience (in-progress) Also fixes: - Provider auto-generation marked as shipped (Q3 2025) - Devcontainer docs link corrected 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(roadmap): enhance featured section and milestone grouping - Add PRD links to featured items (devcontainer, keyring-backends, provisioner-system, terraform-registry-migration) - Fix statuses: Toolchain Management and Beautiful Workflows changed from shipped to in-progress - Update quarters: Native CI/CD, Toolchain, Workflows, and Source Provisioning set to Q1 2026 - Add sorting by status (shipped first) and quarter (earlier first) - Implement milestone grouping with categories (Featured Improvements, Everything Else) and priorities (high, nice-to-have) - Add PR links for Toolchain Management (#1686) and Beautiful Workflows (#1899) - Update Cloud Authentication description with "Zero local config required" - Rename "Initiatives" section to "Major Initiatives" 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(roadmap): add benefits field, clickable featured cards, and overall progress stats - Add benefits field to all 89 milestones explaining "Why It Matters" - Add benefits to 6 featured items with benefit-focused explanations - Create FeaturedDrawer component for featured card detail view - Make featured cards clickable with drawer showing description/benefits - Create RoadmapStats component showing overall progress (shipped/in-progress/planned) - Add animated progress bar with color-coded segments - Expand changelog-check.yml to also require roadmap.js updates for minor/major PRs - Update CLAUDE.md with roadmap update requirements 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(roadmap): simplify drawer links to reduce visual noise Remove heavy background colors from drawer link buttons and convert them to simple inline text links that wrap horizontally. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(roadmap): use plain colors for drawer links, one per line - Change links to use neutral emphasis colors instead of brand colors - Display each link on its own line for cleaner vertical layout 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * style(roadmap): use brightening instead of underline for drawer link hover Change drawer link hover behavior to brighten text color instead of adding underline for cleaner visual feedback. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs(roadmap): add CI/local parity to cloud auth benefits Highlight that cloud authentication works identically in CI and locally, a key differentiator for the unified auth system. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * style(roadmap): increase gap between progress bar and Major Initiatives Add more breathing room between the Overall Progress section and the Major Initiatives heading below it. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * style(roadmap): double gap between progress bar and Major Initiatives Increase bottom margin from 3rem to 6rem for more visual separation. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs(roadmap): mark Terragrunt parity milestones as featured Mark all major Terragrunt feature parity milestones as featured: - File-scoped locals - Imperative stack names - File generation (generate blocks) - Automatic backend provisioning - AWS context YAML functions - Multi-stack formats (YAML, JSON and HCL) - Automatic provider caching - Automatic source provisioning - Provider auto-generation - Concurrent component provisioning Also link PR #1876 (workdir provisioning) to concurrent component provisioning milestone. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(roadmap): address CodeRabbit review feedback - Add target="_blank" and rel="noopener noreferrer" to external GitHub links in FeaturedSection.tsx (PRD and PR links) - Add target="_blank" and rel="noopener noreferrer" to external GitHub links in InitiativeCard.tsx (issues and PRs) - Add target="_blank" and rel="noopener noreferrer" to external GitHub links in FeaturedDrawer.tsx (PRD and PR links) - Guard against division by zero in RoadmapStats.tsx progress bar width calculations 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs(blog): introduce the Atmos product roadmap Add a blog post announcing the new public product roadmap page, highlighting: - Featured improvements like Dev Containers, Cloud Auth, and Workflows - Major initiatives including Terragrunt parity and documentation overhaul - Overall progress tracking across all milestones - How to get involved and provide feedback 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs(roadmap): reconcile changelog announcements with roadmap Add missing changelog links to existing milestones: - workflow-file-auto-discovery - automatic-backend-provisioning - path-based-component-resolution - aws-yaml-functions - product-roadmap Add new milestones for shipped features: - Auth: auth shell, SSO auto-provisioning, identity flag for describe - DX: terminal themes, helpful errors, provenance tracking, global env, metadata inheritance, interactive terraform prompts - Extensibility: !random, !env stack manifest support, circular dependency detection, deferred evaluation, boolean flags Update progress percentages: - Auth: 80% → 85% - DX: 75% → 90% - Extensibility: 50% → 80% 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(roadmap): correct broken documentation links Update doc links to use correct paths: - /cli/configuration/configuration → /cli/configuration - /cli/commands/commands → /cli/commands 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * address comments and feedback * address comments and feedback --------- Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com> Co-authored-by: Andriy Knysh <aknysh@users.noreply.github.com> Co-authored-by: aknysh <andriy.knysh@gmail.com>
Resolve conflicts: - website/pnpm-lock.yaml: take main's version - tests/cli_test.go: remove checkIfRebuildNeeded call (function not available) - pkg/config/process_yaml.go: fix errorlint by using %w for error wrapping 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
Warning Release Documentation RequiredThis PR is labeled
|
what
!appendYAML function that allows fine-grained control over list merging behavior in Atmos stack configurations!appendwill be concatenated with base values instead of replacedwhy
depends_onwhere appending is often the desired behavior rather than replacement!appendtag provides opt-in, per-field control that works alongside the globallist_merge_strategysettingKey Features
!appenduse append mode!appendtag works independently of the globallist_merge_strategysettingExample Usage
Testing
references
🤖 Generated with Claude Code
Summary by CodeRabbit