Google secret manager store implementation#1034
Conversation
📝 WalkthroughWalkthroughThis pull request updates the module’s dependency management and introduces support for a new Google Secret Manager store. The changes include adding and removing specific dependencies in the module file, implementing the store functionality with associated methods and error handling, and creating a comprehensive test suite using a mock client. Additionally, the registry now supports the new store type, the documentation has been updated to reflect this new feature, and new error variables have been defined for detailed error reporting in Google Secret Manager operations. Changes
Sequence Diagram(s)sequenceDiagram
participant U as User
participant GS as GSMStore
participant GC as GSMClient
participant SM as SecretManager API
U->>GS: Call Set(stack, component, key, value)
GS->>GS: Validate inputs and construct key
alt Secret does not exist
GS->>GC: CreateSecret(request)
GC->>SM: Request to create secret
SM-->>GC: Secret created
GS->>GC: AddSecretVersion(request)
GC->>SM: Request to add secret version
SM-->>GC: Secret version added
else Secret exists
GS->>GC: AddSecretVersion(request)
GC->>SM: Request to add secret version
SM-->>GC: Secret version added
end
GS-->>U: Return operation result
sequenceDiagram
participant U as User
participant GS as GSMStore
participant GC as GSMClient
participant SM as SecretManager API
U->>GS: Call Get(stack, component, key)
GS->>GS: Validate inputs and construct resource name
GS->>GC: AccessSecretVersion(request)
GC->>SM: Request to retrieve secret version
SM-->>GC: Return secret data
GC-->>GS: Secret data received
GS-->>U: Return secret data
Warning There were issues while running some tools. Please review the errors and either fix the tool’s configuration or disable the tool if it’s a critical failure. 🔧 golangci-lint (1.62.2)Error: can't load config: the Go language version (go1.23) used to build golangci-lint is lower than the targeted Go version (1.24.0) Tip ⚡🧪 Multi-step agentic review comment chat (experimental)
📜 Recent review detailsConfiguration used: .coderabbit.yaml ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (1)
🧰 Additional context used🧠 Learnings (1)go.mod (1)⏰ Context from checks skipped due to timeout of 90000ms (8)
🔇 Additional comments (4)
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (9)
pkg/store/google_secret_manager_store.go (4)
23-36: Consider environment-based credential management
Storing credentials as a JSON string in configuration risks accidental exposure. Adopting environment variables or secret references can enhance security.
38-76: Consider allowing caller-supplied context
Currently, a background context is used to initialize the store. Passing a custom context would allow more flexibility for cancellation and timeouts.
99-190: Refactor creation & versioning flow
- Consider splitting secret creation and version addition into separate methods for clarity and improved testability.
- Restricting values to strings might be limiting if you need to store JSON or other data.
192-237: Support specifying secret versions
Always retrieving "latest" can be restrictive. Consider allowing a version parameter for more advanced workflows.pkg/store/google_secret_manager_store_test.go (3)
51-55: Remove or utilize the unused helper
Static analysis flags “newMockGSMClient” as unused. Consider removing or integrating it in tests.🧰 Tools
🪛 golangci-lint (1.62.2)
52-52: func
newMockGSMClientis unused(unused)
56-272: Comprehensive sub-tests
“TestGSMStore_Set” covers multiple paths effectively. If concurrency is a concern, consider adding parallel or race tests.
376-429: Handling environment credentials
Providing credentials from the environment is pragmatic. You might mock environment variables for consistent CI runs.website/docs/core-concepts/stacks/yaml-functions/store.mdx (1)
68-74: Effective Google Secret Manager Example Addition
This set of changes demonstrates the usage of Google Secret Manager for various scenarios—including direct retrieval, default values, and cross-stack access. The examples are concise and correctly use templating for dynamic stack names. One minor suggestion would be to include a brief inline note (or reference) on when to prefer using the alias “gsm” compared to “google-secret-manager” for added clarity.website/docs/core-concepts/projects/configuration/stores.mdx (1)
204-248: Detailed GSM Options and Authentication Instructions
The expanded documentation following the configuration sample thoroughly explains the options for GSM—covering the required project_id, optional prefix and credentials, as well as the stack_delimiter. The authentication section is particularly useful, outlining Application Default Credentials, Direct Credentials, and Workload Identity methods. This level of detail greatly aids users. A small suggestion: consider emphasizing best practices for secure handling of credentials (especially when embedding JSON strings), which can further educate users on avoiding potential pitfalls.
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
go.sumis excluded by!**/*.sum
📒 Files selected for processing (6)
go.mod(3 hunks)pkg/store/google_secret_manager_store.go(1 hunks)pkg/store/google_secret_manager_store_test.go(1 hunks)pkg/store/registry.go(1 hunks)website/docs/core-concepts/projects/configuration/stores.mdx(2 hunks)website/docs/core-concepts/stacks/yaml-functions/store.mdx(1 hunks)
🧰 Additional context used
🪛 golangci-lint (1.62.2)
pkg/store/google_secret_manager_store_test.go
52-52: func newMockGSMClient is unused
(unused)
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: Summary
🔇 Additional comments (13)
pkg/store/google_secret_manager_store.go (3)
1-13: Imports are well-organized
Everything looks good here.
15-21: Interface definition is clean
No immediate concerns with concurrency usage or naming in the GSMClient interface.
78-97: Key transformation collisions
Replacing slashes with underscores may cause collisions (e.g., multiple slash segments). Confirm that potential collisions are acceptable or consider a unique delimiter.pkg/store/registry.go (1)
37-48: Registration logic is consistent
Storing the parsed GSM options in the registry matches patterns used for other store types.pkg/store/google_secret_manager_store_test.go (3)
1-15: Imports and initial setup
All included packages appear necessary and relevant.
17-49: Mock implementation looks solid
Neatly uses the testify/mock framework for mocking GSM client methods.
274-374: Sufficient coverage for Get method
Tests multiple outcome scenarios, including error handling. Implementation looks thorough.website/docs/core-concepts/stacks/yaml-functions/store.mdx (1)
63-67: Clear AWS SSM Example in YAML Block
The newly added lines neatly illustrate how to retrieve security group IDs via AWS SSM Parameter Store. The indentation and labeling are clear, adding value to the documentation.website/docs/core-concepts/projects/configuration/stores.mdx (2)
15-15: Support for Google Secret Manager Added to Stores List
Including Google Secret Manager in the list of supported stores is a welcome update. The link provided directs users to the official documentation, ensuring they can obtain more details if needed.
184-203: Comprehensive GSM YAML Configuration Sample
The YAML configuration block for the Google Secret Manager is clear and well-structured. It illustrates both the direct type (“google-secret-manager”) and its alias (“gsm”) for production configurations, along with key options such as project_id, prefix, and credentials. This practical example will help users correctly integrate GSM with their atmos.yaml configurations.go.mod (3)
6-6: New Dependency for GSM Functionality
The addition of the cloud.google.com/go/secretmanager v1.11.5 dependency is well justified by the new GSM store implementation. It’s important to ensure that this version is compatible with other Google Cloud dependencies in the project.
31-31: Direct Requirement for GAX Library
Noting the appearance of github.com/googleapis/gax-go/v2 v2.12.3 as a direct dependency now—this update signals a cleaner dependency graph. Please verify that this change doesn’t introduce any conflicts with transitive dependencies elsewhere in the codebase.
63-63: Explicit Inclusion of google.golang.org/api
The explicit requirement of google.golang.org/api v0.171.0 ensures stability for GSM interactions. Like the GAX update, confirm that making this dependency direct meets your project’s overall dependency management strategy.
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (2)
pkg/store/google_secret_manager_store.go (1)
99-190: Add context timeout for GSM operations.Using context.Background() without a timeout could lead to hanging operations. Consider adding a reasonable timeout.
Apply this diff:
func (s *GSMStore) Set(stack string, component string, key string, value interface{}) error { + ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second) + defer cancel() - ctx := context.Background()pkg/store/google_secret_manager_store_test.go (1)
376-429: Add test case for client cleanup.Consider adding a test case to verify that the client is properly closed when an error occurs during initialization.
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
pkg/store/google_secret_manager_store.go(1 hunks)pkg/store/google_secret_manager_store_test.go(1 hunks)
🧰 Additional context used
🪛 golangci-lint (1.62.2)
pkg/store/google_secret_manager_store_test.go
52-52: func newMockGSMClient is unused
(unused)
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: Summary
🔇 Additional comments (4)
pkg/store/google_secret_manager_store.go (2)
15-36: LGTM! Well-structured interface and struct declarations.The GSMClient interface is well-designed for mocking, and the GSMStore struct with its options provides a clean configuration structure.
192-237: Add context timeout for GSM operations.Similar to the Set method, add a timeout to prevent hanging operations.
pkg/store/google_secret_manager_store_test.go (2)
56-272: LGTM! Comprehensive test coverage.The test cases thoroughly cover success scenarios, error handling, and input validation.
274-374: LGTM! Well-structured test cases.The test cases provide good coverage of both success and error scenarios.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
pkg/store/google_secret_manager_store.go (1)
108-200: Potential concurrency considerations
If multiple calls toSet()operate on the same key concurrently, consider version sprawl and race conditions. Also, storing large secrets might require additional checks.
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
go.sumis excluded by!**/*.sum
📒 Files selected for processing (3)
go.mod(3 hunks)pkg/store/google_secret_manager_store.go(1 hunks)pkg/store/google_secret_manager_store_test.go(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
- pkg/store/google_secret_manager_store_test.go
- go.mod
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: Summary
🔇 Additional comments (7)
pkg/store/google_secret_manager_store.go (7)
1-19: Great package structure
The imports and thegsmOperationTimeoutconstant form a clean foundation. The 30-second timeout is a sensible default.
20-26: Neat interface
TheGSMClientinterface neatly captures the required Secret Manager methods.
28-34: Straightforward store struct
All essential fields (client, project ID, prefix, delimiter) are logically grouped.
36-41: Flexible options
Allowing optional credentials, prefix, and delimiter is convenient for diverse configurations.
46-85: Robust constructor
Resource cleanup on error is carefully handled, preventing leaks. Everything else looks solid.
87-106: Comprehensive key transformation
Stripping slashes and extraneous underscores ensures secret IDs remain valid in Google Secret Manager.
202-248: String-based retrieval
Returning string data is fine but limits storing binary or structured data. This is acceptable for most use cases.
mcalhoun
left a comment
There was a problem hiding this comment.
Thanks for this contribution, @shirkevich! Overall, it looks really good, just left a few small comments.
|
💥 This pull request now has conflicts. Could you fix it @shirkevich? 🙏 |
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
website/docs/core-concepts/projects/configuration/stores.mdx(2 hunks)website/docs/core-concepts/stacks/yaml-functions/store.mdx(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- website/docs/core-concepts/stacks/yaml-functions/store.mdx
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: Summary
🔇 Additional comments (3)
website/docs/core-concepts/projects/configuration/stores.mdx (3)
18-21: LGTM! The list is already alphabetized.The list of supported stores is properly organized in alphabetical order, which aligns with the documentation's structure and improves readability.
135-150: LGTM! Comprehensive examples covering key configurations.The examples effectively demonstrate:
- Both store type identifiers (
google-secret-managerandgsm)- Different authentication methods
- Optional configuration parameters
180-197: LGTM! Comprehensive authentication documentation.The authentication section effectively covers all supported methods with clear setup instructions and security best practices.
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1034 +/- ##
==========================================
+ Coverage 20.81% 21.24% +0.43%
==========================================
Files 171 172 +1
Lines 19139 19305 +166
==========================================
+ Hits 3983 4101 +118
- Misses 14388 14431 +43
- Partials 768 773 +5
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (3)
pkg/store/google_secret_manager_store.go (3)
17-20: Make the operation timeout configurable
Currently,gsmOperationTimeoutis hardcoded to 30 seconds. Exposing it in the configuration can help accommodate different performance environments.
49-88: Add optional debug logging
Currently, the function sets up the client without providing logs to troubleshoot credential or connectivity issues.
144-165: Consider logging version creation
Adding a simple debug log here would help trace newly added secret versions.
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
pkg/store/errors.go(1 hunks)pkg/store/google_secret_manager_store.go(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: Summary
🔇 Additional comments (8)
pkg/store/errors.go (1)
36-39: Nice addition of specialized error messages
They align well with existing error definitions and maintain clarity for GSM-specific usage.pkg/store/google_secret_manager_store.go (7)
22-28: Interface usage is good
Restricting to needed methods fosters easier testing and mocking.
30-36: Struct definition looks solid
Storing minimal state helps simplify store usage and maintenance.
38-44: Well-defined GSM store options
Optional credentials and stack delimiter fields are a flexible approach.
90-109: Well-structured key generation
Replacing slashes and trimming underscores ensures names are valid for Google Secret Manager.
111-142: Robust secret creation
Graceful handling ofAlreadyExistsavoids errors when a secret is already present.
167-202: String-only value storage is clear
Restricting the value type simplifies usage and aligns with typical Secret Manager patterns.
204-246: Get method is straightforward
Returning the secret data as a string aligns well with typical usage expectations.
|
💥 This pull request now has conflicts. Could you fix it @shirkevich? 🙏 |
There was a problem hiding this comment.
Caution
Inline review comments failed to post. This is likely due to GitHub's limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.
Actionable comments posted: 1
🧹 Nitpick comments (10)
tests/fixtures/scenarios/invalid-stacks/stacks/catalog/invalid-template.yaml (1)
1-6: Clarify the Test Fixture's PurposeThis YAML fixture intentionally embeds an undefined templating variable (
{{ .not-exist }}) in thelocationfield to simulate an invalid template scenario for the Terraform componentmyapp. To improve clarity for future maintainers, consider adding a comment at the top of the file that explains its intended use for testing error handling in stack templates.website/docs/core-concepts/share-data/remote-state.mdx (1)
363-364: Punctuation Recommendation for PrimaryCTAA static analysis hint suggests adding a punctuation mark at the end of the PrimaryCTA content. Consider appending a period (or other appropriate punctuation) after the text in the PrimaryCTA component on these lines to ensure the paragraph ends neatly.
🧰 Tools
🪛 LanguageTool
[grammar] ~363-~363: Please add a punctuation mark at the end of paragraph.
Context: ...n how to use 'atmos.Component' template function(PUNCTUATION_PARAGRAPH_END)
internal/exec/workflow.go (1)
119-129: Improved error messaging with available workflow optionsThe enhanced error handling provides useful guidance to users when a workflow isn't found by listing all available workflows. This is a significant usability improvement.
Consider simplifying the sorting operation with
sort.Strings(validWorkflows)instead ofsort.Sort(sort.StringSlice(validWorkflows))for better readability.- sort.Sort(sort.StringSlice(validWorkflows)) + sort.Strings(validWorkflows)pkg/utils/markdown_utils_test.go (1)
96-96: Test command using incorrect test function name.The command is running
TestPrintErrorMarkdownAndExitwhen testingPrintInvalidUsageErrorAndExit. This should be updated to use the correct test function name.-cmd := exec.Command(execPath, "-test.run=TestPrintErrorMarkdownAndExit") +cmd := exec.Command(execPath, "-test.run=TestPrintInvalidUsageErrorAndExit")tests/fixtures/scenarios/vendor2/atmos.yaml (1)
10-10: Minor nitpick: Remove trailing spaces.Line 10 contains trailing whitespace per the YAMLlint feedback. Please remove it for cleaner formatting.
🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 10-10: trailing spaces
(trailing-spaces)
tests/fixtures/scenarios/atmos-overrides-section/atmos.yaml (1)
10-10: Minor nitpick: Remove trailing whitespace.Line 10 seems to have extraneous trailing spaces. Removing these will help maintain consistency with YAML lint guidelines.
🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 10-10: trailing spaces
(trailing-spaces)
cmd/workflow_test.go (4)
23-23: Consider handling error from os.Pipe()The ignored error from
os.Pipe()should be captured and checked. While pipe creation rarely fails, proper error handling improves robustness.- r, w, _ := os.Pipe() + r, w, err := os.Pipe() + assert.NoError(t, err, "Creating pipe for stdout capture should execute without error")
15-19: Add cleanup for environment variablesEnvironment variables set during tests should be restored to their original values when the test completes to prevent side effects on other tests.
+ oldAtmosCliConfigPath := os.Getenv("ATMOS_CLI_CONFIG_PATH") + oldAtmosBasePath := os.Getenv("ATMOS_BASE_PATH") + defer func() { + os.Setenv("ATMOS_CLI_CONFIG_PATH", oldAtmosCliConfigPath) + os.Setenv("ATMOS_BASE_PATH", oldAtmosBasePath) + }() err := os.Setenv("ATMOS_CLI_CONFIG_PATH", stacksPath) assert.NoError(t, err, "Setting 'ATMOS_CLI_CONFIG_PATH' environment variable should execute without error")
40-40: Correct the error message for pipe closure operationThe error message for closing the writer pipe incorrectly references the workflow command. Use a more descriptive message specific to the pipe closing operation.
- assert.NoError(t, err, "'atmos workflow' command should execute without error") + assert.NoError(t, err, "Closing stdout pipe writer should execute without error")
13-13: Consider using test helper for path resolutionThe hardcoded relative path "../tests/fixtures/scenarios/atmos-overrides-section" might break if the directory structure changes. Consider using a test helper function to locate test fixtures relative to the repository root.
🛑 Comments failed to post (1)
cmd/workflow_test.go (1)
38-42: 🛠️ Refactor suggestion
Use defer for stdout restoration
The current implementation could leave stdout in a corrupted state if the test fails before restoration. Using defer at the beginning of the test ensures proper cleanup regardless of execution path.
// Capture stdout oldStdout := os.Stdout r, w, _ := os.Pipe() os.Stdout = w + defer func() { + os.Stdout = oldStdout + }() // ...test execution... // Close the writer and restore stdout err = w.Close() assert.NoError(t, err, "'atmos workflow' command should execute without error") - os.Stdout = oldStdout📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.// Capture stdout oldStdout := os.Stdout r, w, _ := os.Pipe() os.Stdout = w defer func() { os.Stdout = oldStdout }() // ...test execution... // Close the writer; stdout is restored automatically by the deferred call err = w.Close() assert.NoError(t, err, "'atmos workflow' command should execute without error")
what
google-secret-managerand its aliasgsmwhy
references
Code changes:
pkg/store/google_secret_manager_store.go: Main GSM store implementationpkg/store/google_secret_manager_store_test.go: Test coveragepkg/store/registry.go: Added GSM store type registrationwebsite/docs/core-concepts/projects/configuration/stores.mdx: Added GSM documentationRelated GCP documentation:
Summary by CodeRabbit
New Features
Documentation
Tests
Chores