Add configurable AWS SDK logging with secure defaults #893

aditmeno · 2025-11-24T01:45:18Z

Summary

This PR adds flexible AWS SDK logging configuration to vals while changing the default to no logging for security, preventing credential leakage.

Implementation: AWS log level passed through function parameters to each AWS provider - no global state mutation, no package-level variables, no synchronization primitives needed.

Related to: helmfile/helmfile#2270

Problem

Previously, vals defaulted to logging AWS SDK retries and requests (aws.LogRetries | aws.LogRequest), which exposed sensitive information in logs:

🔓 AWS tokens and authorization headers
🔓 Request/response bodies containing credentials
🔓 Secret metadata

Real-world impact: Users reported seeing this in helmfile output:

SDK 2025/11/24 07:54:06 DEBUG Request
PUT /latest/api/token HTTP/1.1
Host: 169.254.169.254

Solution

1. Added `AWSLogLevel` Field to `Options` Struct

type Options struct {
    LogOutput             io.Writer
    CacheSize             int
    ExcludeSecret         bool
    FailOnMissingKeyInMap bool
    // New field:
    AWSLogLevel string  // Controls AWS SDK logging behavior
}

Supported values:

"" or "off" (default) - No logging (secure)
"minimal" - Log retries only
"standard" - Log retries + requests (previous default)
"verbose" - Log everything (debug mode)
Custom - Comma-separated: "request,response,signing"

2. Clean Implementation: Function Parameters Instead of Global State

Key improvements:

✅ No package-level global variables
✅ No os.Setenv() - no environment mutation
✅ No sync/atomic or synchronization primitives
✅ Variadic parameters for backward compatibility
✅ Simple, idiomatic Go code
✅ Thread-safe by design - each provider gets its own log level

How it works:

vals.go - Runtime stores log level in Options:

func New(opts Options) (*Runtime, error) {
    r := &Runtime{
        providers: map[string]api.Provider{},
        Options:   opts,  // Stores opts.AWSLogLevel
        // ... rest of initialization
    }
    return r, nil
}

vals.go - Passes log level when creating AWS providers:

case ProviderSSM:
    p := ssm.New(r.logger, conf, r.Options.AWSLogLevel)
    return p, nil

AWS providers - Each provider stores log level and passes it to session:

// pkg/providers/ssm/ssm.go
type provider struct {
    ssmClient   *ssm.Client
    log         *log.Logger
    AWSLogLevel string  // Stored per-provider
    // ... other fields
}

func New(l *log.Logger, cfg api.StaticConfig, awsLogLevel string) *provider {
    p := &provider{
        log:         l,
        AWSLogLevel: awsLogLevel,  // Store the log level
    }
    // ... initialize fields from config
    return p
}

func (p *provider) getSSMClient() *ssm.Client {
    if p.ssmClient != nil {
        return p.ssmClient
    }
    
    // Pass log level to session creation
    cfg := awsclicompat.NewSession(p.Region, p.Profile, p.RoleARN, p.AWSLogLevel)
    
    p.ssmClient = ssm.NewFromConfig(cfg)
    return p.ssmClient
}

awsclicompat - Uses environment variable first, then parameter:

// pkg/awsclicompat/session.go
func NewSession(region, profile, roleARN string, logLevel ...string) aws.Config {
    var level string
    if len(logLevel) > 0 {
        level = logLevel[0]
    }
    cfg, err := newConfig(ctx, region, profile, level)
    // ...
}

func parseAWSLogLevel(paramDefault string) aws.ClientLogMode {
    // Priority: AWS_SDK_GO_LOG_LEVEL env var > paramDefault > secure default
    logLevel := strings.TrimSpace(os.Getenv("AWS_SDK_GO_LOG_LEVEL"))
    
    if logLevel == "" {
        logLevel = paramDefault  // Use passed parameter
    }
    
    if logLevel == "" {
        return 0  // Secure default - no logging
    }
    // ... preset handling
}

3. Changed Default Behavior

Aspect	Before	After
Default	`aws.LogRetries \| aws.LogRequest`	`0` (no logging)
Security	❌ Credentials exposed	✅ Credentials protected
Debugging	Always on	Opt-in via config/env var
Thread Safety	❌ Data race potential	✅ No shared state
Dependencies	N/A	✅ No sync/atomic needed
Global State	N/A	✅ No mutation
Code Complexity	N/A	✅ Simple parameter passing

Usage Examples

Secure by Default (Recommended)

runtime, _ := vals.New(vals.Options{})
// No AWS SDK logging - credentials safe

Enable Minimal Logging

runtime, _ := vals.New(vals.Options{
    AWSLogLevel: "minimal",  // Retries only
})

Restore Previous Behavior

runtime, _ := vals.New(vals.Options{
    AWSLogLevel: "standard",  // Retries + requests (old default)
})

Full Debugging

runtime, _ := vals.New(vals.Options{
    AWSLogLevel: "verbose",  // Everything
})

Environment Variable Override

# Always takes precedence over Options.AWSLogLevel
export AWS_SDK_GO_LOG_LEVEL=verbose
vals eval ...

Priority Order

✅ AWS_SDK_GO_LOG_LEVEL environment variable (highest)
✅ Options.AWSLogLevel parameter (passed through to providers)
✅ Secure default (no logging)

Breaking Change

⚠️ Default behavior changes from logging to not logging.

Migration path for users who need logs:

Option 1 - Environment variable (no code change):

export AWS_SDK_GO_LOG_LEVEL=standard

Option 2 - Update code:

vals.New(vals.Options{
    AWSLogLevel: "standard",
})

Rationale: Security should be the default. Credentials and secrets should never leak unless explicitly enabled for debugging.

Testing

✅ Updated existing tests:

TestDefaultSecureBehavior - Verifies secure default (no logging)
TestParseAWSLogLevel - Updated to use new function signature
TestParseAWSLogLevelIndividualFlags - Still works

✅ Added new tests:

TestPresetLevels - Tests minimal/standard/verbose presets

✅ All tests pass:

go test ./pkg/awsclicompat/... -v
=== RUN   TestParseAWSLogLevel
=== RUN   TestParseAWSLogLevelIndividualFlags  
=== RUN   TestDefaultSecureBehavior
=== RUN   TestPresetLevels
--- PASS: TestPresetLevels (0.00s)
PASS

# Race detector clean
go test ./pkg/awsclicompat/... -race -v
PASS

Implementation Details

Why function parameters instead of environment variables or globals?

No synchronization needed: Each provider instance has its own log level
No global state mutation: No os.Setenv() calls
Standard Go patterns: Variadic parameters for optional arguments
Backward compatible: Existing calls work without changes
Simpler code: No atomic operations, no environment manipulation
Thread-safe by design: No shared mutable state

Addressing Copilot Feedback:

The implementation evolved through several iterations:

V1: Package-level defaultLogLevel variable (not thread-safe)
V2: sync/atomic.Value for thread-safety (complex)
V3: os.Setenv() in vals.New() (global state mutation)
V4 (final): Function parameters passed through call stack (clean, no side effects)

Files Changed

AWS Providers (added AWSLogLevel field + parameter):

File	Changes	Description
`pkg/providers/ssm/ssm.go`	Added field + param	Pass log level to NewSession
`pkg/providers/s3/s3.go`	Added field + param	Pass log level to NewSession
`pkg/providers/awssecrets/awssecrets.go`	Added field + param	Pass log level to NewSession
`pkg/providers/awskms/awskms.go`	Added field + param	Pass log level to NewSession

Provider Factories (accept + forward log level):

File	Changes	Description
`pkg/stringprovider/stringprovider.go`	Added param	Forward to AWS providers
`pkg/stringmapprovider/stringmapprovider.go`	Added param	Forward to AWS providers

Core Runtime:

File	Changes	Description
`vals.go`	Added AWSLogLevel option + ctx field	Pass through to providers
`pkg/awsclicompat/session.go`	Already had variadic params	No changes needed
`pkg/awsclicompat/session_test.go`	Cleaned up	Removed global variable tests

Total: 9 files changed, 159 insertions(+), 80 deletions(-)

Security Impact

🔒 Before: AWS credentials potentially exposed in logs by default
🔐 After: AWS credentials protected by default, opt-in for debugging

This change aligns with security best practices:

Principle of least privilege - No logging unless needed
Secure by default - Users must explicitly enable sensitive logging
Defense in depth - Prevents accidental credential exposure
Clean implementation - No global state, no side effects

Checklist

Related Issues

Fixes Helmfile 1.1.6+ uses Debug log level when making requests via the AWS Go SDK helmfile#2270 (AWS SDK debug logging exposure)
Enables secure logging defaults across all vals users
Enables Fix AWS SDK debug logging by making it configurable (issue #2270) helmfile#2290 (helmfile integration)

Upgrade Guide

For library users (like helmfile):

If you were relying on default AWS SDK logging, update your code:

// Old (implicit logging)
runtime, _ := vals.New(vals.Options{
    CacheSize: 512,
})

// New (explicit logging if needed)
runtime, _ := vals.New(vals.Options{
    CacheSize: 512,
    AWSLogLevel: "standard",  // or "minimal", "verbose"
})

For CLI users:

Set environment variable if you need logs:

export AWS_SDK_GO_LOG_LEVEL=standard  # or minimal, verbose

This commit properly fixes issue helmfile#2270 where AWS SDK debug logs were still appearing despite the fix in PR helmfile#2288. ## Problem The original fix in PR helmfile#2288 only suppressed vals' own logging via LogOutput: io.Discard, but did not address: 1. AWS SDK v2's separate logging system 2. AWS SDK clients created directly in remote.go 3. No way for users to enable logs for debugging As reported in PR helmfile#2288 comment, users still see: SDK 2025/11/24 07:54:06 DEBUG Request PUT /latest/api/token HTTP/1.1 Host: 169.254.169.254 ## Root Cause AWS SDK v2 uses its own logging mechanism controlled by: - AWS_SDK_GO_LOG_LEVEL environment variable (for vals library) - WithClientLogMode config option (for direct AWS SDK usage) The LogOutput option in vals.Options only affects vals' internal logging, not the AWS SDK's debug output. ## Solution Two-part solution across helmfile and vals: ### Part 1: vals library enhancement (PR helmfile/vals#893) - Added Options.AWSLogLevel field for programmatic control - Changed default from logging to no logging (secure by default) - Supports presets: "off", "minimal", "standard", "verbose" ### Part 2: helmfile changes (this PR) Made logging configurable with secure defaults via HELMFILE_ENABLE_AWS_SDK_LOGS: 1. **Added environment variable** (pkg/envvar/const.go) - HELMFILE_ENABLE_AWS_SDK_LOGS controls all AWS SDK logging - Default: false (suppress logs for security) - Set to "true" to enable debugging 2. **Conditional AWS SDK log suppression in remote.go** - Added enableAWSSDKLogs flag (reads env var on init) - Made WithClientLogMode(0) conditional in all 3 AWS config locations - Only suppresses when HELMFILE_ENABLE_AWS_SDK_LOGS != "true" 3. **Conditional vals configuration in vals.go** - Uses new Options.AWSLogLevel field from vals PR helmfile#893 - Sets to "off" by default (secure) - Sets to "standard" when HELMFILE_ENABLE_AWS_SDK_LOGS=true ## Security Impact - Default behavior: Suppresses sensitive information (tokens, auth headers, credentials) from appearing in logs - User control: Enables debugging when explicitly requested via HELMFILE_ENABLE_AWS_SDK_LOGS=true ## Dependencies - Requires vals PR helmfile/vals#893 to be merged - Once merged, helmfile's go.mod should be updated to use new vals version ## Testing - All existing tests pass - golangci-lint passes with no issues - Build succeeds Fixes helmfile#2270 Depends-on: helmfile/vals#893 Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>

Changes: 1. Updated go.mod to use vals PR helmfile#893 (aditmeno/vals@a97336ce2bf6) - Added replace directive to reference PR branch for testing - Will be updated to official release once PR merges 2. Added comprehensive unit tests for AWS SDK logging behavior: pkg/plugins/vals_test.go: - TestAWSSDKLogLevelConfiguration: Tests all log levels (off, minimal, standard, verbose, custom) - TestEnvironmentVariableReading: Tests env var reading with whitespace trimming and defaults - Verifies LogOutput is set to io.Discard only when level is "off" - All 9 test cases passing pkg/remote/remote_test.go: - TestAWSSDKLogLevelInit: Tests init() logic for reading HELMFILE_AWS_SDK_LOG_LEVEL - Tests default behavior, explicit values, and whitespace handling - All 6 test cases passing 3. All existing tests continue to pass: - pkg/plugins: PASS (3/3 test suites) - pkg/remote: PASS (all test suites) - Linting: 0 issues This commit ensures AWS SDK logging is properly tested and validates the secure default behavior (no logging) across both vals integration and direct AWS SDK usage in remote S3 operations. Related: helmfile/vals#893 Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>

This commit properly fixes issue helmfile#2270 where AWS SDK debug logs were still appearing despite the fix in PR helmfile#2288. ## Problem The original fix in PR helmfile#2288 only suppressed vals' own logging via LogOutput: io.Discard, but did not address: 1. AWS SDK v2's separate logging system 2. AWS SDK clients created directly in remote.go 3. No way for users to enable logs for debugging As reported in PR helmfile#2288 comment, users still see: SDK 2025/11/24 07:54:06 DEBUG Request PUT /latest/api/token HTTP/1.1 Host: 169.254.169.254 ## Root Cause AWS SDK v2 uses its own logging mechanism controlled by: - AWS_SDK_GO_LOG_LEVEL environment variable (for vals library) - WithClientLogMode config option (for direct AWS SDK usage) The LogOutput option in vals.Options only affects vals' internal logging, not the AWS SDK's debug output. ## Solution Two-part solution across helmfile and vals: ### Part 1: vals library enhancement (PR helmfile/vals#893) - Added Options.AWSLogLevel field for programmatic control - Changed default from logging to no logging (secure by default) - Supports presets: "off", "minimal", "standard", "verbose" ### Part 2: helmfile changes (this PR) Made logging configurable with secure defaults via HELMFILE_ENABLE_AWS_SDK_LOGS: 1. **Added environment variable** (pkg/envvar/const.go) - HELMFILE_ENABLE_AWS_SDK_LOGS controls all AWS SDK logging - Default: false (suppress logs for security) - Set to "true" to enable debugging 2. **Conditional AWS SDK log suppression in remote.go** - Added enableAWSSDKLogs flag (reads env var on init) - Made WithClientLogMode(0) conditional in all 3 AWS config locations - Only suppresses when HELMFILE_ENABLE_AWS_SDK_LOGS != "true" 3. **Conditional vals configuration in vals.go** - Uses new Options.AWSLogLevel field from vals PR helmfile#893 - Sets to "off" by default (secure) - Sets to "standard" when HELMFILE_ENABLE_AWS_SDK_LOGS=true ## Security Impact - Default behavior: Suppresses sensitive information (tokens, auth headers, credentials) from appearing in logs - User control: Enables debugging when explicitly requested via HELMFILE_ENABLE_AWS_SDK_LOGS=true ## Dependencies - Requires vals PR helmfile/vals#893 to be merged - Once merged, helmfile's go.mod should be updated to use new vals version ## Testing - All existing tests pass - golangci-lint passes with no issues - Build succeeds Fixes helmfile#2270 Depends-on: helmfile/vals#893 Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>

Changes: 1. Updated go.mod to use vals PR helmfile#893 (aditmeno/vals@a97336ce2bf6) - Added replace directive to reference PR branch for testing - Will be updated to official release once PR merges 2. Added comprehensive unit tests for AWS SDK logging behavior: pkg/plugins/vals_test.go: - TestAWSSDKLogLevelConfiguration: Tests all log levels (off, minimal, standard, verbose, custom) - TestEnvironmentVariableReading: Tests env var reading with whitespace trimming and defaults - Verifies LogOutput is set to io.Discard only when level is "off" - All 9 test cases passing pkg/remote/remote_test.go: - TestAWSSDKLogLevelInit: Tests init() logic for reading HELMFILE_AWS_SDK_LOG_LEVEL - Tests default behavior, explicit values, and whitespace handling - All 6 test cases passing 3. All existing tests continue to pass: - pkg/plugins: PASS (3/3 test suites) - pkg/remote: PASS (all test suites) - Linting: 0 issues This commit ensures AWS SDK logging is properly tested and validates the secure default behavior (no logging) across both vals integration and direct AWS SDK usage in remote S3 operations. Related: helmfile/vals#893 Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>

This PR fixes issue helmfile#2270 where AWS SDK debug logs expose sensitive credentials in helmfile output, by adding flexible, configurable AWS SDK logging with secure defaults. Problem: -------- Despite PR helmfile#2288's fix, AWS SDK debug logs still appeared in helmfile output, exposing sensitive information: - AWS tokens and authorization headers - Request/response bodies containing credentials - Secret metadata from vals providers Root Cause: ----------- 1. PR helmfile#2288 only suppressed vals' own logging via LogOutput: io.Discard 2. AWS SDK v2 uses separate logging (AWS_SDK_GO_LOG_LEVEL, WithClientLogMode) 3. Vals library defaulted to verbose logging (aws.LogRetries | aws.LogRequest) 4. No programmatic way to control AWS SDK logging Solution: --------- Two-part fix in conjunction with vals PR helmfile#893: 1. Vals library enhancement (helmfile/vals#893): - Added Options.AWSLogLevel field for programmatic control - Changed default from verbose to secure (no logging) - Added preset levels: off, minimal, standard, verbose - Maintains AWS_SDK_GO_LOG_LEVEL precedence 2. Helmfile changes (this PR): - Added HELMFILE_AWS_SDK_LOG_LEVEL environment variable - Enhanced vals configuration to use new AWSLogLevel field - Added conditional AWS SDK log suppression in remote.go (3 locations) - Comprehensive unit tests (15 test cases) Configuration: -------------- Preset levels via HELMFILE_AWS_SDK_LOG_LEVEL: - "off" (default) - No logging, secure, prevents credential leakage - "minimal" - Log retries only - "standard" - Log retries + requests (previous default behavior) - "verbose" - Log everything (requests, responses, bodies, signing) - Custom - Comma-separated values (e.g., "request,response") Priority order: 1. AWS_SDK_GO_LOG_LEVEL env var (highest) 2. HELMFILE_AWS_SDK_LOG_LEVEL env var 3. Secure default ("off") Testing: -------- Added comprehensive unit tests: - pkg/plugins/vals_test.go: 9 test cases * TestAWSSDKLogLevelConfiguration - all preset levels * TestEnvironmentVariableReading - env var parsing - pkg/remote/remote_test.go: 6 test cases * TestAWSSDKLogLevelInit - init() logic All tests passing: - pkg/plugins: PASS (3/3 test suites) - pkg/remote: PASS (all test suites) - golangci-lint: 0 issues Files changed: 7 files, 271 insertions(+), 31 deletions(-) Security: --------- Before: Credentials exposed by default (aws.LogRetries | aws.LogRequest) After: Credentials protected by default (no logging unless explicitly enabled) Follows security principles: - Secure by default - Principle of least privilege - Explicit opt-in for sensitive logging - Defense in depth Dependency: ----------- Depends on: helmfile/vals#893 Currently using: aditmeno/vals@a97336ce2bf6 (via go.mod replace) After vals PR merges: Update to official release Fixes: helmfile#2270 Related: helmfile#2288, helmfile#2289, helmfile/vals#893 Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>

aditmeno · 2025-11-24T02:41:02Z

@yxxhero PR is ready for review

This PR fixes issue helmfile#2270 where AWS SDK debug logs expose sensitive credentials in helmfile output, by adding flexible, configurable AWS SDK logging with secure defaults. Problem: -------- Despite PR helmfile#2288's fix, AWS SDK debug logs still appeared in helmfile output, exposing sensitive information: - AWS tokens and authorization headers - Request/response bodies containing credentials - Secret metadata from vals providers Root Cause: ----------- 1. PR helmfile#2288 only suppressed vals' own logging via LogOutput: io.Discard 2. AWS SDK v2 uses separate logging (AWS_SDK_GO_LOG_LEVEL, WithClientLogMode) 3. Vals library defaulted to verbose logging (aws.LogRetries | aws.LogRequest) 4. No programmatic way to control AWS SDK logging Solution: --------- Two-part fix in conjunction with vals PR helmfile#893: 1. Vals library enhancement (helmfile/vals#893): - Added Options.AWSLogLevel field for programmatic control - Changed default from verbose to secure (no logging) - Added preset levels: off, minimal, standard, verbose - Maintains AWS_SDK_GO_LOG_LEVEL precedence 2. Helmfile changes (this PR): - Added HELMFILE_AWS_SDK_LOG_LEVEL environment variable - Enhanced vals configuration to use new AWSLogLevel field - Added conditional AWS SDK log suppression in remote.go (3 locations) - Comprehensive unit tests (15 test cases) Configuration: -------------- Preset levels via HELMFILE_AWS_SDK_LOG_LEVEL: - "off" (default) - No logging, secure, prevents credential leakage - "minimal" - Log retries only - "standard" - Log retries + requests (previous default behavior) - "verbose" - Log everything (requests, responses, bodies, signing) - Custom - Comma-separated values (e.g., "request,response") Priority order: 1. AWS_SDK_GO_LOG_LEVEL env var (highest) 2. HELMFILE_AWS_SDK_LOG_LEVEL env var 3. Secure default ("off") Testing: -------- Added comprehensive unit tests: - pkg/plugins/vals_test.go: 9 test cases * TestAWSSDKLogLevelConfiguration - all preset levels * TestEnvironmentVariableReading - env var parsing - pkg/remote/remote_test.go: 6 test cases * TestAWSSDKLogLevelInit - init() logic All tests passing: - pkg/plugins: PASS (3/3 test suites) - pkg/remote: PASS (all test suites) - golangci-lint: 0 issues Files changed: 7 files, 271 insertions(+), 31 deletions(-) Security: --------- Before: Credentials exposed by default (aws.LogRetries | aws.LogRequest) After: Credentials protected by default (no logging unless explicitly enabled) Follows security principles: - Secure by default - Principle of least privilege - Explicit opt-in for sensitive logging - Defense in depth Dependency: ----------- Depends on: helmfile/vals#893 Currently using: aditmeno/vals@a97336ce2bf6 (via go.mod replace) After vals PR merges: Update to official release Fixes: helmfile#2270 Related: helmfile#2288, helmfile#2289, helmfile/vals#893 Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>

Copilot

Pull request overview

This PR adds configurable AWS SDK logging to vals while changing the default behavior from logging AWS SDK requests/retries to no logging for improved security. The change prevents potential credential leakage in logs by making logging opt-in rather than opt-out. The implementation uses a clean package-level configuration approach via SetDefaultLogLevel() instead of modifying global environment variables.

Key Changes:

Added AWSLogLevel field to Options struct with support for preset levels ("minimal", "standard", "verbose") and custom values
Introduced package-level defaultLogLevel variable and SetDefaultLogLevel() function in pkg/awsclicompat package
Changed default AWS SDK logging behavior from aws.LogRetries | aws.LogRequest to no logging (secure default)

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
vals.go	Added `AWSLogLevel` field to `Options` struct with documentation; calls `awsclicompat.SetDefaultLogLevel()` in `New()` function if AWSLogLevel is provided
pkg/awsclicompat/session.go	Added package-level `defaultLogLevel` variable, `SetDefaultLogLevel()` function, and `LogModeOff` constant; enhanced `parseAWSLogLevel()` to accept parameter default and support preset levels; changed default to no logging
pkg/awsclicompat/session_test.go	Updated test expectations to reflect secure default; added `TestPresetLevels()` to test new preset log levels; renamed `TestBackwardCompatibility` to `TestDefaultSecureBehavior`
go.mod	Updated Go version to 1.25.4
.github/workflows/lint.yaml	Updated golangci-lint version to v2.6.2

pkg/awsclicompat/session.go

pkg/awsclicompat/session_test.go

vals.go

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

pkg/awsclicompat/session.go

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

vals.go

pkg/awsclicompat/session.go

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

pkg/providers/ssm/ssm.go

pkg/providers/s3/s3.go

This commit introduces configurable AWS SDK logging to address credential leakage concerns while maintaining backward compatibility. Key changes: - Add AWSLogLevel field to all AWS provider structs (SSM, S3, Secrets Manager, KMS) - Pass AWS log level through function parameters instead of environment variables - Update provider factories to accept and forward awsLogLevel parameter - Implement preset log levels: off, minimal, standard, verbose - Default to no logging (secure default) to prevent credential exposure - Support AWS_SDK_GO_LOG_LEVEL environment variable override - Add LogModeOff constant for consistent 'no logging' representation - Consolidate preset handling in single switch statement Benefits: - Thread-safe by design (no shared mutable state) - No global state mutation via os.Setenv() - Clear precedence: env var > parameter > secure default - Improved code readability and maintainability Fixes: helmfile/helmfile#2270 Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>

Copilot

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

) * fix: make AWS SDK debug logging configurable (issue #2270) This PR fixes issue #2270 where AWS SDK debug logs expose sensitive credentials in helmfile output, by adding flexible, configurable AWS SDK logging with secure defaults. Problem: -------- Despite PR #2288's fix, AWS SDK debug logs still appeared in helmfile output, exposing sensitive information: - AWS tokens and authorization headers - Request/response bodies containing credentials - Secret metadata from vals providers Root Cause: ----------- 1. PR #2288 only suppressed vals' own logging via LogOutput: io.Discard 2. AWS SDK v2 uses separate logging (AWS_SDK_GO_LOG_LEVEL, WithClientLogMode) 3. Vals library defaulted to verbose logging (aws.LogRetries | aws.LogRequest) 4. No programmatic way to control AWS SDK logging Solution: --------- Two-part fix in conjunction with vals PR #893: 1. Vals library enhancement (helmfile/vals#893): - Added Options.AWSLogLevel field for programmatic control - Changed default from verbose to secure (no logging) - Added preset levels: off, minimal, standard, verbose - Maintains AWS_SDK_GO_LOG_LEVEL precedence 2. Helmfile changes (this PR): - Added HELMFILE_AWS_SDK_LOG_LEVEL environment variable - Enhanced vals configuration to use new AWSLogLevel field - Added conditional AWS SDK log suppression in remote.go (3 locations) - Comprehensive unit tests (15 test cases) Configuration: -------------- Preset levels via HELMFILE_AWS_SDK_LOG_LEVEL: - "off" (default) - No logging, secure, prevents credential leakage - "minimal" - Log retries only - "standard" - Log retries + requests (previous default behavior) - "verbose" - Log everything (requests, responses, bodies, signing) - Custom - Comma-separated values (e.g., "request,response") Priority order: 1. AWS_SDK_GO_LOG_LEVEL env var (highest) 2. HELMFILE_AWS_SDK_LOG_LEVEL env var 3. Secure default ("off") Testing: -------- Added comprehensive unit tests: - pkg/plugins/vals_test.go: 9 test cases * TestAWSSDKLogLevelConfiguration - all preset levels * TestEnvironmentVariableReading - env var parsing - pkg/remote/remote_test.go: 6 test cases * TestAWSSDKLogLevelInit - init() logic All tests passing: - pkg/plugins: PASS (3/3 test suites) - pkg/remote: PASS (all test suites) - golangci-lint: 0 issues Files changed: 7 files, 271 insertions(+), 31 deletions(-) Security: --------- Before: Credentials exposed by default (aws.LogRetries | aws.LogRequest) After: Credentials protected by default (no logging unless explicitly enabled) Follows security principles: - Secure by default - Principle of least privilege - Explicit opt-in for sensitive logging - Defense in depth Dependency: ----------- Depends on: helmfile/vals#893 Currently using: aditmeno/vals@a97336ce2bf6 (via go.mod replace) After vals PR merges: Update to official release Fixes: #2270 Related: #2288, #2289, helmfile/vals#893 Signed-off-by: Aditya Menon <amenon@canarytechnologies.com> * chore: update vals to use parameter-based AWS log level configuration Updated vals dependency to commit 06d7cd29 which implements clean parameter-based AWS SDK logging configuration instead of using global state mutation. Changes in vals implementation: - AWS log level passed through function parameters to each provider - No os.Setenv() - no environment mutation - No package-level global variables - No sync/atomic dependency needed - Thread-safe by design - each provider instance has its own log level This maintains the same functionality as before but with a cleaner implementation that avoids global state mutation. Signed-off-by: Aditya Menon <amenon@canarytechnologies.com> * deps: update vals to upstream v0.42.6 Update from vals fork (aditmeno/vals) to official release v0.42.6. Remove replace directive now that vals PR #893 has been merged upstream. This brings in the AWS SDK log level configuration improvements: - SetDefaultLogLevel() package-level function - Options.AWSLogLevel field support - Secure default (no logging) - Preset log levels (off, minimal, standard, verbose) Also updates related dependencies: - Azure SDK and auth libraries - AWS SDK config and credentials - OAuth2 library Signed-off-by: Aditya Menon <amenon@canarytechnologies.com> --------- Signed-off-by: Aditya Menon <amenon@canarytechnologies.com>

aditmeno requested review from itscaro, mumoshu, xiaomudk and yxxhero as code owners November 24, 2025 01:45

This was referenced Nov 24, 2025

Fix AWS SDK debug logging by making it configurable (issue #2270) helmfile/helmfile#2290

Merged

disable AWS GO SDK logging helmfile/helmfile#2289

Closed

aditmeno force-pushed the fix/configurable-aws-sdk-logging branch 3 times, most recently from 54249b5 to 847469e Compare November 24, 2025 02:31

aditmeno force-pushed the fix/configurable-aws-sdk-logging branch from 847469e to 640e10d Compare November 24, 2025 02:36

aditmeno force-pushed the fix/configurable-aws-sdk-logging branch from 640e10d to 42f18f6 Compare November 24, 2025 02:39

aditmeno force-pushed the fix/configurable-aws-sdk-logging branch from 42f18f6 to 019660e Compare November 24, 2025 02:44

yxxhero requested a review from Copilot November 24, 2025 03:35

Copilot started reviewing on behalf of yxxhero November 24, 2025 03:35 View session

Copilot finished reviewing on behalf of yxxhero November 24, 2025 03:38

Copilot AI reviewed Nov 24, 2025

View reviewed changes

pkg/awsclicompat/session.go Outdated Show resolved Hide resolved

pkg/awsclicompat/session.go Outdated Show resolved Hide resolved

pkg/awsclicompat/session_test.go Show resolved Hide resolved

vals.go Show resolved Hide resolved

aditmeno requested a review from Copilot November 24, 2025 04:28

Copilot started reviewing on behalf of aditmeno November 24, 2025 04:28 View session

Copilot finished reviewing on behalf of aditmeno November 24, 2025 04:32

Copilot AI reviewed Nov 24, 2025

View reviewed changes

pkg/awsclicompat/session.go Show resolved Hide resolved

aditmeno force-pushed the fix/configurable-aws-sdk-logging branch from c72f4b7 to fae10e2 Compare November 24, 2025 04:41

aditmeno requested a review from Copilot November 24, 2025 04:42

Copilot started reviewing on behalf of aditmeno November 24, 2025 04:44 View session

Copilot finished reviewing on behalf of aditmeno November 24, 2025 04:48

Copilot AI reviewed Nov 24, 2025

View reviewed changes

vals.go Show resolved Hide resolved

pkg/awsclicompat/session.go Outdated Show resolved Hide resolved

aditmeno force-pushed the fix/configurable-aws-sdk-logging branch from fae10e2 to cde7b22 Compare November 24, 2025 04:53

aditmeno requested a review from Copilot November 24, 2025 04:53

Copilot started reviewing on behalf of aditmeno November 24, 2025 04:54 View session

Copilot finished reviewing on behalf of aditmeno November 24, 2025 04:57

Copilot AI reviewed Nov 24, 2025

View reviewed changes

pkg/providers/ssm/ssm.go Show resolved Hide resolved

pkg/providers/s3/s3.go Show resolved Hide resolved

aditmeno force-pushed the fix/configurable-aws-sdk-logging branch 2 times, most recently from 3d1f847 to b01290e Compare November 24, 2025 04:59

aditmeno force-pushed the fix/configurable-aws-sdk-logging branch from b01290e to ef997bd Compare November 24, 2025 05:01

aditmeno requested a review from Copilot November 24, 2025 05:03

Copilot started reviewing on behalf of aditmeno November 24, 2025 05:04 View session

Copilot finished reviewing on behalf of aditmeno November 24, 2025 05:07

Copilot AI reviewed Nov 24, 2025

View reviewed changes

yxxhero merged commit c36c996 into helmfile:main Nov 24, 2025
11 checks passed

aditmeno deleted the fix/configurable-aws-sdk-logging branch November 24, 2025 15:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add configurable AWS SDK logging with secure defaults #893

Add configurable AWS SDK logging with secure defaults #893

Uh oh!

aditmeno commented Nov 24, 2025 •

edited

Loading

Uh oh!

aditmeno commented Nov 24, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Add configurable AWS SDK logging with secure defaults #893

Add configurable AWS SDK logging with secure defaults #893

Uh oh!

Conversation

aditmeno commented Nov 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Problem

Solution

1. Added AWSLogLevel Field to Options Struct

2. Clean Implementation: Function Parameters Instead of Global State

3. Changed Default Behavior

Usage Examples

Secure by Default (Recommended)

Enable Minimal Logging

Restore Previous Behavior

Full Debugging

Environment Variable Override

Priority Order

Breaking Change

Testing

Implementation Details

Files Changed

Security Impact

Checklist

Related Issues

Upgrade Guide

Uh oh!

aditmeno commented Nov 24, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

aditmeno commented Nov 24, 2025 •

edited

Loading

1. Added `AWSLogLevel` Field to `Options` Struct