Skip to content

fix(set): error on encryption failure; use LocalStack for AWS tests#324

Merged
jdx merged 6 commits intomainfrom
refactor/localstack-aws-tests
Mar 8, 2026
Merged

fix(set): error on encryption failure; use LocalStack for AWS tests#324
jdx merged 6 commits intomainfrom
refactor/localstack-aws-tests

Conversation

@jdx
Copy link
Copy Markdown
Owner

@jdx jdx commented Mar 8, 2026

Summary

  • Fix encryption failure fallback: fnox set previously swallowed encryption errors from any encryption provider (age, aws-kms, azure-kms, gcp-kms, fido2, yubikey) and silently stored secrets as plaintext. Now encryption failures are hard errors.
  • Add endpoint optional field to AWS KMS, Secrets Manager, and Parameter Store providers for custom endpoint support (e.g., LocalStack)
  • Convert all AWS test files to use LocalStack instead of real AWS credentials
  • Expand LocalStack services in CI from sts,iam to sts,iam,kms,secretsmanager,ssm

Test plan

  • CI passes with LocalStack-backed AWS tests on Ubuntu
  • macOS tests skip gracefully (no LocalStack available)
  • KMS error-path test verifies encryption failure is a hard error
  • All existing non-AWS tests unaffected

🤖 Generated with Claude Code


Note

Medium Risk
Changes secret-writing semantics by failing instead of silently storing plaintext, which may break existing workflows that relied on the fallback. Adds new AWS endpoint configuration paths and rewires AWS tests/CI to LocalStack, which could surface environment-specific issues.

Overview
Behavior change: fnox set no longer falls back to storing plaintext when an encryption provider fails; encryption errors now fail the command.

AWS provider updates: Adds optional endpoint support to aws-kms, aws-sm, and aws-ps provider configs (schema + wizard metadata), and wires it through the AWS SDK clients so requests can target custom endpoints (e.g., LocalStack).

Test/CI changes: CI now starts LocalStack with additional services and pre-creates KMS/Secrets Manager resources; AWS bats tests are rewritten to use LOCALSTACK_ENDPOINT with dummy creds and updated assertions (including expecting failures on wrong keys).

Written by Cursor Bugbot for commit 1c4aeae. This will update automatically on new commits. Configure here.

Add `endpoint` optional field to AWS KMS, Secrets Manager, and Parameter
Store providers to support custom endpoints (like LocalStack). Convert
all AWS test files to use LocalStack instead of real AWS credentials,
removing the need for live AWS access in CI. Expand LocalStack services
in CI from `sts,iam` to `sts,iam,kms,secretsmanager,ssm` and set up
test resources (KMS key, secrets) during CI setup.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@gemini-code-assist
Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the development and testing workflow for AWS integrations by transitioning all related tests to use LocalStack. This change allows developers to run AWS-dependent tests locally and in CI without requiring actual AWS credentials, leading to faster feedback cycles and reduced operational costs. The introduction of an optional endpoint field across AWS providers also offers greater flexibility for custom or mock AWS environments.

Highlights

  • AWS Provider Configuration: Added an optional endpoint field to the AWS KMS, Secrets Manager, and Parameter Store providers, allowing custom endpoint configurations (e.g., for LocalStack).
  • AWS Test Refactoring: Converted all AWS integration tests (aws_kms.bats, aws_secrets_manager.bats, aws_parameter_store.bats) to utilize LocalStack for mock AWS services, eliminating the dependency on real AWS credentials.
  • CI/CD Enhancements: Expanded the LocalStack services configured in CI from sts,iam to sts,iam,kms,secretsmanager,ssm to support the new testing approach.
  • Test Setup Automation: Implemented automated setup for KMS keys, test secrets, and dummy AWS credentials within the CI environment to facilitate LocalStack-based testing.
  • Test Optimization: Removed the @expensive tag from Secrets Manager tests, as they no longer incur real AWS costs or significant delays due to the LocalStack integration.
Changelog
  • docs/public/schema.json
    • Updated the schema to include an optional endpoint field for AWS KMS, Parameter Store, and Secrets Manager providers, and reordered other provider definitions.
  • providers/aws-kms.toml
    • Added an optional endpoint field configuration.
  • providers/aws-ps.toml
    • Added an optional endpoint field configuration.
  • providers/aws-sm.toml
    • Added an optional endpoint field configuration.
  • src/commands/provider/add.rs
    • Modified the AddCommand to include the endpoint field for AWS KMS, Parameter Store, and Secrets Manager providers when initializing new configurations.
  • src/providers/aws_kms.rs
    • Modified the AwsKmsProvider struct and its constructor to include an optional endpoint field.
    • Updated the create_client method to use the provided endpoint if available.
  • src/providers/aws_ps.rs
    • Modified the AwsParameterStoreProvider struct and its constructor to include an optional endpoint field.
    • Updated the create_client method to use the provided endpoint if available.
  • src/providers/aws_sm.rs
    • Modified the AwsSecretsManagerProvider struct and its constructor to include an optional endpoint field.
    • Updated the create_client method to use the provided endpoint if available.
  • test/aws_kms.bats
    • Updated the test script to use LocalStack, including changes to prerequisites, dummy AWS credential setup, LocalStack health checks, KMS key/alias creation, and fnox configuration with the LOCALSTACK_ENDPOINT.
    • Removed a test case related to fnox set warnings for wrong KMS keys.
  • test/aws_parameter_store.bats
    • Updated the test script to use LocalStack, including changes to prerequisites, dummy AWS credential setup, and fnox configuration with the LOCALSTACK_ENDPOINT.
    • Removed a sleep command previously used for AWS propagation delays.
  • test/aws_secrets_manager.bats
    • Removed the @expensive tag from the test file.
    • Updated the test script to use LocalStack, including changes to prerequisites, dummy AWS credential setup, LocalStack health checks, secret creation, and fnox configuration with the LOCALSTACK_ENDPOINT.
    • Removed a sleep command previously used for AWS propagation delays.
Ignored Files
  • Ignored by pattern: .github/workflows/** (1)
    • .github/workflows/ci.yml
Activity
  • CI passes with LocalStack-backed AWS tests on Ubuntu.
  • macOS tests skip gracefully as LocalStack is not available on that platform.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the AWS provider tests to use LocalStack, aiming to make the CI faster and more reliable. While the changes to add the endpoint field to AWS providers are well-implemented for testing, the unvalidated endpoint field introduces a potential credential theft vulnerability if a user is tricked into using a malicious configuration. It is recommended to add validation to the endpoint URL to ensure it uses secure schemes or is restricted to local addresses. Additionally, there is a suggestion to discuss a removed test case.

I am having trouble creating individual review comments. Click here to see my feedback.

src/providers/aws_kms.rs (145-147)

security-medium medium

The endpoint field allows users to specify a custom URL for AWS KMS requests. This URL is used directly by the AWS SDK without validation. If a user is tricked into using a malicious configuration file (e.g., in a shared repository), an attacker could point the endpoint to a server they control. When fnox attempts to communicate with AWS, it will send the user's AWS credentials (e.g., in the Authorization header) to the attacker's server, leading to potential credential theft.

Remediation: Validate the endpoint URL. Restrict it to https:// schemes unless the host is localhost or 127.0.0.1. Alternatively, consider only allowing the endpoint to be set via environment variables (like AWS_ENDPOINT_URL) rather than in the persistent configuration file, or provide a clear warning to the user when a custom endpoint is being used.

src/providers/aws_ps.rs (161-163)

security-medium medium

The endpoint field allows users to specify a custom URL for AWS Parameter Store requests. This URL is used directly by the AWS SDK without validation. If a user is tricked into using a malicious configuration file (e.g., in a shared repository), an attacker could point the endpoint to a server they control. When fnox attempts to communicate with AWS, it will send the user's AWS credentials (e.g., in the Authorization header) to the attacker's server, leading to potential credential theft.

Remediation: Validate the endpoint URL. Restrict it to https:// schemes unless the host is localhost or 127.0.0.1. Alternatively, consider only allowing the endpoint to be set via environment variables (like AWS_ENDPOINT_URL) rather than in the persistent configuration file, or provide a clear warning to the user when a custom endpoint is being used.

src/providers/aws_sm.rs (190-192)

security-medium medium

The endpoint field allows users to specify a custom URL for AWS Secrets Manager requests. This URL is used directly by the AWS SDK without validation. If a user is tricked into using a malicious configuration file (e.g., in a shared repository), an attacker could point the endpoint to a server they control. When fnox attempts to communicate with AWS, it will send the user's AWS credentials (e.g., in the Authorization header) to the attacker's server, leading to potential credential theft.

Remediation: Validate the endpoint URL. Restrict it to https:// schemes unless the host is localhost or 127.0.0.1. Alternatively, consider only allowing the endpoint to be set via environment variables (like AWS_ENDPOINT_URL) rather than in the persistent configuration file, or provide a clear warning to the user when a custom endpoint is being used.

test/aws_kms.bats (200-209)

medium

This test case, which verifies the behavior of fnox set when KMS encryption fails, has been removed. Removing tests can risk future regressions. Could you please explain why this test was removed? If the behavior it tested (falling back to plaintext) is no longer correct, please consider adding a new test to verify the current expected behavior (e.g., failing with an error).

@greptile-apps
Copy link
Copy Markdown

greptile-apps Bot commented Mar 8, 2026

Greptile Summary

This PR delivers two related improvements: it fixes a silent-plaintext security regression in fnox set (encryption failures are now hard errors across all encryption providers), and it migrates all AWS integration tests to LocalStack, adding an optional endpoint field to the aws-kms, aws-sm, and aws-ps providers to support custom endpoint URLs.

Key changes:

  • src/commands/set.rs: Encryption provider errors now propagate immediately via await? instead of warning and silently storing plaintext — the correct security behavior. Affected tests (test/local_config.bats, test/profile_config.bats) were correctly updated to use the plain provider type to avoid triggering encryption where not needed.
  • src/providers/aws_kms.rs, aws_ps.rs, aws_sm.rs: Each provider gains an optional endpoint field; when set, the AWS SDK client is built via Builder::from(&config).endpoint_url(...), the idiomatic SDK v2 pattern. Provider TOML files and JSON schema updated consistently.
  • test/aws_kms.bats, test/aws_secrets_manager.bats, test/aws_parameter_store.bats: Migrated from real-AWS credentials to LocalStack. Real-AWS credential checks replaced with LOCALSTACK_ENDPOINT guards and health-check retry loops; all aws CLI invocations receive --endpoint-url "$LOCALSTACK_ENDPOINT". The test for wrong-key encryption correctly asserts failure and verifies error output contains the provider name.
  • .github/workflows/ci.yml: LocalStack services expanded from sts,iam to sts,iam,kms,secretsmanager,ssm. KMS key/alias and Secrets Manager secret are pre-created after health check, and LOCALSTACK_ENDPOINT is persisted to $GITHUB_ENV for the test step.
  • The # bats file_tags=expensive tag was removed from aws_secrets_manager.bats, allowing tests to run on release-plz PRs where they were previously skipped (reflecting the shift to cheaper LocalStack-based tests instead of real AWS).

All changes are focused and correct. The production code changes are minimal and well-scoped; the endpoint plumbing is symmetric across all three AWS providers. Test migration is thorough and consistent.

Confidence Score: 5/5

  • All changes are focused, correct, and well-tested. The production security fix (hard fail on encryption errors) is the intended behaviour and properly scoped. AWS test migration to LocalStack is thorough and consistent across all three providers.
  • Confidence 5/5 — The PR accomplishes exactly what it sets out to do with minimal, correct code changes. The security fix (encryption hard-fail) is explicitly described in the PR and is the right behaviour. The endpoint plumbing is idiomatic and symmetric across all AWS providers. Test migration is comprehensive and includes proper health checks, guard clauses, and assertions. No logic bugs or inconsistencies detected.
  • No files require special attention. All changes are correct and well-integrated.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["fnox set KEY value --provider P"] --> B{"provider is<br/>encryption type?"}
    B -- No --> C{"provider is<br/>remote storage?"}
    C -- No --> D["Store plaintext in fnox.toml"]
    C -- Yes --> E["provider.put_secret(key, value)"]
    E -- error --> F["Propagate error ❌"]
    E -- ok --> G["Store remote key ref in fnox.toml"]
    B -- Yes --> H{"dry-run?"}
    H -- Yes --> I["Store '<encrypted>' placeholder"]
    H -- No --> J["provider.encrypt(value)"]
    J -- error --> K["Hard fail — propagate error ❌<br/>(previously: warn + store plaintext)"]
    J -- ok --> L["Store base64 ciphertext in fnox.toml"]

    subgraph AWS Providers
        M["AwsKmsProvider<br/>(encrypt)"] --> N{"endpoint set?"}
        N -- Yes --> O["Build client with endpoint_url<br/>e.g. http://localhost:4566"]
        N -- No --> P["Build client with default AWS endpoint"]
    end

    J --> M
Loading

Last reviewed commit: 1c4aeae

Comment thread test/aws_kms.bats
Comment thread test/aws_parameter_store.bats
jdx and others added 2 commits March 8, 2026 00:14
- Add back KMS error-path test using nonexistent key alias
- Add LocalStack health-check polling to Parameter Store setup_file()
- Remove dummy AWS credentials from GITHUB_ENV (tests set their own)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Previously, if an encryption provider (age, aws-kms, azure-kms, etc.)
failed to encrypt a value, fnox would silently warn and store the secret
as plaintext. This is a security issue — encryption failures should be
hard errors, not silent fallbacks.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Comment thread src/commands/set.rs
Comment on lines +150 to +153
// Encrypt with the provider — fail on error rather than
// silently storing plaintext
let encrypted = provider.encrypt(value).await?;
(Some(encrypted), None)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is scoped as a test-infrastructure refactor (refactor(aws): use LocalStack for all AWS tests), but this change in set.rs affects production behavior for all users, not just tests.

Previously, a failed KMS encryption call would warn and fall back to storing plaintext (lines 150-156 in the prior version). Now it propagates the error via await? (line 152).

This is the correct security behavior (silent plaintext fallback was a security hole), but it is a breaking change: any user with a transient KMS error or a misconfigured key will now get a hard failure from fnox set instead of only a warning. This warrants a semver-minor or semver-major bump and a changelog entry, not just an incidental commit in a test refactor PR.

Fix in Claude Code

@jdx jdx changed the title refactor(aws): use LocalStack for all AWS tests fix(set): error on encryption failure; use LocalStack for AWS tests Mar 8, 2026
@jdx
Copy link
Copy Markdown
Owner Author

jdx commented Mar 8, 2026

bugbot run

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Comment thread .github/workflows/ci.yml
Comment on lines 119 to +144
@@ -126,6 +126,22 @@ jobs:
curl -sf http://localhost:8200/v1/sys/health && break
sleep 1
done
# Setup LocalStack resources for AWS provider tests
export AWS_ACCESS_KEY_ID=test
export AWS_SECRET_ACCESS_KEY=test
export AWS_DEFAULT_REGION=us-east-1
# Create KMS key for tests
LOCALSTACK_KMS_KEY_ID=$(aws --endpoint-url http://localhost:4566 kms create-key \
--region us-east-1 --query 'KeyMetadata.KeyId' --output text)
aws --endpoint-url http://localhost:4566 kms create-alias \
--alias-name alias/fnox-testing \
--target-key-id "$LOCALSTACK_KMS_KEY_ID" \
--region us-east-1
# Create shared test secret in Secrets Manager
aws --endpoint-url http://localhost:4566 secretsmanager create-secret \
--name "fnox/test-secret" \
--secret-string "This is a test secret in AWS Secrets Manager!" \
--region us-east-1
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LocalStack service health check doesn't verify per-service readiness

The existing health-check loop (lines 120–123) only validates that the LocalStack container returns HTTP 200 from /_localstack/health. However, LocalStack reports container health before individual services like kms, secretsmanager, and ssm have finished initializing — the JSON response body will have those services in "starting" state, not "available" or "running", immediately after the health endpoint first starts responding.

This means aws kms create-key (line 134) can be called while KMS is still starting up, causing the command to fail and the entire CI step to exit non-zero. The result is an intermittent CI failure on cold runners where LocalStack takes longer to start each service.

A more robust check would verify per-service availability:

# Wait for LocalStack services to be ready
for _i in $(seq 1 30); do
  if curl -sf http://localhost:4566/_localstack/health | \
      python3 -c "
import sys, json
data = json.load(sys.stdin)
services = data.get('services', {})
required = ['kms', 'secretsmanager', 'ssm']
all_ok = all(services.get(s) in ('available', 'running') for s in required)
sys.exit(0 if all_ok else 1)
" 2>/dev/null; then
    break
  fi
  sleep 2
done

Without this, the kms create-keycreate-aliassecretsmanager create-secret block will fail intermittently on CI runners, causing the LOCALSTACK_ENDPOINT env var to never be exported and all downstream AWS tests to be silently skipped rather than run.

Fix in Claude Code

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Comment thread test/aws_kms.bats
@@ -1,61 +1,72 @@
#!/usr/bin/env bats
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bats file_tags=expensive tag silently removed

The # bats file_tags=expensive line was removed from this file. Looking at .github/workflows/ci.yml:

BATS_FILTER_TAGS: ${{ (github.event_name != 'pull_request' || !startsWith(github.head_ref, 'release-plz')) && '!expensive' || '' }}

The filter excludes expensive-tagged tests on release-plz PRs. By removing this tag, the Secrets Manager tests will now run on release-plz PRs where they previously were skipped. This may be intentional (LocalStack tests are much cheaper than real AWS), but it's a behaviour change worth calling out — if LocalStack isn't available in the release-plz environment, these tests will attempt to run and then skip (or fail) rather than being excluded at the tag level.

Fix in Claude Code

@jdx jdx enabled auto-merge (squash) March 8, 2026 00:47
- Use plain provider in default_provider override tests (these test
  config layering, not encryption, so fake age recipients broke them)
- Assert error output content in wrong-key tests to verify failures are
  caused by the expected provider error, not unrelated issues

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jdx jdx merged commit d87de3a into main Mar 8, 2026
16 checks passed
@jdx jdx deleted the refactor/localstack-aws-tests branch March 8, 2026 01:14
jdx pushed a commit that referenced this pull request Mar 8, 2026
### 🐛 Bug Fixes

- **(set)** error on encryption failure; use LocalStack for AWS tests by
[@jdx](https://github.com/jdx) in
[#324](#324)

### 📦️ Dependency Updates

- add Cross.toml to install libudev-dev for linux cross-compilation by
[@jdx](https://github.com/jdx) in
[#326](#326)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant