Skip to content

feat(aws): support for aws tags#4538

Merged
Skarlso merged 14 commits intoexternal-secrets:mainfrom
gofogo:feat-aws-tags-1821
Apr 4, 2025
Merged

feat(aws): support for aws tags#4538
Skarlso merged 14 commits intoexternal-secrets:mainfrom
gofogo:feat-aws-tags-1821

Conversation

@ivankatliarchuk
Copy link
Copy Markdown
Contributor

@ivankatliarchuk ivankatliarchuk commented Mar 12, 2025

Problem Statement

What is the problem you're trying to solve?

Related Issue

Partially resolves #1821 . Provides a capability to create secrets with KSM key, Description and Tags.

It does not:

  • modify existing keys if/when tags added/removed
  • provide a way to configure capability to set KSM resource policy
  • secret replication in other region

Proposed Changes

How do you like to solve the issue and why?

Added support to create a secret with

  • tags
  • description
  • kms key default and non-default

very similar to

There going to be a breaking change, as before it was

metada:
   secretPushFormat: string

now

      metadata:
        apiVersion: kubernetes.external-secrets.io/v1alpha1
        kind: PushSecretMetadata
        spec:
          secretPushFormat: string # When not set, default to binary 

It is possible to support both, with slightly more code complexity. My understanding, from reading design docs https://github.com/external-secrets/external-secrets/blob/main/design/010-pushsecret-metadata.md v1alpha1 should support both, or we need to bump version of the API?

This PR only covers new secret creation. To support secrets that already managed with secrets operator, I'm planning to open a second pull request. As the code is slightly more complex.

In follow-up I'll have a look how easy is to support Secrets resources policies and region replications.

Tested in the account. Manifests below

apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: secretstore-sample-ik
spec:
  provider:
    aws:
      service: SecretsManager
      region: eu-west-1
      secretsManager:
        forceDeleteWithoutRecovery: true
      auth:
        secretRef:
          accessKeyIDSecretRef:
            name: awssm-secret
            key: access-key
          secretAccessKeySecretRef:
            name: awssm-secret
            key: secret-access-key
---
apiVersion: generators.external-secrets.io/v1alpha1
kind: Password
metadata:
  name: my-password
  namespace: external-secrets
spec:
  length: 12
  digits: 5
  symbols: 5
  symbolCharacters: "-_$@"
  noUpper: false
  allowRepeat: true
---
apiVersion: external-secrets.io/v1alpha1
kind: PushSecret
metadata:
  name: pushsecret-to-aws-example # Customisable
  namespace: external-secrets # Same of the SecretStores
  labels:
    this-is-the-label: "lol"
  annotations:
    this-is-the-annotation: "haha"
spec:
  deletionPolicy: Delete
  refreshInterval: 1m # Refresh interval for which push secret will reconcile
  secretStoreRefs: # A list of secret stores to push secrets to
    - name: secretstore-sample-ik
      kind: SecretStore
  selector:
    generatorRef:
      apiVersion: generators.external-secrets.io/v1alpha1
      kind: Password
      name: my-password
  template:
    metadata:
      annotations:
        a-key2: value1
      labels:
        l-key2: value1
        pp.kubernetes.io/part-of: testing
  data:
    - conversionStrategy: None
      match:
        secretKey: password # Source Kubernetes secret key to be pushed
        remoteRef:
          remoteKey: teamb-my-first-parameter-6 # Remote reference (where the secret is going to be pushed)
      metadata:
        apiVersion: kubernetes.external-secrets.io/v1alpha1
        kind: PushSecretMetadata
        spec:
          kmsKeyID: bb123123-b2b0-4f60-ac3a-44a13f0e6b6c
          secretPushFormat: string
          description: "this is key description"
          tags: # Tags to be added to the secret in Azure Key Vault
            secret-store: teamb-secret-store
            refresh-interval: 1h

Secret is created with Tags, Description, and user specieif kmsKeyID

Checklist

Test with CLI

❯❯ aws secretsmanager list-secrets
❯❯ aws kms list-aliases
❯❯ aws kms list-keys

Signed-off-by: ivan katliarchuk <ivan.katliarchuk@gmail.com>
@ivankatliarchuk ivankatliarchuk requested a review from a team as a code owner March 12, 2025 19:47
@ivankatliarchuk ivankatliarchuk requested a review from Skarlso March 12, 2025 19:47
@ivankatliarchuk ivankatliarchuk marked this pull request as draft March 12, 2025 19:47
@Skarlso
Copy link
Copy Markdown
Contributor

Skarlso commented Mar 15, 2025

Thanks for doing this. Do you require me to look at it, or are you still working on things here? :)

@ivankatliarchuk
Copy link
Copy Markdown
Contributor Author

I will be working on it thanks. Trying to better understand the setup.

* main:
  chore: update helm charts to v0.15.0 (external-secrets#4566)
  feat: cluster push secret with pushing all secrets from a namespace (external-secrets#4162)
  fix: check if secret is being deleted during fetch (external-secrets#4562)
  feat: add cloud.ru secret manager support (external-secrets#3716)
  update dependencies (external-secrets#4556)
  feat: non standard templating delimiters (external-secrets#4558)
  chore(deps): bump distroless/static from `3f2b64e` to `95ea148` (external-secrets#4554)
  Fix Grafana generator not passing desired SA role to creation request (external-secrets#4533)
  fix: define top level permissions and fix token scope (external-secrets#4543)
  chore(deps): bump ubi8/ubi from `ecbeb81` to `5993454` (external-secrets#4553)
  fix: skip none-existing keys (external-secrets#4517)
  chore(deps): bump golangci/golangci-lint-action from 6.5.0 to 6.5.1 (external-secrets#4552)
  chore(deps): bump docker/login-action from 3.3.0 to 3.4.0 (external-secrets#4551)
  chore(deps): bump aquasecurity/trivy-action from 0.29.0 to 0.30.0 (external-secrets#4550)
  chore(deps): bump mkdocs-material from 9.6.7 to 9.6.8 in /hack/api-docs (external-secrets#4555)
  Update full-pushsecret.yaml (external-secrets#4547)
  feat: add metadata setting to encode secrets as decoded values (external-secrets#4535)
  feat: introduce codeql scan for code sections (external-secrets#4198)
  Improve Grafana generator integration with in-cluster Grafana (external-secrets#4519)
Signed-off-by: ivan katliarchuk <ivan.katliarchuk@gmail.com>
Signed-off-by: ivan katliarchuk <ivan.katliarchuk@gmail.com>
* main:
  Clarify that setting `spec.refreshInterval` to 0 disables all update behaviour (external-secrets#4567)
  Helm: disable ClusterPushSecret reconciler when using scoped RBAC (external-secrets#4571)
  Exclude unused resources from rbac (external-secrets#4572)
Signed-off-by: ivan katliarchuk <ivan.katliarchuk@gmail.com>
Signed-off-by: ivan katliarchuk <ivan.katliarchuk@gmail.com>
Signed-off-by: ivan katliarchuk <ivan.katliarchuk@gmail.com>
* main:
  update dependencies (external-secrets#4589)
  chore(deps): bump fossas/fossa-action from 1.5.0 to 1.6.0 (external-secrets#4586)
  chore(deps): bump mkdocs-material from 9.6.8 to 9.6.9 in /hack/api-docs (external-secrets#4588)
  chore(deps): bump platformdirs from 4.3.6 to 4.3.7 in /hack/api-docs (external-secrets#4587)
  chore(deps): bump github/codeql-action from 3.28.11 to 3.28.12 (external-secrets#4585)
  chore(deps): bump actions/setup-go from 5.3.0 to 5.4.0 (external-secrets#4584)
  chore(deps): bump golangci/golangci-lint-action from 6.5.1 to 6.5.2 (external-secrets#4583)
  chore(deps): bump actions/cache from 4.2.2 to 4.2.3 (external-secrets#4582)
  chore(deps): bump ubi8/ubi from `5993454` to `8bd1b63` (external-secrets#4581)
  Lookup cluster identity from instance metadata (external-secrets#4575)
  adding conjur description (external-secrets#4578)
  fix: bump jwt for cve fix (external-secrets#4580)
  Add `webhook.create: false` warning comment (external-secrets#4579)
Signed-off-by: ivan katliarchuk <ivan.katliarchuk@gmail.com>
@ivankatliarchuk
Copy link
Copy Markdown
Contributor Author

Hi @Skarlso .

So it looks like I need some help with unit testing. Basically I not fully understand the framework. Example tests I would like to have, but not sure how to.

  1. Validate that secret contains required metadata field. When I push secret to fake store, not sure how to retieve it for validation
  2. Validate the supplied input before the secret pushed to AWS/fake-store.
  3. constructMetadataWithDefaults(). It looks like, only public methods currently unit-tested. No issue with that, but how to reliably test that for example constructMetadataWithDefaults() configure default values if not set?

@Skarlso
Copy link
Copy Markdown
Contributor

Skarlso commented Mar 25, 2025

I'll take a look. :)

Signed-off-by: ivan katliarchuk <ivan.katliarchuk@gmail.com>
@Skarlso
Copy link
Copy Markdown
Contributor

Skarlso commented Mar 26, 2025

@ivankatliarchuk Need anything else with the tests? Or is this now ready to be reviewed? :)

@ivankatliarchuk
Copy link
Copy Markdown
Contributor Author

Is ready to an extend. Pls review

@ivankatliarchuk ivankatliarchuk marked this pull request as ready for review March 26, 2025 20:39
@ivankatliarchuk ivankatliarchuk requested a review from Skarlso March 26, 2025 20:39
Signed-off-by: ivan katliarchuk <ivan.katliarchuk@gmail.com>
Copy link
Copy Markdown
Contributor

@Skarlso Skarlso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me just one tiny comment.

Signed-off-by: ivan katliarchuk <ivan.katliarchuk@gmail.com>
@ivankatliarchuk ivankatliarchuk requested a review from Skarlso March 27, 2025 07:58
@gusfcarvalho
Copy link
Copy Markdown
Member

/ok-to-test sha=df3730b5a80092a35082fe5efd6c86c06e6baf7d

@eso-service-account-app
Copy link
Copy Markdown
Contributor

@Skarlso
Copy link
Copy Markdown
Contributor

Skarlso commented Apr 4, 2025

@ivankatliarchuk Hello. If you update your branch this is ready to merge. :)

@ivankatliarchuk
Copy link
Copy Markdown
Contributor Author

done

@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud bot commented Apr 4, 2025

@Skarlso Skarlso merged commit 87afb67 into external-secrets:main Apr 4, 2025
3 checks passed
@ivankatliarchuk ivankatliarchuk deleted the feat-aws-tags-1821 branch April 4, 2025 06:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PushSecret: option to add additional tags\labels

3 participants