Skip to content

Allow to get parameters by path for AWS Systems Manager Parameter Store provider #915

@jacekpuczko

Description

@jacekpuczko

I'm looking into migrating from deprecated Kubernetes External Secrets but I found out that the functionality of fetching all parameters by the path from AWS Systems Manager Parameter Store, available in Kubernetes External Secrets, is missing in the External Secrets Operator. It was already reported in:

#431

Currently, I'm using the path prefix in the parameter store to identify all secrets keys owned by a single service and use kubernetes-client.io/ExternalSecret resource to create a k8s secret with multiple keys without a need to specify every key or need to update any k8s resources when I add a new parameter. Because there is a lot of keys per secret and many services it's a huge overhead having to declare every key in the external secret resource.

Instead of having to specify every parameter:

apiVersion: external-secrets.io/v1alpha1
kind: ExternalSecret
metadata:
  name: service-name
spec:
  # [omitted for brevity]
  data:
    - secretKey: PASSWORD1
      remoteRef:
        key: /service-name/PASSWORD1
    - secretKey: PASSWORD2
      remoteRef:
        key: /service-name/PASSWORD2
    # Followed by more
  target:
    name: service-name

I'd like to use the path prefix to fetch all keys for the service (not sure if dataFrom was intended for this purpose)

apiVersion: external-secrets.io/v1alpha1
kind: ExternalSecret
metadata:
  name: service-name
spec:
  # [omitted for brevity]
  dataFrom:
    - key: /service-name/
  target:
    name: service-name

Example of the current usage with Kubernetes external secrets:

apiVersion: 'kubernetes-client.io/v1'
kind: ExternalSecret
metadata:
  name: service-name
  namespace: namespace
spec:
  backendType: systemManager
  data:
    - path: /service-name/
      recursive: false

The proposed alternative solution using a JSON string to store multiple keys wouldn't work for me for 2 reasons:

  1. AWS Systems Manager Parameter Store has a value limit of 4 KB in the free tier, 8 KB for the advanced option, this will be not enough to store all the keys for some of the services.
  2. A JSON, especially a big one, won't be readable in the AWS Web console.

Is there a plan to implement this functionality?
If not, Could you please provide some direction on how to contribute to adding this feature?
Is it possible to implement one using the current externalsecret.external-secrets.io resource, dataFrom element or it would require changes to CRDs? Would the changes be limited to the Parameter store provider adapter (parameterstore.go file)?

Metadata

Metadata

Assignees

Labels

area/awsIndicates an issue or PR related to AWS.kind/featureCategorizes issue or PR as related to a new feature.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions