GCP regional PushSecrets are not working #4656
Description
Describe the bug
A recent change in the external-secrets operator (commit d932fce) introduced support for GCP regional secrets. When using the PushSecrets feature with a GCP secret store configured for regional secrets, we encounter an error related to the replication configuration.
Error Message Example
status:
conditions:
- lastTransitionTime: "2025-04-14T13:49:13Z"
message: 'set secret failed: could not write remote ref password to target secretstore
gcp-store: rpc error: code = InvalidArgument desc = Secret.replication
should not be provided.'
reason: Errored
status: "False"
type: Ready
syncedPushSecrets:
ClusterSecretStore/gcp-store: {}Background & Analysis:
-
Global Secrets (without location):
Prior to commit d932fce), the location was used to set aUserManagedreplication policy. In this case, replication was explicitly configured. When the location was not defined, the replication policy was set toAutomatic.
(Reference in the GCP Client: Global Secret Handling) -
Regional Secrets (with location):
When now the location is provided (intended to signify regional secrets), the new logic still applies a replication configuration. However, for regional secrets on GCP, replication is implicitly defined by the selected region, and manually setting the replication field is not required and leads to errors.
(Reference in the GCP Client: Regional Secret Handling)
To Reproduce
- Environment Setup:
- Use a Kubernetes cluster with Workload Identity configured.
- Ensure that GCP Secret Manager contains a secret with the corresponding key (e.g., your-secret-key).
- Install ExternalSecrets v0.15.1:
- Deploy the ExternalSecrets controller using version v0.15.1.
- Deploy ClusterSecretStore:
Use a configuration similar to the following (with internal names replaced) (the service account has the role roles/secretmanager.admin):
apiVersion: external-secrets.io/v1beta1
kind: ClusterSecretStore
metadata:
name: <your-clustersecretstore-name>
spec:
provider:
gcpsm:
auth:
workloadIdentity:
clusterLocation: <your-cluster-location>
clusterName: <your-cluster-name>
serviceAccountRef:
name: <your-serviceaccount-name>
namespace: <your-serviceaccount-namespace>
location: <your-cluster-location>
projectID: <your-gcp-project-id>- Deploy following resources:
apiVersion: generators.external-secrets.io/v1alpha1
kind: Password
metadata:
name: test-pwgen
spec:
allowRepeat: true
digits: 5
length: 32
noUpper: false
symbolCharacters: -_
symbols: 5
---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: test-password
spec:
dataFrom:
- sourceRef:
generatorRef:
apiVersion: generators.external-secrets.io/v1alpha1
kind: Password
name: test-pwgen
refreshInterval: 0s
target:
creationPolicy: Owner
deletionPolicy: Retain
name: test-password-generated
---
apiVersion: external-secrets.io/v1alpha1
kind: PushSecret
metadata:
name: test-secret
spec:
data:
- conversionStrategy: None
match:
remoteRef:
remoteKey: test-secret
secretKey: password
metadata:
apiVersion: kubernetes.external-secrets.io/v1alpha1
kind: PushSecretMetadata
spec:
labels:
managed-by: external-secrets
deletionPolicy: None
refreshInterval: 1h
secretStoreRefs:
- kind: ClusterSecretStore
name: <your-clustersecretstore-name>
selector:
secret:
name: test-password-generated
updatePolicy: Replace- Get the error message
$ kubectl describe pushsecrets.external-secrets.io test-secret
(...)
Warning Errored 45s (x5 over 98s) pushsecret set secret failed: could not write remote ref password to target secretstore gcp-store: rpc error: code = InvalidArgument desc = Secret.replication should not be provided.Expected behavior
-
For Global Secrets:
- Continue to support user-managed replication when no location is specified.
-
For Regional Secrets:
- Avoid setting the replication configuration if a location is defined.
Additional context
Synchronising regional secrets works fine, only using PushSecrets does not work.
Please let me know if further details (logs, configuration excerpts, etc.) are needed.