GCP Provider- GetSecret method successfully parses broken JSON - it should fail. #4336
Description
Is your feature request related to a problem? Please describe.
It would be nice if the dataFrom / extract form of an ExternalSecret could handle a trailing comma, a la:
{
"TEST1": "abc123",
}
With a trailing comma, ESO produces the error:
unable to unmarshal secret: invalid character '}' looking for beginning of object key string
(Technically, this is indeed invalid JSON, according to https://www.rfc-editor.org/rfc/rfc4627. However, allowing trailing commas would make maintaining JSON secrets a bit easier for humans; and many JSON parsers do accept trailing commas -- including the External Secrets Operator data / secretKey parser, which happily pulls each property out of a comma-tailed JSON secret.)
Describe the solution you'd like
Allow JSON with a trailing comma in the dataFrom / extract code: { "TEST1": "abc123", }
Describe alternatives you've considered
This would just be a nicety. Creating a JSON secret with no trailing comma works fine: { "TEST1": "abc123" }
Additional context
If accepted, and if someone could point me in the direction of the JSON parsing code for dataFrom / extract (even if it's a third party library), I'd be happy to take a shot at a PR to relax the no-trailing-comma rule.
Full ExternalSecret example:
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: johann-test-secrets
namespace: default
spec:
refreshInterval: 5m # rate External Secret Operator pulls from GSM
secretStoreRef:
kind: ClusterSecretStore
name: gcp-sa-connectivity # name of the SecretStore (or kind specified)
target:
name: johann-test-secrets # name of the k8s Secret to be created
creationPolicy: Owner
#
# https://external-secrets.io/latest/guides/all-keys-one-secret/
#
dataFrom:
- extract:
key: johann-test # Google Secret Manager Secret
version: "1"
In Google Secret Manager "johann-test" secret version "1" is:
{
"TEST1": "abc123",
}
Error logs:
{
"insertId": "mq2flcgbkj1zrklr",
"jsonPayload": {
"logger": "controllers.ExternalSecret",
"stacktrace": "github.com/external-secrets/external-secrets/pkg/controllers/externalsecret.(*Reconciler).markAsFailed\n\t/home/runner/work/external-secrets/external-secrets/pkg/controllers/externalsecret/externalsecret_controller.go:359\ngithub.com/external-secrets/external-secrets/pkg/controllers/externalsecret.(*Reconciler).Reconcile\n\t/home/runner/work/external-secrets/external-secrets/pkg/controllers/externalsecret/externalsecret_controller.go:228\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.0/pkg/internal/controller/controller.go:116\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.0/pkg/internal/controller/controller.go:303\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.0/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.0/pkg/internal/controller/controller.go:224",
"msg": "could not get secret data from provider",
"ExternalSecret": {
"name": "johann-test-secrets",
"namespace": "default"
},
"ts": 1737727568.8965504,
"level": "error",
"error": "unable to unmarshal secret: invalid character '}' looking for beginning of object key string"
},
...etc...
Thank you ESO folks, you rock! 🙏
Metadata
Metadata
Assignees
Labels
Type
Projects
Status