Describe the bug
I have a webhook ClusterSecretStore in which I declared a templated result.jsonPath expression, that looks like this:
jsonPath: "$.data.attributes.{{ or .remoteRef.property \"password\" }}"Using this construct, I can use different properties of the returned result in my ExternalSecret (and when no property is provided, password is used by default). This works fine when I source a value in my ExternalSecret using data. However, when I use dataFrom, I get this strange error:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning UpdateFailed 3m58s (x17 over 9m35s) external-secrets failed to get response path $.data.attributes.{{ or .remoteRef.property "password" }}: parsing error: $.data.attributes.{{ or .remoteRef.property "password" }} :1:19 - 1:20 unexpected "{" while scanning JSON select expected Ident, "." or "*"
In the API reference, I see that spec.data[*].remoteRef and spec.dataFrom[*].extract reference the same data types (not sure if that's the correct term?). And since I am using remoteRef in the jsonPath template, that seems off because there is no remoteRef reference for dataFrom sources.
I do notice that at the HTTP endpoint, a request is made with the correct name GET parameter, so the retrieval part works (so at least remoteRef.key seems to exist).
To Reproduce
My webhook ClusterSecretStore, where I use the result.jsonPath:
apiVersion: external-secrets.io/v1beta1
kind: ClusterSecretStore
metadata:
name: webhook-test
spec:
provider:
webhook:
url: https://example.com/secret?name={{ .remoteRef.key }}
timeout: 1s
result:
jsonPath: "$.data.attributes.{{ or .remoteRef.property \"password\" }}"And the ExternalSecret with the failing dataFrom extraction:
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: datafrom-test
spec:
refreshInterval: 24h
secretStoreRef:
kind: ClusterSecretStore
name: webhook-test
dataFrom:
- extract:
key: users-test
conversionStrategy: Default
decodingStrategy: None
metadataPolicy: None
target:
template:
engineVersion: v2
data:
users: |
{{- range $username, $password := . }}
{{ htpasswd $username $password }}
{{- end }}The contents of users-test at example.com is simply:
{
"user-1": "password-1",
"user-2": "password-2"
}Expected behavior
I expect that a datafrom-test secret would contain some htpasswd lines.
Additional context
As mentioned, regular data sourced values work as expected, so there is a way we can work around it:
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: data-test
spec:
refreshInterval: 24h
secretStoreRef:
kind: ClusterSecretStore
name: webhook-test
data:
- secretKey: credentials
remoteRef:
key: users-test
conversionStrategy: Default
decodingStrategy: None
metadataPolicy: None
target:
template:
engineVersion: v2
data:
users: |
{{- range $username, $password := mustFromJson .credentials }}
{{ htpasswd $username $password }}
{{- end }}
Describe the bug
I have a
webhookClusterSecretStorein which I declared a templatedresult.jsonPathexpression, that looks like this:Using this construct, I can use different properties of the returned result in my
ExternalSecret(and when nopropertyis provided,passwordis used by default). This works fine when I source a value in myExternalSecretusingdata. However, when I usedataFrom, I get this strange error:In the API reference, I see that
spec.data[*].remoteRefandspec.dataFrom[*].extractreference the same data types (not sure if that's the correct term?). And since I am usingremoteRefin thejsonPathtemplate, that seems off because there is noremoteRefreference fordataFromsources.I do notice that at the HTTP endpoint, a request is made with the correct
nameGET parameter, so the retrieval part works (so at leastremoteRef.keyseems to exist).To Reproduce
My
webhookClusterSecretStore, where I use theresult.jsonPath:And the
ExternalSecretwith the failingdataFromextraction:The contents of
users-testatexample.comis simply:{ "user-1": "password-1", "user-2": "password-2" }Expected behavior
I expect that a
datafrom-testsecret would contain somehtpasswdlines.Additional context
As mentioned, regular
datasourced values work as expected, so there is a way we can work around it: