Fail to sync with Bitwarden Secrets Manager (status code 400) #4824
Description
Describe the bug
I've discovered recently that ESO fails to login in to Bitwarden Secrets Manager. This has been working for a while, but now doesn't.
I get the following error:
{
"level": "error",
"ts": 1747866352.9282637,
"msg": "Reconciler error",
"controller": "externalsecret",
"controllerGroup": "external-secrets.io",
"controllerKind": "ExternalSecret",
"ExternalSecret": {
"name": "s3-env",
"namespace": "media"
},
"namespace": "media",
"name": "s3-env",
"reconcileID": "c2eb52d7-97a5-4f4a-b1b0-3c01e923d4b0",
"error": "error processing spec.dataFrom[0].extract, err: failed to get secret: failed to get secret: failed to perform http request, got response: failed to login to bitwarden using access token: bitwarden login: API error: error sending request for url (https://identity.bitwarden.com/connect/token)\n with status code 400",
"stacktrace": "sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.4/pkg/internal/controller/controller.go:347\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.4/pkg/internal/controller/controller.go:294\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.4/pkg/internal/controller/controller.go:255"
}I have confirmed that my token works:
bws secret get --access-token $(kubectl get secret -n external-secrets bitwarden-access-token -o yaml | yq '.data.token' | base64 -d) 52d8368a-6ce4-482f-a9cd-b2a000334e75
{
"id": "52d8368a-6ce4-482f-a9cd-b2a000334e75",
"organizationId": "XXXXXXXX-a7df-4134-96df-XXXXXXXXXXXX",
"projectId": "XXXXXXXX-d703-453e-a58e-XXXXXXXXXXXX",
"key": "cloudnativepg-db-backup-s3",
"value": "accessKeyId: \"XXXXXXXXXXX\"\nsecretAccessKey: \"XXXXXXXXXXXXX\"",
"note": "",
"creationDate": "2025-03-14T03:06:48.066014600Z",
"revisionDate": "2025-03-14T03:11:00.840777100Z"
}To Reproduce
Steps to reproduce the behavior:
- provide all relevant manifests
ClusterSecretStore Resource
apiVersion: external-secrets.io/v1
kind: ClusterSecretStore
metadata:
creationTimestamp: "2025-03-10T16:30:16Z"
generation: 4
labels:
argocd.argoproj.io/instance: external-secrets
name: bitwarden-secretsmanager
resourceVersion: "393669591"
uid: 9bdcaee9-21a1-4e96-bf46-9b7d9391de37
spec:
provider:
bitwardensecretsmanager:
auth:
secretRef:
credentials:
key: token
name: bitwarden-access-token
namespace: external-secrets
bitwardenServerSDKURL: https://bitwarden-sdk-server.external-secrets.svc.cluster.local:9998
caProvider:
key: ca.crt
name: bitwarden-tls-certs
namespace: external-secrets
type: Secret
organizationID: XXXXXXXX-a7df-4134-96df-XXXXXXXXXXXX
projectID: XXXXXXXX-d703-453e-a58e-XXXXXXXXXXXXExternal Secret Resource
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
annotations:
creationTimestamp: "2025-04-28T16:59:28Z"
generation: 1
labels:
argocd.argoproj.io/instance: media
name: s3-env
namespace: media
resourceVersion: "393663441"
uid: cddab853-7f57-41c3-a9fc-a81084718a2e
spec:
dataFrom:
- extract:
conversionStrategy: Default
decodingStrategy: None
key: 52d8368a-6ce4-482f-a9cd-b2a000334e75
metadataPolicy: None
refreshInterval: 1h
secretStoreRef:
kind: ClusterSecretStore
name: bitwarden-secretsmanager
target:
creationPolicy: Owner
deletionPolicy: Delete
name: s3-env
template:
data:
ACCESS_KEY_ID: '{{ .accessKeyId }}'
ACCESS_SECRET_KEY: '{{ .secretAccessKey }}'
engineVersion: v2
mergePolicy: Replace- provide the Kubernetes and ESO version
ESO ➡v0.17.0
bitwarden-sdk-server ➡v0.4.0
Deployed using Helm Chart version0.17.0
Expected behavior
ESO can log in to Bitwarden Secrets and sync secrets