Support Disabling EC2 Metadata Server on AWS Related SecretStores/Generators #4679
Description
Describe the bug
When trying to issue AWS credentials using STSSessionToken and IRSA (option 2 described here), I get the IAM error AccessDenied: Cannot call GetSessionToken with session credentials
Here is the full logs in debug mode
{
"level": "debug",
"ts": 1744835217.7023861,
"logger": "provider.aws",
"msg": "using credentials via service account",
"role": "arn:aws:iam::...:role/....",
"region": "eu-west-2"
}
{
"level": "info",
"ts": 1744835217.7024636,
"logger": "provider.aws",
"msg": "using aws session",
"region": "eu-west-2",
"credentials": {}
}
{
"level": "debug",
"ts": 1744835217.7027402,
"logger": "provider.aws",
"msg": "fetching token",
"ns": "my-namespace",
"sa": "mysa"
}
{
"level": "error",
"ts": 1744835217.812459,
"msg": "Reconciler error",
"controller": "externalsecret",
"controllerGroup": "external-secrets.io",
"controllerKind": "ExternalSecret",
"ExternalSecret": {
"name": "my-creds",
"namespace": "my-namespace"
},
"namespace": "my-namespace",
"name": "my-creds",
"reconcileID": "35e11311-a822-47a8-9260-9c43acb6f4f4",
"error": "error processing spec.dataFrom[0].sourceRef.generatorRef, err: error using generator: unable to get authorization token: AccessDenied: Cannot call GetSessionToken with session credentials\n\tstatus code: 403, request id: 0505287d-dabd-42f0-a7ce-be3bd6e0705a",
"stacktrace": "sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.4/pkg/internal/controller/controller.go:347\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.4/pkg/internal/controller/controller.go:294\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.4/pkg/internal/controller/controller.go:255"
}
{
"level": "debug",
"ts": 1744835217.812619,
"logger": "events",
"msg": "error processing spec.dataFrom[0].sourceRef.generatorRef, err: error using generator: unable to get authorization token: AccessDenied: Cannot call GetSessionToken with session credentials\n\tstatus code: 403, request id: 0505287d-dabd-42f0-a7ce-be3bd6e0705a",
"type": "Warning",
"object": {
"kind": "ExternalSecret",
"namespace": "my-namespace",
"name": "my-creds",
"uid": "0e53a6a6-5de2-4a69-916e-e9aa374f1a91",
"apiVersion": "external-secrets.io/v1",
"resourceVersion": "1036190"
},
"reason": "UpdateFailed"
}To Reproduce
(Require valid IRSA for the used mysa)
apiVersion: generators.external-secrets.io/v1alpha1
kind: STSSessionToken
metadata:
name: my-generator
spec:
auth:
jwt:
serviceAccountRef:
name: mysa
region: eu-west-2
requestParameters:
sessionDuration: 3600
---
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
name: my-creds
spec:
dataFrom:
- sourceRef:
generatorRef:
apiVersion: generators.external-secrets.io/v1alpha1
kind: STSSessionToken
name: my-generator
refreshInterval: 1m
target:
name: my-credsExpected behavior
The ES my-creds should successfully get credentials
Additional context
Note that the ES controller is itself running with IRSA, using a different role. it isn't clear if the sessionToken that IAM is complaining about is coming from this.