Skip to content

Missing Region error when authenticating to AWS with IRSA ServiceAccount in 0.18.0 #4934

@spiarh

Description

@spiarh

Describe the bug

We are testing 0.18.0 and facing an issue with an AWS ClusterSecretStore (auth with IRSA ServiceAccount) that is not being synced due to authentication errors. This issue does not occur with 0.17.0.

This shows up in the logs:

could not validate provider: failed to refresh cached credentials, failed to retrieve credentials, operation error STS: AssumeRoleWithWebIdentity, failed to resolve service endpoint, endpoint rule error, Invalid Configuration: Missing Region

I think this is due to the migration to AWS go sdk v2 where spec.provider.region is somehow not being used.

A workaround for 0.18.0 is to set AWS_REGION=eu-central-1 environment variable in the external-secrets pods:

env:                 
- name: AWS_REGION   
  value: eu-central-1

To Reproduce

  1. Create the following
apiVersion: external-secrets.io/v1
kind: ClusterSecretStore
metadata:
  name: aws-secretstore
spec:
  provider:
    aws:
      auth:
        jwt:
          serviceAccountRef:
            name: external-secrets-aws-sm
            namespace: external-secrets
      region: eu-central-1
      service: SecretsManager
  1. The ClusterSecretStore uses the following ServiceAccount
apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::1111111111111111:role/external-secrets
  name: external-secrets-aws-sm
  namespace: external-secrets

The AWS Role can have broad IAM permissions:

{
    "Statement": [
        {
            "Action": [
                "secretsmanager:ListSecretVersionIds",
                "secretsmanager:GetSecretValue",
                "secretsmanager:GetResourcePolicy",
                "secretsmanager:DescribeSecret"
            ],
            "Effect": "Allow",
            "Resource": "*"
        }
    ],
    "Version": "2012-10-17"
}
  1. The AWS ClusterSecretStore now shows as not valid and can't sync secrets.

Expected behavior
The ClusterSecretStore should be successfully validated.

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugCategorizes issue or PR as related to a bug.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions