Skip to content

GKE - cannot list resource \"secrets\" in API group \"\" at the cluster scope" #4481

@tasnyccbs

Description

@tasnyccbs

Describe the bug
Running ESO in GKE pulling from Google secrets manager. I am only using namespaced (kind: SecretStore & kind: ExternalSecret )level secrets and no clusterlevel. From the documentation, by setting scopedRBAC: true and scopedNamespace: my-namespace this should only look within the my-namespace namespace but for some reason when looking at the logs of the controller in the namespace i see

"Unhandled Error" err="pkg/mod/k8s.io/client-go@v0.32.1/tools/cache/reflector.go:251: Failed to watch *v1.Secret: failed to list *v1.Secret: secrets is forbidden: User \"system:serviceaccount:my-namespace:my-ksa\" cannot list resource \"secrets\" in API group \"\" at the cluster scope" logger="UnhandledError"

Also getting "Reconciler error" in logs which i believe is related.

I am not sure why its looking at the cluster scope at all, but constantly getting this error which sometimes causes a pod restart.

To Reproduce
I have three GKE clusters and getting the same errors in each namespace

Expected behavior
I would assume It doesn't need to list from cluster scope since only running in namespace.
If that is in fact the default behavior , it seems to be missing a ClusterRole and ClusterRoleBinding or this is a bug and shouldn't be looking there at all

Additional context
For now to stop it from erroring i have set extraObjects in the values file creating the ClusterRole and ClusterRoleBinding it's requesting for. Unless I'm setting something wrong I feel I shouldn't have to set this.
With all that said, despite the errors eso is working as intended.
Here is what I added for the errors to stop.

extraObjects:
- apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: external-secrets-cluster-role-fix
rules:
- apiGroups:
- ""
resources:
- secrets
verbs:
- get
- list
- watch
- apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: external-secrets-cluster-rolebindin-fix
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: external-secrets-cluster-role-cribl-fix
subjects:
- kind: ServiceAccount
name: eso-ksa
namespace: my-namespace

Maybe i'm doing something wrong, maybe its something with GKE as I am not having these issues with microk8s in home lab. Hopefully it's not something stupid on my end.
Thanks for the help!

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugCategorizes issue or PR as related to a bug.triage/needs-informationIndicates an issue needs more information in order to work on it.

    Type

    Projects

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions