-
-
Notifications
You must be signed in to change notification settings - Fork 1.2k
GKE - cannot list resource \"secrets\" in API group \"\" at the cluster scope" #4481
Description
Describe the bug
Running ESO in GKE pulling from Google secrets manager. I am only using namespaced (kind: SecretStore & kind: ExternalSecret )level secrets and no clusterlevel. From the documentation, by setting scopedRBAC: true and scopedNamespace: my-namespace this should only look within the my-namespace namespace but for some reason when looking at the logs of the controller in the namespace i see
"Unhandled Error" err="pkg/mod/k8s.io/client-go@v0.32.1/tools/cache/reflector.go:251: Failed to watch *v1.Secret: failed to list *v1.Secret: secrets is forbidden: User \"system:serviceaccount:my-namespace:my-ksa\" cannot list resource \"secrets\" in API group \"\" at the cluster scope" logger="UnhandledError"
Also getting "Reconciler error" in logs which i believe is related.
I am not sure why its looking at the cluster scope at all, but constantly getting this error which sometimes causes a pod restart.
To Reproduce
I have three GKE clusters and getting the same errors in each namespace
Expected behavior
I would assume It doesn't need to list from cluster scope since only running in namespace.
If that is in fact the default behavior , it seems to be missing a ClusterRole and ClusterRoleBinding or this is a bug and shouldn't be looking there at all
Additional context
For now to stop it from erroring i have set extraObjects in the values file creating the ClusterRole and ClusterRoleBinding it's requesting for. Unless I'm setting something wrong I feel I shouldn't have to set this.
With all that said, despite the errors eso is working as intended.
Here is what I added for the errors to stop.
extraObjects:
- apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: external-secrets-cluster-role-fix
rules:
- apiGroups:
- ""
resources:
- secrets
verbs:
- get
- list
- watch
- apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: external-secrets-cluster-rolebindin-fix
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: external-secrets-cluster-role-cribl-fix
subjects:
- kind: ServiceAccount
name: eso-ksa
namespace: my-namespace
Maybe i'm doing something wrong, maybe its something with GKE as I am not having these issues with microk8s in home lab. Hopefully it's not something stupid on my end.
Thanks for the help!
Metadata
Metadata
Assignees
Labels
Type
Projects
Status