After tinkering about with this issue (#162), we realize we were still experiencing this issue when creating a stand-alone ExternalSecret CRD that is not touched/managed by any other controller.

The secret is retrieved successfully but the controller seems to be generating an excessive amount of Vault tokens. I can query for an external secret and the status.refreshTime field would change within seconds. Doing a get -w on the external-secret will output new rows too.

Checking the metrics externalsecret_sync_calls_total shows a curious amount of calls. Not sure how ours compare to everyone else's but 200 increase in sync_calls over 15 minutes seems excessive at around 13 calls per minute (1 call every 5 seconds-ish).

Seems like vault.go:NewClient.setAuth() is called for each new client along with a new token on Vault's side.

The above is called for each externalsecret_controller.go:Reconcile() here

Later in the same function, we retrieve the refreshInterval here but I'm unsure about the syntax of .Duration. I thought it would just be dur = externalSecret.Spec.RefreshInterval instead (without .Duration).

Then I believe this is where I see my status.refreshTime gets updated even though my refreshInterval is configured for 24h. it is surprising to see a new token to be created first even though the refreshInterval has not been reached yet.

I'm not too familiar with Go yet so I'm probably missing something obvious and it may be a configuration issue on my end... Could someone provide some insight on my interpretation of the controller and the amount of tokens generated by the controller? The amount generated is causing issues with our Vault since we have multiple clusters each generating around 2k~4k tokens despite having TTL of 5m and spec.refreshInterval of 24h. Thank you!!

Below are manifests for this example.

❯ kubectl get externalsecrets.external-secrets.io -n falco test -o json | jq '.status'
{
  "conditions": [
    {
      "lastTransitionTime": "2021-05-26T23:50:41Z",
      "message": "Secret was synced",
      "reason": "SecretSynced",
      "status": "True",
      "type": "Ready"
    }
  ],
  "refreshTime": "2021-05-27T00:01:08Z"
}
❯ kubectl get externalsecrets.external-secrets.io -n falco test -o json | jq '.status'
{
  "conditions": [
    {
      "lastTransitionTime": "2021-05-26T23:50:41Z",
      "message": "Secret was synced",
      "reason": "SecretSynced",
      "status": "True",
      "type": "Ready"
    }
  ],
  "refreshTime": "2021-05-27T00:01:17Z"
}

# Each row is generated a few seconds after the other.
❯ kubectl get externalsecrets.external-secrets.io -n falco test -w
NAME   STORE               REFRESH INTERVAL
test   containerservices   24h
test   containerservices   24h
test   containerservices   24h
...
...

Screenshot from metrics

ExternalSecret manifest

apiVersion: external-secrets.io/v1alpha1
kind: ExternalSecret
metadata:
  name: test
  namespace: falco
spec:
  data:
  - remoteRef:
      key: rancher/shared_cluster_data/aws-falco
      property: AWS_ACCESSKEYID
    secretKey: AWS_ACCESSKEYID
  - remoteRef:
      key: rancher/shared_cluster_data/aws-falco
      property: AWS_SECRETACCESSKEY
    secretKey: AWS_SECRETACCESSKEY
  - remoteRef:
      key: rancher/shared_cluster_data/aws-falco
      property: AWS_REGION
    secretKey: AWS_REGION
  - remoteRef:
      key: rancher/shared_cluster_data/aws-falco
      property: AWS_SQS_URL
    secretKey: AWS_SQS_URL
  refreshInterval: 24h
  secretStoreRef:
    kind: ClusterSecretStore
    name: containerservices
  target:
    name: test
status:
  conditions:
  - lastTransitionTime: "2021-05-27T00:07:49Z"
    message: Secret was synced
    reason: SecretSynced
    status: "True"
    type: Ready
  refreshTime: "2021-05-27T00:26:51Z"

ClusterSecretstore manifest

apiVersion: external-secrets.io/v1alpha1
kind: ClusterSecretStore
metadata:
  name: containerservices
spec:
  controller: default
  provider:
    vault:
      auth:
        kubernetes:
          mountPath: containerservices_usw1-testingcluster
          role: kube-webhook-role
          serviceAccountRef:
            name: external-secrets-kubernetes-external-secrets
            namespace: vault-infra
      caBundle: 123123
      path: secret
      server: https://vault...io
      version: v2

Version: ghcr.io/external-secrets/external-secrets:v0.1.3