-
-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Controller leaks Vault token leases #376
Description
Describe the solution you'd like
There are a few potential solutions (ordered by personal preference):
- Cache the clients created for the SecretStores
- Revoke the acquired token as a part of the
Closemethod
What is the added value?
Right now, the controller creates a new Vault token on each reconcile call if they aren't using token auth (which is less secure than the other options). By default, Vault auth backends will issue a service token with a TTL of 768 hours (32 days). As long as a service token has not been revoked or expired, Vault will maintain a lease for that token in the backend. This resulted in the number of leases in our Vault cluster eventually building up to the point that the cluster was unable to function anymore. Before #169 was fixed this took our cluster completely down in the space of a few days.
This could be worked around by using a short-lived service token or batch token, but this would be counter-intuitive for most users since the controller is a long-running service and require users to configure their Vault in a particular way to support this tool.
Give us examples of the outcome
- The Vault provider client is cached between reconciliations and handles renewing the token automatically with something like the provided LifetimeWatcher
- The controller makes a call to revoke the token when it's done
Observations (Constraints, Context, etc):
N/A