Skip to content

Revoke Vault token on Close#381

Merged
knelasevero merged 1 commit intoexternal-secrets:mainfrom
cooperbenson-qz:cooperbenson-qz/issue376
Oct 5, 2021
Merged

Revoke Vault token on Close#381
knelasevero merged 1 commit intoexternal-secrets:mainfrom
cooperbenson-qz:cooperbenson-qz/issue376

Conversation

@cooperbenson-qz
Copy link
Copy Markdown
Contributor

Fixes #376

The Vault client now revokes it's token on Close if one exists and it was obtained via any auth method except for TokenSecretRef

Copy link
Copy Markdown
Contributor

@paul-the-alien paul-the-alien bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Greetings!
Thank you for contributing to this project!
If this is your first time contributing, please make
sure to read the Developer and Contributing Process guides.
Please also mind and follow our Code of Conduct.

@knelasevero
Copy link
Copy Markdown
Member

/ok-to-test

@sonarqubecloud
Copy link
Copy Markdown

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 1 Code Smell

No Coverage information No Coverage information
0.0% 0.0% Duplication

@knelasevero
Copy link
Copy Markdown
Member

/ok-to-test

@knelasevero
Copy link
Copy Markdown
Member

@mcavoyk @cnmcavoy do you see any problem with this approach?

@mcavoyk
Copy link
Copy Markdown
Contributor

mcavoyk commented Sep 24, 2021

The code change looks good to me, but I've yet been unable to verify the leaked token behavior in vault. Would the list token accessors endpoint in Vault show the the leaked tokens, or what way were you able to verify this issue @cooperbenson-qz?

@knelasevero
Copy link
Copy Markdown
Member

Hey @cooperbenson-qz! Do you have more info on how we can verify this issue?

@cooperbenson-qz
Copy link
Copy Markdown
Contributor Author

cooperbenson-qz commented Sep 30, 2021

Apologies, I was out of the office for the last few days. We saw the problem most clearly in the vault.expire.num_leases metrics, though I would expect that API method to show a similar result

@knelasevero
Copy link
Copy Markdown
Member

/approve

@knelasevero
Copy link
Copy Markdown
Member

knelasevero commented Oct 1, 2021

@mcavoyk would you send a slash merge if you think this one is fine? Even if we can't check the issue, revoking the token on every close should be fine, right?

@cooperbenson-qz
Copy link
Copy Markdown
Contributor Author

cooperbenson-qz commented Oct 1, 2021

Looking at the docs, sys/internal/counter/tokens might be an easier way to get the token count.

Even if we can't check the issue, revoking the token on every close should be fine, right?

This is preferable to creating and then abandoning a new token on every reconcile, but it does have the drawback of doubling token-related API calls. This probably won't matter at smaller scales, but I could imagine it causing issues at scale.

@cnmcavoy
Copy link
Copy Markdown
Contributor

cnmcavoy commented Oct 1, 2021

We have a few hundred external secrets and nearly all have a refresh interval set, but haven't seen any particular issues with Vault so far. I tried to check and see if we are affected by this but the Vault version we are running (1.3.5) did not have the metric vault.expire.num_leases or anything similar that seemed to indicate this was a problem with our Vault.

I can update to this after it is merged and report back if we see any regression in behavior on our side.

@mcavoyk
Copy link
Copy Markdown
Contributor

mcavoyk commented Oct 4, 2021

/approve

@mcavoyk
Copy link
Copy Markdown
Contributor

mcavoyk commented Oct 4, 2021

/merge

@knelasevero knelasevero merged commit 91851bb into external-secrets:main Oct 5, 2021
@knelasevero
Copy link
Copy Markdown
Member

Hey @cnmcavoy! Thanks a lot for the response. Also, it is nice to know that you are able to use the project with hundreds of secrets with no problems.

Here is the new release with that merged: https://artifacthub.io/packages/helm/external-secrets-operator/external-secrets

Would be great if you guys could report back any problems that you may find. Thanks!

@cnmcavoy
Copy link
Copy Markdown
Contributor

cnmcavoy commented Oct 5, 2021

I upgraded all our clusters to v0.3.6 and will report back if we see anything funny.

@cooperbenson-qz cooperbenson-qz deleted the cooperbenson-qz/issue376 branch October 7, 2021 18:12
@cooperbenson-qz
Copy link
Copy Markdown
Contributor Author

I've also upgraded our clusters and re-enabled the Vault secret store and secrets, I'll report back with my observations as well

@cooperbenson-qz
Copy link
Copy Markdown
Contributor Author

Just finished testing for a few hours at a 1s refreshInterval and can confirm my lease count is not increasing. So I think my problem is solved by this 😁

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Controller leaks Vault token leases

4 participants