vmcluster/statefulset: run rolling upgrades in batches of multiple pods#1458
Merged
f41gh7 merged 2 commits intoVictoriaMetrics:masterfrom Jul 8, 2025
Merged
Conversation
f41gh7
reviewed
Jul 7, 2025
f41gh7
reviewed
Jul 7, 2025
Collaborator
f41gh7
left a comment
There was a problem hiding this comment.
Thanks for the pull request, it looks good to me!
Could you please mention new API field at docs/CHANGELOG.md as a feature?
Also, it looks like this PR fixes possible issue with breaching of PodDisruption policy by using Eviction instead of Delete
Collaborator
|
Thanks for contribution! |
f41gh7
added a commit
that referenced
this pull request
Jul 9, 2025
Previously operator perform pod deletion without check for PodDisruptionBudget interption. It was fixed at #1458 Signed-off-by: f41gh7 <nik@victoriametrics.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes: #1457
This PR introduces a new
VMClusterCRD property calledmaxUnavailablein:spec.vmstorage.rollingUpdateStrategyBehavior.maxUnavailablespec.vmselect.rollingUpdateStrategyBehavior.maxUnavailableIt's very similar to this (alpha) feature of Kubernetes:
https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#maximum-unavailable-pods
This PR makes it work with the
OnDeleteupgrade policy used in the operator by default.Additionally, this PR makes it available for much older Kubernetes versions or, for example, it makes it finally available for those running GKE clusters that don't support alpha features.
Some examples for the
maxUnavailablefield:maxUnavailable: 1- default, which mostly (see details on PDBs below) follows the current behavior of the operator.maxUnavailable: 2- for cases when-replicationFactor=3.maxUnavailable: 100%- for cases when the "minimum downtime strategy" is preferable.Since more than one pod can now become unavailable during an upgrade (i.e. voluntary disruptions) it's important to respect a PDB configuration to some degree. So this PR introduces a partial support for PDBs targeted at StatefulSets of
VMClusteraccording to this:https://kubernetes.io/docs/tasks/run-application/configure-pdb/#arbitrary-controllers-and-selectors
This is achieved with the Eviction API instead of directly deleting Pods, as described here:
https://kubernetes.io/docs/concepts/scheduling-eviction/api-eviction/
And since it appears that "write" RBAC permission for Pods is no longer needed, I've moved it to the "read-only" section.
Please let me know if this makes sense!