Skip to content

Define PodDisruptionBudget API type.#24697

Merged
k8s-github-robot merged 2 commits intokubernetes:masterfrom
mml:disruption-budget
May 7, 2016
Merged

Define PodDisruptionBudget API type.#24697
k8s-github-robot merged 2 commits intokubernetes:masterfrom
mml:disruption-budget

Conversation

@mml
Copy link
Copy Markdown
Contributor

@mml mml commented Apr 22, 2016

No description provided.

@mml mml added priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. release-note-none Denotes a PR that doesn't merit a release note. labels Apr 22, 2016
@googlebot
Copy link
Copy Markdown

We found a Contributor License Agreement for you (the sender of this pull request) and all commit authors, but as best as we can tell these commits were authored by someone else. If that's the case, please add them to this pull request and have them confirm that they're okay with these commits being contributed to Google. If we're mistaken and you did author these commits, just reply here to confirm.

@mml
Copy link
Copy Markdown
Contributor Author

mml commented Apr 22, 2016

@mqliang can you please confirm as the bot requests?

@mml
Copy link
Copy Markdown
Contributor Author

mml commented Apr 23, 2016

Also, I guess the way I squashed the history here has obscured who wrote what. Should I separate my part of the change somehow to clarify that? Not sure who to ask, so I pick @davidopp

@k8s-github-robot k8s-github-robot added kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Apr 23, 2016
@mml mml force-pushed the disruption-budget branch from eec13e1 to ce82aca Compare April 23, 2016 00:03
@mml
Copy link
Copy Markdown
Contributor Author

mml commented Apr 23, 2016

@davidopp Never mind that last question. git user error.

@mqliang
Copy link
Copy Markdown
Contributor

mqliang commented Apr 23, 2016

@mml Actually, you can squash the commits for easier review, I am fine with that.

@davidopp davidopp assigned davidopp and unassigned thockin Apr 23, 2016
pkg/api/types.go Outdated
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this representation for "exactly one of these" but Deployment uses intstr.IntOrString so for consistency I would suggest you do the same (i.e. replace these two fields with a single IntOrString field).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My reason here was that I didn't want to constantly be checking if it's an int or string, decoding the string into a number and multiplying. I preferred a back-end representation that already had the percentage decoded, at which point it made more sense to have the public representation match the back-end.

If I switch to IntOrString, I guess all consumers of this info will just do this work over and over again. It's not horrible inefficiency, but it feels wrong to be wasteful merely for consistency. Perhaps if we want consistency we should revise Deployment to use this "exactly one" approach in v2?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It occurs to me that it's going to be tricky to infer the correct number of expected pods in order to do the multiplication when this is specified as a percentage. If the RC (or other pod controller) is lagging, we could be off by one or a lot more than one. To do a better job than that, we'd need knowledge of the controller's specs and its semantics.

Given the distributed-responsibility design of k8s, I'm not sure of a cleaner path forward than just trusting that the count of all the pods we see as the correct number of "expected" pods.

This approach would not work with DaemonSets, though, since the DS controller doesn't create pods when it doesn't believe they'll fit.

Copy link
Copy Markdown
Contributor

@davidopp davidopp Apr 27, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding your last comment: That's a really great point. As you alluded to, to compute the denominator for the percentage, you'd need to iterate through all controllers in the system, and add up the number of replicas in the "intended state" of all controllers' specs whose pods use this DisruptionBudget. But in fact that would still be wrong, because in theory you can have pods with no controller, that point back to a DisruptionBudget. The denominator is fundamentally not well-defined.

I don't think it works to use the number of pods you see as the "expected" number of pods, because then by definition 100% will always be available (or at least existing, if not available in the sense of "up"). (Though maybe I misunderstood that part of your comment.)

I guess we should just do MinAvailable (an int), and expect that the config system would have templating that would allow you to say "MinAvailable = 0.8 * X" (e.g. you use X in your PodTemplate to set number of replicas) and the value would get filled in when the config is evaluated...

@davidopp
Copy link
Copy Markdown
Contributor

I think you should rename the type to PodDisruptionBudget. You could imagine having something similar for nodes (i.e. NodeDisruptionBudget) to control how many can be taken down for maintenance at a time, or the rate of taking them down.

@mml mml force-pushed the disruption-budget branch from ce82aca to a736dc1 Compare April 26, 2016 22:19
@googlebot
Copy link
Copy Markdown

CLAs look good, thanks!

@mml
Copy link
Copy Markdown
Contributor Author

mml commented Apr 26, 2016

@davidopp PTAL

@mml mml changed the title API definitation of DisruptionBudget Define PodDisruptionBudget API type. Apr 26, 2016
@mml mml force-pushed the disruption-budget branch 3 times, most recently from 0719062 to 4c9fc3d Compare April 27, 2016 01:49
@k8s-github-robot k8s-github-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 27, 2016
@mml mml force-pushed the disruption-budget branch from cc848ff to 84196a8 Compare April 27, 2016 19:26
@k8s-github-robot k8s-github-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 27, 2016
@mml
Copy link
Copy Markdown
Contributor Author

mml commented May 6, 2016

@davidopp We should be good to go.

Keeping the actual edits separate from the codegen in the commit history requires a lot of git acrobatics, especially as the PR progresses.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can omit "Selector is a", but it's fine to fix this in a followup PR.

@davidopp
Copy link
Copy Markdown
Contributor

davidopp commented May 6, 2016

LGTM

@davidopp davidopp added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 6, 2016
@davidopp
Copy link
Copy Markdown
Contributor

davidopp commented May 6, 2016

cc/ @bgrant0607 @lavalamp

@mml mml force-pushed the disruption-budget branch from 5862baf to fb879eb Compare May 6, 2016 20:22
@k8s-github-robot k8s-github-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 6, 2016
@mml mml added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 6, 2016
@mml
Copy link
Copy Markdown
Contributor Author

mml commented May 6, 2016

Sigh. This passes on my workstation.

diffing pkg/apis against freshly generated protobuf
diff -Naupr -I 'Auto generated by' /go/src/k8s.io/kubernetes/pkg/apis/apps/v1alpha1/generated.proto /go/src/k8s.io/kubernetes/_tmp/pkg/apis/apps/v1alpha1/generated.proto
--- /go/src/k8s.io/kubernetes/pkg/apis/apps/v1alpha1/generated.proto    2016-05-06 14:23:14.066699432 -0700
+++ /go/src/k8s.io/kubernetes/_tmp/pkg/apis/apps/v1alpha1/generated.proto   2016-05-06 14:06:56.482815594 -0700
@@ -24,6 +24,7 @@ package k8s.io.kubernetes.pkg.apis.apps.
 import "k8s.io/kubernetes/pkg/api/resource/generated.proto";
 import "k8s.io/kubernetes/pkg/api/unversioned/generated.proto";
 import "k8s.io/kubernetes/pkg/api/v1/generated.proto";
+import "k8s.io/kubernetes/pkg/apis/policy/v1alpha1/generated.proto";
 import "k8s.io/kubernetes/pkg/util/intstr/generated.proto";

 // Package-wide variables from generator "generated".
pkg/apis is out of date. Please run hack/update-generated-protobuf.sh
!!! Error in ./hack/../hack/verify-generated-protobuf.sh:26
  '"${KUBE_ROOT}/hack/after-build/verify-generated-protobuf.sh" "$@"' exited with status 1
Call stack:
  1: ./hack/../hack/verify-generated-protobuf.sh:26 main(...)
Exiting with status 1
FAILED   ./hack/../hack/verify-generated-protobuf.sh    28s
Verifying ./hack/../hack/verify-generated-swagger-docs.sh

@mml
Copy link
Copy Markdown
Contributor Author

mml commented May 6, 2016

@kubernetes/sig-api-machinery and @kubernetes/test-infra-admins any idea what's going on here? I verified that jenkins and I are testing at the same SHA-1.

import "k8s.io/kubernetes/pkg/api/resource/generated.proto";
import "k8s.io/kubernetes/pkg/api/unversioned/generated.proto";
import "k8s.io/kubernetes/pkg/api/v1/generated.proto";
import "k8s.io/kubernetes/pkg/apis/policy/v1alpha1/generated.proto";
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you've somehow picked up this spurious line. If you remove it and rerun update-codegen.sh, does the proto generator put it back? @smarterclayton might have more insight.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it does.

% git log -n1 -u
commit 5b5ba10a9673baa88e7a93b3527d05e444bca58b
Author: Matt Liggett <mml@google.com>
Date:   Fri May 6 15:24:00 2016 -0700

    wtf

diff --git a/pkg/apis/apps/v1alpha1/generated.proto b/pkg/apis/apps/v1alpha1/generated.proto
index de4d560..6cb15bf 100644
--- a/pkg/apis/apps/v1alpha1/generated.proto
+++ b/pkg/apis/apps/v1alpha1/generated.proto
@@ -24,7 +24,6 @@ package k8s.io.kubernetes.pkg.apis.apps.v1alpha1;
 import "k8s.io/kubernetes/pkg/api/resource/generated.proto";
 import "k8s.io/kubernetes/pkg/api/unversioned/generated.proto";
 import "k8s.io/kubernetes/pkg/api/v1/generated.proto";
-import "k8s.io/kubernetes/pkg/apis/policy/v1alpha1/generated.proto";
 import "k8s.io/kubernetes/pkg/util/intstr/generated.proto";

 // Package-wide variables from generator "generated".

[..after re-running u-g-p..]

% git di
diff --git a/pkg/apis/apps/v1alpha1/generated.proto b/pkg/apis/apps/v1alpha1/generated.proto
index 6cb15bf..de4d560 100644
--- a/pkg/apis/apps/v1alpha1/generated.proto
+++ b/pkg/apis/apps/v1alpha1/generated.proto
@@ -24,6 +24,7 @@ package k8s.io.kubernetes.pkg.apis.apps.v1alpha1;
 import "k8s.io/kubernetes/pkg/api/resource/generated.proto";
 import "k8s.io/kubernetes/pkg/api/unversioned/generated.proto";
 import "k8s.io/kubernetes/pkg/api/v1/generated.proto";
+import "k8s.io/kubernetes/pkg/apis/policy/v1alpha1/generated.proto";
 import "k8s.io/kubernetes/pkg/util/intstr/generated.proto";

 // Package-wide variables from generator "generated".

@smarterclayton
Copy link
Copy Markdown
Contributor

smarterclayton commented May 6, 2016 via email

@mml
Copy link
Copy Markdown
Contributor Author

mml commented May 6, 2016

% hack ugp      
+hack:51> KUBE_RELEASE_RUN_TESTS=n KUBE_FASTBUILD=true /usr/local/google/home/mml/gocode/src/k8s.io/kubernetes/hack/update-generated-protobuf.sh
+++ [0506 16:00:51] Verifying Prerequisites....
+++ [0506 16:00:51] Building Docker image kube-build:build-dfcf80fc21.
+++ [0506 16:00:52] Running build command....
Go version: go version go1.6.2 linux/amd64
+++ [0506 16:00:53] Building go targets for linux/amd64:
    cmd/libs/go2idl/go-to-protobuf
    cmd/libs/go2idl/go-to-protobuf/protoc-gen-gogo
+++ [0506 16:00:56] Placing binaries

[...]

@smarterclayton
Copy link
Copy Markdown
Contributor

Ok, so my second theory is that there is an unstable sort order in
reflection/go2idl that results in certain packages being flagged before
others (and proto generation does flood fill so an unstable order could
result in different discovered dependencies and hence different imports).
Go package processing order is supposed to be stable in 1.6, but we've
still seen instances of this in various forms because your changes change
the order that dependencies are traversed in go itself.

Crazy idea - try running "git clean -fd" on your directory and trying
again.

I'll look into the ordering in the protobufs generation again.

At worst, check in the version expected and wojciech and I will look into
this.

On May 6, 2016, at 7:02 PM, Matt Liggett notifications@github.com wrote:

% hack ugp
+hack:51> KUBE_RELEASE_RUN_TESTS=n KUBE_FASTBUILD=true
/usr/local/google/home/mml/gocode/src/k8s.io/kubernetes/hack/update-generated-protobuf.sh
+++ [0506 16:00:51] Verifying Prerequisites....
+++ [0506 16:00:51] Building Docker image kube-build:build-dfcf80fc21.
+++ [0506 16:00:52] Running build command....
Go version: go version go1.6.2 linux/amd64
+++ [0506 16:00:53] Building go targets for linux/amd64:
cmd/libs/go2idl/go-to-protobuf
cmd/libs/go2idl/go-to-protobuf/protoc-gen-gogo
+++ [0506 16:00:56] Placing binaries

[...]


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#24697 (comment)

@mml
Copy link
Copy Markdown
Contributor Author

mml commented May 7, 2016

OK. Same result after git clean -fd. I'll fake it in the proto file.

@mml mml force-pushed the disruption-budget branch from fb879eb to e1fa2a0 Compare May 7, 2016 00:25
@k8s-github-robot k8s-github-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 7, 2016
@mml mml added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 7, 2016
@k8s-bot
Copy link
Copy Markdown

k8s-bot commented May 7, 2016

GCE e2e build/test passed for commit e1fa2a0.

@k8s-github-robot
Copy link
Copy Markdown

@k8s-bot test this [submit-queue is verifying that this PR is safe to merge]

@k8s-bot
Copy link
Copy Markdown

k8s-bot commented May 7, 2016

GCE e2e build/test passed for commit e1fa2a0.

@k8s-github-robot
Copy link
Copy Markdown

Automatic merge from submit-queue

@k8s-github-robot k8s-github-robot merged commit 9643548 into kubernetes:master May 7, 2016
@mml mml deleted the disruption-budget branch May 7, 2016 03:21
k8s-github-robot pushed a commit that referenced this pull request May 13, 2016
Automatic merge from submit-queue

A few followups from #24697
caseydavenport added a commit to caseydavenport/kubernetes that referenced this pull request May 15, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. release-note-none Denotes a PR that doesn't merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.