[sig-node] Add presubmit for kubelet-serial by ehashman · Pull Request #22284 · kubernetes/test-infra

ehashman · 2021-05-20T20:21:11Z

Mostly copied from

test-infra/config/jobs/kubernetes/sig-node/node-kubelet.yaml

Lines 205 to 233 in 29f8f50

    
           - name: ci-kubernetes-node-kubelet-serial 
        
             interval: 4h30m 
        
             labels: 
        
               preset-service-account: "true" 
        
               preset-k8s-ssh: "true" 
        
             spec: 
        
               containers: 
        
               - image: gcr.io/k8s-testimages/kubekins-e2e:v20210512-b8d1b30-master 
        
                 args: 
        
                 - --repo=k8s.io/kubernetes=master 
        
                 - --timeout=320 
        
                 - --root=/go/src 
        
                 - --scenario=kubernetes_e2e 
        
                 - -- 
        
                 - --deployment=node 
        
                 - --gcp-project-type=node-e2e-project 
        
                 - --gcp-zone=us-west1-b 
        
                 - --node-args=--image-config-file=/workspace/test-infra/jobs/e2e_node/image-config-serial.yaml 
        
                 - --node-test-args=--feature-gates=DynamicKubeletConfig=true,LocalStorageCapacityIsolation=true --kubelet-flags="--cgroups-per-qos=true --cgroup-root=/" 
        
                 - --node-tests=true 
        
                 - --provider=gce 
        
                 - --test_args=--nodes=1 --focus="\[Serial\]" --skip="\[Flaky\]|\[Benchmark\]|\[NodeSpecialFeature:.+\]|\[NodeAlphaFeature:.+\]" 
        
                 - --timeout=300m 
        
                 env: 
        
                 - name: GOPATH 
        
                   value: /go 
        
             annotations: 
        
               testgrid-dashboards: sig-node-kubelet 
        
               testgrid-tab-name: node-kubelet-serial

/cc @dims @BenTheElder
/sig node

dims · 2021-05-20T20:32:27Z

/hold

let's please explore a containerd version of this job!

SergeyKanzhelev · 2021-05-20T21:57:34Z

/assign

ehashman · 2021-05-20T22:26:42Z

/hold

let's please explore a containerd version of this job!

@dims while I agree we really need a non-docker version of this job, right now this is just a presubmit duplicate of the job that's currently running as a periodic. I don't think that we should block being able to trigger the existing job on a PR.

dims · 2021-05-20T23:44:50Z

@ehashman that's fair, i am just pushing on us to come up with a containerd based job.

ehashman · 2021-05-21T00:07:44Z

We definitely need to do it 🙏 But I would like to see the hold cancelled here so we can at least test this on PRs rather than merge first test as periodic 😅

harche · 2021-05-21T00:08:31Z

/cc @odinuge

harche · 2021-05-21T06:18:00Z

I will add similar job with crio + systemd configuration.

harche · 2021-05-21T06:41:49Z

@ehashman @dims #22290

odinuge

Except for the timeout stuff;

/lgtm

ref. @dims' comment, I think it is best to keep this "similar" to the old node-kubelet-serial for now, and then add/change it to use containerd later. @harche is doing a cri-o version with systemd, which is way more important!

odinuge · 2021-05-21T07:04:19Z

+        - --node-tests=true
+        - --provider=gce
+        - --test_args=--nodes=1 --focus="\[Serial\]" --skip="\[Flaky\]|\[Benchmark\]|\[NodeSpecialFeature:.+\]|\[NodeAlphaFeature:.+\]"
+        - --timeout=300m


I am pretty sure this timeout is the reason the period job is failing/we get no output. Bump this to 420 or 360 in order to see if it makes any difference? We can always go back and bump the value later in case it is needed.

Bumped it to 420m.

odinuge · 2021-05-27T09:32:34Z

/retest
/lgtm

odinuge · 2021-05-27T09:51:02Z

Job execution failed: Pod got deleted unexpectedly
/retest

BenTheElder · 2021-05-27T10:02:39Z

jobs_test.go:894: Invalid Scenario Args : jobs pull-kubernetes-node-kubelet-serial - kubetest timeout(7h0m0s), >bootstrap timeout(5h20m0s): bootstrap timeout need to be 20min more than kubetest timeout!

Also ideally new jobs should not be using https://github.com/kubernetes/test-infra/blob/master/jenkins/bootstrap.py, instead using decorate: true and the podutils, but that's another matter. Something to put on the backlog.

SergeyKanzhelev · 2021-06-02T16:49:12Z

@ehashman can you please fix the broken build. I agree that we can unhold and address the containerd comment later

ehashman · 2021-06-02T17:29:09Z

Should be fixed now 🤞

ehashman · 2021-06-02T22:03:11Z

/retest

ehashman · 2021-06-03T00:46:05Z

Should be GTG now! @SergeyKanzhelev @BenTheElder

SergeyKanzhelev · 2021-06-03T00:48:10Z

/lgtm

SergeyKanzhelev · 2021-06-03T00:48:59Z

/hold cancel

@dims are you ok with the plan to enable it first and later we can either add another job with containerd or change this one?

dims · 2021-06-03T01:53:22Z

@SergeyKanzhelev sounds good!

BenTheElder

LGTM

k8s-ci-robot · 2021-06-03T18:43:17Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: BenTheElder, ehashman, odinuge

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~config/jobs/kubernetes/sig-node/OWNERS~~ [BenTheElder]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot · 2021-06-03T18:58:08Z

@ehashman: Updated the job-config configmap in namespace default at cluster test-infra-trusted using the following files:

key sig-node-presubmit.yaml using file config/jobs/kubernetes/sig-node/sig-node-presubmit.yaml

Details

In response to this:

Mostly copied from

test-infra/config/jobs/kubernetes/sig-node/node-kubelet.yaml

Lines 205 to 233 in 29f8f50

- name: ci-kubernetes-node-kubelet-serial

interval: 4h30m

labels:

preset-service-account: "true"

preset-k8s-ssh: "true"

spec:

containers:

- image: gcr.io/k8s-testimages/kubekins-e2e:v20210512-b8d1b30-master

args:

- --repo=k8s.io/kubernetes=master

- --timeout=320

- --root=/go/src

- --scenario=kubernetes_e2e

- --

- --deployment=node

- --gcp-project-type=node-e2e-project

- --gcp-zone=us-west1-b

- --node-args=--image-config-file=/workspace/test-infra/jobs/e2e_node/image-config-serial.yaml

- --node-test-args=--feature-gates=DynamicKubeletConfig=true,LocalStorageCapacityIsolation=true --kubelet-flags="--cgroups-per-qos=true --cgroup-root=/"

- --node-tests=true

- --provider=gce

- --test_args=--nodes=1 --focus="\[Serial\]" --skip="\[Flaky\]|\[Benchmark\]|\[NodeSpecialFeature:.+\]|\[NodeAlphaFeature:.+\]"

- --timeout=300m

env:

- name: GOPATH

value: /go

annotations:

testgrid-dashboards: sig-node-kubelet

testgrid-tab-name: node-kubelet-serial

/cc @dims @BenTheElder
/sig node

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot requested review from BenTheElder and dims May 20, 2021 20:21

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 20, 2021

k8s-ci-robot assigned SergeyKanzhelev May 20, 2021

ehashman force-pushed the node-serial-presubmit branch from f6f7563 to c282db3 Compare May 20, 2021 22:29

ehashman mentioned this pull request May 20, 2021

vendor: bump runc to rc95 kubernetes/kubernetes#102147

Merged

k8s-ci-robot requested a review from odinuge May 21, 2021 00:08

odinuge approved these changes May 21, 2021

View reviewed changes

k8s-ci-robot assigned odinuge May 21, 2021

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 21, 2021

ehashman force-pushed the node-serial-presubmit branch from c282db3 to cccb359 Compare May 26, 2021 17:19

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 26, 2021

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 27, 2021

[sig-node] Add presubmit for kubelet-serial

29b7275

ehashman force-pushed the node-serial-presubmit branch from cccb359 to 29b7275 Compare June 2, 2021 17:28

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 2, 2021

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 3, 2021

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 3, 2021

BenTheElder approved these changes Jun 3, 2021

View reviewed changes

k8s-ci-robot assigned BenTheElder Jun 3, 2021

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 3, 2021

k8s-ci-robot merged commit 679b0c7 into kubernetes:master Jun 3, 2021

k8s-ci-robot added this to the v1.22 milestone Jun 3, 2021

ehashman mentioned this pull request Jun 3, 2021

The whole node-kubelet-serial job is failing because of global timeout kubernetes/kubernetes#102148

Closed

ehashman deleted the node-serial-presubmit branch March 17, 2022 22:24

	- name: ci-kubernetes-node-kubelet-serial
	interval: 4h30m
	labels:
	preset-service-account: "true"
	preset-k8s-ssh: "true"
	spec:
	containers:
	- image: gcr.io/k8s-testimages/kubekins-e2e:v20210512-b8d1b30-master
	args:
	- --repo=k8s.io/kubernetes=master
	- --timeout=320
	- --root=/go/src
	- --scenario=kubernetes_e2e
	- --
	- --deployment=node
	- --gcp-project-type=node-e2e-project
	- --gcp-zone=us-west1-b
	- --node-args=--image-config-file=/workspace/test-infra/jobs/e2e_node/image-config-serial.yaml
	- --node-test-args=--feature-gates=DynamicKubeletConfig=true,LocalStorageCapacityIsolation=true --kubelet-flags="--cgroups-per-qos=true --cgroup-root=/"
	- --node-tests=true
	- --provider=gce
	- --test_args=--nodes=1 --focus="\[Serial\]" --skip="\[Flaky\]\|\[Benchmark\]\|\[NodeSpecialFeature:.+\]\|\[NodeAlphaFeature:.+\]"
	- --timeout=300m
	env:
	- name: GOPATH
	value: /go
	annotations:
	testgrid-dashboards: sig-node-kubelet
	testgrid-tab-name: node-kubelet-serial

Conversation

ehashman commented May 20, 2021

Uh oh!

dims commented May 20, 2021

Uh oh!

SergeyKanzhelev commented May 20, 2021

Uh oh!

ehashman commented May 20, 2021

Uh oh!

dims commented May 20, 2021

Uh oh!

ehashman commented May 21, 2021

Uh oh!

harche commented May 21, 2021

Uh oh!

harche commented May 21, 2021

Uh oh!

harche commented May 21, 2021

Uh oh!

odinuge left a comment

Choose a reason for hiding this comment

Uh oh!

odinuge May 21, 2021

Choose a reason for hiding this comment

Uh oh!

ehashman May 26, 2021

Choose a reason for hiding this comment

Uh oh!

odinuge commented May 27, 2021

Uh oh!

odinuge commented May 27, 2021

Uh oh!

BenTheElder commented May 27, 2021

Uh oh!

SergeyKanzhelev commented Jun 2, 2021

Uh oh!

ehashman commented Jun 2, 2021

Uh oh!

ehashman commented Jun 2, 2021

Uh oh!

ehashman commented Jun 3, 2021

Uh oh!

SergeyKanzhelev commented Jun 3, 2021

Uh oh!

SergeyKanzhelev commented Jun 3, 2021

Uh oh!

dims commented Jun 3, 2021

Uh oh!

BenTheElder left a comment

Choose a reason for hiding this comment

Uh oh!

k8s-ci-robot commented Jun 3, 2021

Uh oh!

k8s-ci-robot commented Jun 3, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants