Skip to content

add cgroup2 CI (Vagrant on GitHub Actions)#41918

Closed
AkihiroSuda wants to merge 5 commits intomoby:masterfrom
AkihiroSuda:cgroup2ci
Closed

add cgroup2 CI (Vagrant on GitHub Actions)#41918
AkihiroSuda wants to merge 5 commits intomoby:masterfrom
AkihiroSuda:cgroup2ci

Conversation

@AkihiroSuda
Copy link
Member

@AkihiroSuda AkihiroSuda commented Jan 22, 2021

Currently Vagrant on GitHub Actions is used for creating Fedora VMs with cgroup2.
Eventually this should be replaced with real Jenkins nodes.

Close #41218

Contains #41917 ("TestCgroupNamespacesRunOlderClient: support cgroup v2")

Logs: https://github.com/AkihiroSuda/docker/actions?query=workflow%3A%22Cgroup+v2%22


The following tests are currently skipped. Will be analyzed and fixed in follow-up PRs.

    === FAIL: amd64.integration.service TestCreateServiceSecretFileMode (15.72s)
        create_test.go:291: assertion failed: 3 (int) != 1 (int)

    === FAIL: amd64.integration.service TestCreateServiceConfigFileMode (9.62s)
        create_test.go:355: assertion failed: 2 (int) != 1 (int)

    === FAIL: amd64.integration.container TestInspectOomKilledTrue (0.53s)
        kill_test.go:171: assertion failed: true (true bool) != false (inspect.State.OOMKilled bool)

    === FAIL: amd64.integration.container TestHealthKillContainer (13.00s)
        health_test.go:62: timeout hit after 10s: waiting for container to become healthy

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
```
=== FAIL: amd64.integration.container TestHealthKillContainer (13.00s)
    health_test.go:62: timeout hit after 10s: waiting for container to become healthy
```

TODO: analysis

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
```
=== FAIL: amd64.integration.container TestInspectOomKilledTrue (0.53s)
    kill_test.go:171: assertion failed: true (true bool) != false (inspect.State.OOMKilled bool)
```

TODO: analysis

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
@AkihiroSuda AkihiroSuda marked this pull request as draft January 22, 2021 10:02
…ion with "fc")

The tests fail on Fedora 33 Vagrant:

```
=== FAIL: amd64.integration.service TestCreateServiceConfigFileMode (9.62s)
      create_test.go:355: assertion failed: 2 (int) != 1 (int)

=== FAIL: amd64.integration.service TestCreateServiceSecretFileMode (15.72s)
    create_test.go:291: assertion failed: 3 (int) != 1 (int)
```

Probably unrelated to cgroup version, as the test works fine on Ubuntu
20.10 + cgroup v2.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Currently Vagrant on GitHub Actions is used for creating Fedora VMs with cgroup2.
Eventually this should be replaced with real Jenkins nodes.

Logs: https://github.com/AkihiroSuda/docker/actions?query=workflow%3A%22Cgroup+v2%22

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
@AkihiroSuda AkihiroSuda marked this pull request as ready for review January 22, 2021 12:41
@AkihiroSuda
Copy link
Member Author

@thaJeztah
Copy link
Member

Eventually this should be replaced with real Jenkins nodes.

ISTR that @StefanScherer added nodes to jenkins with cgroups v2 (recent Ubuntu versions); don't recall what label had to be set to pick that type of node though 😓

@StefanScherer
Copy link
Contributor

@AkihiroSuda oh very nice, I didn't know that Vagrant is preinstalled on the macOS runners. 🥇

@thaJeztah I remember we tested cgroups2 on CentOS 8 with Jenkins label centos-8, we have docker 19.03.13 preinstalled.

I had a quick look and it seems there are Fedora 33 Cloud Base images in EC2 https://alt.fedoraproject.org/cloud/
So if these would be better we could take a look to prepare them for CI with a packer template.

@AkihiroSuda
Copy link
Member Author

I remember we tested cgroups2 on CentOS 8 with Jenkins label centos-8, we have docker 19.03.13 preinstalled.

cgroup2 requires Docker 20.10 with kernel >= 5.2.

CentOS 8 uses kernel 4.18 and do not enable cgroup2 by default. Kernel 4.18 is probably actually enough for running Docker with cgroup2, but it lacks support for freezer.

@StefanScherer
Copy link
Contributor

StefanScherer commented Jan 22, 2021

Oh wait, we have Ubuntu 20.04 machines with Jenkins label ubuntu-2004, currently with Docker 19.03.13 installed, but that's easy to update. Is cgroup2 enabled by default?

@AkihiroSuda
Copy link
Member Author

Ubuntu still uses cgroup v1 by default. Ubuntu 21.10 is likely to switch to cgroup2 (#41916).

@AkihiroSuda
Copy link
Member Author

AkihiroSuda commented Jan 22, 2021

Note: cgroup v1 and v2 cannot be enabled simultaneously. (There is a mode called "hybrid", but meaningless for containers.)

Enabling cgroup v2 (which incurs disabling v1) needs adding systemd.unified_cgroup_hierarchy=1 to GRUB_CMDLINE_LINUX line in /etc/default/grub.

@StefanScherer
Copy link
Contributor

Thanks @AkihiroSuda for the details.
I found we already have prepared something in our packer templates, but haven't activated cgroup2 yet. I'll create another AMI with it enabled, the plan is to have Jenkins label ubuntu-2004 && docker && cgroup2 && amd64 to spin up these machines.

@AkihiroSuda
Copy link
Member Author

Thanks, opened another PR to use Jenkins #41925

@AkihiroSuda AkihiroSuda added the area/cgroup2 cgroup v2 label Jan 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/cgroup2 cgroup v2

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Setup cgroups v2 CI

3 participants