[ci] upgrade CI test containers from Ubuntu 20.04 to 22.04 by andrew-anyscale · Pull Request #61533 · ray-project/ray

andrew-anyscale · 2026-03-05T21:11:31Z

Upgrades CI test container base images from Ubuntu 20.04 to 22.04 (jammy), including the mandatory clang-12 to clang-14 toolchain upgrade available in 22.04.

focal → jammy for all base images and wanda configs (base.test, base.gpu, base.cu128, forge wanda)
clang-12 → clang-14 in base.test, base.gpu, base.build Dockerfiles and install-llvm-binaries.sh
libncurses5 → libncurses6
libgl1-mesa-glx → libgl1 (dropped in 22.04)
netcat → netcat-openbsd (22.04 no longer provides the bare netcat metapackage)
mongodb-org apt source: focal → jammy
Java 8 update-alternatives: 22.04 defaults to Java 11, pin to Java 8 where needed
test_dashboard_profiler.py: skip py-spy --native on Linux (known issue, see [Dashboard] Getting stacktrace from the dashboard triggers "failed to get os threadid" when --native option is used #30566)
onnx 1.15.0 → 1.16.0 (compatibility with newer environment)
Simplified Podman doc (20.04 workaround removed)

Topic: ubuntu-22-04-test
Relative: data-mongodb-direct-launch

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
Signed-off-by: andrew andrew@anyscale.com

andrew-anyscale · 2026-03-05T21:11:32Z

Reviews in this chain:
└#61533 [ci] upgrade CI test containers from Ubuntu 20.04 to 22.04
　└#61662 [ci] Upgrade forge Dockerfiles from clang-12 to clang-14

andrew-anyscale · 2026-03-05T21:11:33Z

#	head	base	diff	date	summary
0	1dac0b9e	5270e8ce	diff	Mar 5 13:11 PM	19 files changed, 77 insertions(+), 45 deletions(-)
1	2f035d8c	5270e8ce	diff	Mar 5 13:12 PM	0 files changed
2	9de24404	5270e8ce	diff	Mar 5 13:38 PM	3 files changed, 13 insertions(+), 5 deletions(-)
3	4be924b4	5270e8ce	diff	Mar 5 14:01 PM	1 file changed, 3 insertions(+), 3 deletions(-)
4	fb42727f	5270e8ce	diff	Mar 6 9:17 AM	1 file changed, 3 insertions(+)
5	65100c8e	1f3a82ee	diff	Mar 6 12:37 PM	1 file changed, 3 insertions(+)
6	9c10e320	1f3a82ee	diff	Mar 6 23:46 PM	1 file changed, 4 insertions(+), 3 deletions(-)
7	3376a60b	1f3a82ee	diff	Mar 7 7:21 AM	3 files changed, 2 insertions(+), 6 deletions(-)
8	adf994bb	1f3a82ee	diff	Mar 7 14:23 PM	1 file changed, 4 insertions(+)
9	d47c6a74	1f3a82ee	diff	Mar 7 19:45 PM	1 file changed, 3 insertions(+), 3 deletions(-)
10	71222aea	1f3a82ee	diff	Mar 9 9:08 AM	3 files changed, 25 insertions(+), 4 deletions(-)
11	da866255	a8a13fe3	diff	Mar 10 7:35 AM	1 file changed, 8 insertions(+)
12	3b35b763	aa6c579c	rebase	Mar 10 9:57 AM	0 files changed
13	4b63e4e0	d5391f90	diff	Mar 10 12:47 PM	2 files changed, 32 insertions(+)
14	cd58aef5	1a16658a	diff	Mar 11 7:05 AM	2 files changed, 2 insertions(+), 1 deletion(-)
15	5c139b2b	1a16658a	diff	Mar 11 7:17 AM	0 files changed
16	eb583c96	1a16658a	diff	Mar 11 7:18 AM	0 files changed
17	74c9dfa0	348d195d	diff	Mar 11 13:05 PM	0 files changed
18	8f1b2106	8147de2f	rebase	Mar 11 13:54 PM	0 files changed
19	f66b0677	19bd6b10	diff	Mar 11 15:27 PM	0 files changed
20	4d398b00	b250e059	rebase	Mar 11 16:24 PM	0 files changed
21	ada6fd6c	4323abf4	rebase	Mar 11 16:38 PM	0 files changed
22	f3d28505	73b0d455	rebase	Mar 11 17:52 PM	0 files changed
23	c70134e2	7cf438c7	diff	Mar 12 20:36 PM	2 files changed, 57 insertions(+), 19 deletions(-)
24	58a676aa	7913eb8b	diff	Mar 17 10:22 AM	0 files changed

gemini-code-assist

Code Review

This pull request upgrades the CI test containers from Ubuntu 20.04 to 22.04, which is a significant and necessary infrastructure update. The changes across Dockerfiles, configuration files, and test code are mostly correct and well-justified. I have a few suggestions for improvement. First, for better compatibility, I recommend using LLVM binaries specifically built for Ubuntu 22.04. Second, in the documentation, it would be beneficial to retain installation instructions for Podman on Ubuntu 20.04, as it is a widely used LTS release. Lastly, I've suggested a small refactoring in a test file to improve resource management by ensuring a temporary directory is always cleaned up.

ci/env/install-llvm-binaries.sh

doc/source/serve/advanced-guides/multi-app-container.md

aslonnie

could you resolve conflicts?

Upgrades CI test container base images from Ubuntu 20.04 to 22.04 (jammy), including the mandatory clang-12 to clang-14 toolchain upgrade available in 22.04. - focal → jammy for all base images and wanda configs (base.test, base.gpu, base.cu128, forge wanda) - clang-12 → clang-14 in base.test, base.gpu, base.build Dockerfiles and install-llvm-binaries.sh - libncurses5 → libncurses6 - libgl1-mesa-glx → libgl1 (dropped in 22.04) - netcat → netcat-openbsd (22.04 no longer provides the bare netcat metapackage) - mongodb-org apt source: focal → jammy - Java 8 update-alternatives: 22.04 defaults to Java 11, pin to Java 8 where needed - test_dashboard_profiler.py: skip py-spy --native on Linux (known issue, see #30566) - onnx 1.15.0 → 1.16.0 (compatibility with newer environment) - Simplified Podman doc (20.04 workaround removed) Topic: ubuntu-22-04-test Relative: data-mongodb-direct-launch Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: andrew <andrew@anyscale.com>

14.0.6 does not ship an ubuntu-22.04 tarball (404). No LLVM 14.x release does. Use 14.0.0 with the ubuntu-18.04 tarball, which is glibc backwards-compatible and works on Ubuntu 22.04. Topic: ubuntu-22-04-test Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: andrew <andrew@anyscale.com>

The base.test container was missing CC/CXX unlike base.build and base.gpu. Set them explicitly so tools running inside the test container (e.g. test_exported_symbols_unix which builds _raylet via Bazel) use the correct clang-14 compiler. Topic: ubuntu-22-04-test Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: andrew <andrew@anyscale.com>

After upgrading from Ubuntu 20.04 (clang-12) to Ubuntu 22.04 (clang-14), early builds with the new ubuntu22.04 image populated the S3 Bazel remote cache with stale .cppmap artifacts under the old ubuntu22.04_py* cache key. Subsequent builds hit those cached entries and failed with undeclared inclusion errors in gRPC/absl targets. Encode the compiler version in RAY_BUILD_ENV (ubuntu22.04_clang14_py*) so the cache key is guaranteed unique and no poisoned entries are reused. RAY_BUILD_ENV is already passed to all Bazel actions via --action_env, making it the correct place to encode incompatible environment changes. Topic: ubuntu-22-04-test Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: andrew <andrew@anyscale.com>

… format Ubuntu 22.04's binutils nm (2.38+) emits versioned symbols as symbol@@VERSION_1.0 rather than just symbol. Update the Linux golden file for test_exported_symbols_unix to match. Topic: ubuntu-22-04-test Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: andrew <andrew@anyscale.com>

andrew-anyscale · 2026-03-17T17:24:21Z

conflict resolved

…ct#61533) Upgrades CI test container base images from Ubuntu 20.04 to 22.04 (jammy), including the mandatory clang-12 to clang-14 toolchain upgrade available in 22.04. - focal → jammy for all base images and wanda configs (base.test, base.gpu, base.cu128, forge wanda) - clang-12 → clang-14 in base.test, base.gpu, base.build Dockerfiles and install-llvm-binaries.sh - libncurses5 → libncurses6 - libgl1-mesa-glx → libgl1 (dropped in 22.04) - netcat → netcat-openbsd (22.04 no longer provides the bare netcat metapackage) - mongodb-org apt source: focal → jammy - Java 8 update-alternatives: 22.04 defaults to Java 11, pin to Java 8 where needed - test_dashboard_profiler.py: skip py-spy --native on Linux (known issue, see ray-project#30566) - onnx 1.15.0 → 1.16.0 (compatibility with newer environment) - Simplified Podman doc (20.04 workaround removed) Signed-off-by: andrew <andrew@anyscale.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: Pedro Jeronimo <pedro.jeronimo@tecnico.ulisboa.pt>

…ct#61533) Upgrades CI test container base images from Ubuntu 20.04 to 22.04 (jammy), including the mandatory clang-12 to clang-14 toolchain upgrade available in 22.04. - focal → jammy for all base images and wanda configs (base.test, base.gpu, base.cu128, forge wanda) - clang-12 → clang-14 in base.test, base.gpu, base.build Dockerfiles and install-llvm-binaries.sh - libncurses5 → libncurses6 - libgl1-mesa-glx → libgl1 (dropped in 22.04) - netcat → netcat-openbsd (22.04 no longer provides the bare netcat metapackage) - mongodb-org apt source: focal → jammy - Java 8 update-alternatives: 22.04 defaults to Java 11, pin to Java 8 where needed - test_dashboard_profiler.py: skip py-spy --native on Linux (known issue, see ray-project#30566) - onnx 1.15.0 → 1.16.0 (compatibility with newer environment) - Simplified Podman doc (20.04 workaround removed) Signed-off-by: andrew <andrew@anyscale.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

andrew-anyscale added the ci label Mar 5, 2026

andrew-anyscale changed the title ~~[ci] Upgrade CI test containers from Ubuntu 20.04 to 22.04~~ [ci] upgrade CI test containers from Ubuntu 20.04 to 22.04 Mar 5, 2026

andrew-anyscale force-pushed the andrew/revup/master/ubuntu-22-04-test branch from 1dac0b9 to 2f035d8 Compare March 5, 2026 21:12

gemini-code-assist bot reviewed Mar 5, 2026

View reviewed changes

ci/env/install-llvm-binaries.sh Show resolved Hide resolved

doc/source/serve/advanced-guides/multi-app-container.md Show resolved Hide resolved

andrew-anyscale force-pushed the andrew/revup/master/ubuntu-22-04-test branch from 2f035d8 to 9de2440 Compare March 5, 2026 21:38

andrew-anyscale added the go add ONLY when ready to merge, run all tests label Mar 6, 2026

andrew-anyscale mentioned this pull request Mar 6, 2026

[release] Bump torch version to 2.7.0+cu128 and torchvision #61328

Merged

andrew-anyscale force-pushed the andrew/revup/master/ubuntu-22-04-test branch 4 times, most recently from d47c6a7 to 71222ae Compare March 9, 2026 16:08

exalate-issue-sync bot mentioned this pull request Mar 9, 2026

Release test deepspeed_finetune.aws failed anyscale/ray#894

Closed

andrew-anyscale force-pushed the andrew/revup/master/ubuntu-22-04-test branch from 71222ae to da86625 Compare March 10, 2026 14:35

andrew-anyscale mentioned this pull request Mar 10, 2026

[ci] Patch gRPC BUILD files to disable layering_check #61623

Closed

andrew-anyscale changed the base branch from master to andrew/revup/master/layering-check March 10, 2026 14:36

andrew-anyscale force-pushed the andrew/revup/master/layering-check branch from a8a13fe to aa6c579 Compare March 10, 2026 16:57

andrew-anyscale force-pushed the andrew/revup/master/ubuntu-22-04-test branch from da86625 to 3b35b76 Compare March 10, 2026 16:57

andrew-anyscale changed the title ~~[ci] upgrade CI test containers from Ubuntu 20.04 to 22.04~~ [ci] Patch gRPC BUILD files to disable layering_check Mar 10, 2026

andrew-anyscale changed the base branch from andrew/revup/master/layering-check to master March 10, 2026 19:47

andrew-anyscale force-pushed the andrew/revup/master/ubuntu-22-04-test branch 3 times, most recently from cd58aef to 5c139b2 Compare March 11, 2026 14:17

andrew-anyscale changed the title ~~[ci] Patch gRPC BUILD files to disable layering_check~~ [ci] upgrade CI test containers from Ubuntu 20.04 to 22.04 Mar 11, 2026

andrew-anyscale force-pushed the andrew/revup/master/ubuntu-22-04-test branch from 5c139b2 to eb583c9 Compare March 11, 2026 14:18

andrew-anyscale marked this pull request as ready for review March 11, 2026 17:26

andrew-anyscale requested review from a team as code owners March 11, 2026 17:26

andrew-anyscale marked this pull request as ready for review March 11, 2026 21:50

andrew-anyscale mentioned this pull request Mar 11, 2026

[ci] Upgrade forge Dockerfiles from clang-12 to clang-14 #61662

Merged

andrew-anyscale marked this pull request as draft March 11, 2026 21:52

andrew-anyscale force-pushed the andrew/revup/master/ubuntu-22-04-test branch from 8f1b210 to f66b067 Compare March 11, 2026 22:27

andrew-anyscale force-pushed the andrew/revup/master/data-mongodb-direct-launch branch from 8147de2 to 19bd6b1 Compare March 11, 2026 22:27

andrew-anyscale marked this pull request as ready for review March 11, 2026 22:27

andrew-anyscale force-pushed the andrew/revup/master/ubuntu-22-04-test branch from f66b067 to 4d398b0 Compare March 11, 2026 23:24

andrew-anyscale force-pushed the andrew/revup/master/data-mongodb-direct-launch branch from 19bd6b1 to b250e05 Compare March 11, 2026 23:24

andrew-anyscale force-pushed the andrew/revup/master/ubuntu-22-04-test branch from 4d398b0 to ada6fd6 Compare March 11, 2026 23:38

andrew-anyscale force-pushed the andrew/revup/master/data-mongodb-direct-launch branch 2 times, most recently from 4323abf to 73b0d45 Compare March 12, 2026 00:52

andrew-anyscale force-pushed the andrew/revup/master/ubuntu-22-04-test branch 2 times, most recently from f3d2850 to c70134e Compare March 13, 2026 03:36

andrew-anyscale force-pushed the andrew/revup/master/data-mongodb-direct-launch branch from 73b0d45 to 7cf438c Compare March 13, 2026 03:36

Base automatically changed from andrew/revup/master/data-mongodb-direct-launch to master March 17, 2026 17:19

aslonnie reviewed Mar 17, 2026

View reviewed changes

andrew-anyscale and others added 5 commits March 17, 2026 10:22

andrew-anyscale force-pushed the andrew/revup/master/ubuntu-22-04-test branch from c70134e to 58a676a Compare March 17, 2026 17:23

andrew-anyscale requested a review from aslonnie March 17, 2026 17:24

aslonnie approved these changes Mar 18, 2026

View reviewed changes

aslonnie merged commit c52a126 into master Mar 18, 2026
6 checks passed

aslonnie deleted the andrew/revup/master/ubuntu-22-04-test branch March 18, 2026 00:42

ans9868 mentioned this pull request Mar 24, 2026

[tune] Modernize AxSearch for Ax Platform 1.0.0+ #60522

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ci] upgrade CI test containers from Ubuntu 20.04 to 22.04#61533

[ci] upgrade CI test containers from Ubuntu 20.04 to 22.04#61533
aslonnie merged 5 commits intomasterfrom
andrew/revup/master/ubuntu-22-04-test

andrew-anyscale commented Mar 5, 2026 •

edited

Loading

Uh oh!

andrew-anyscale commented Mar 5, 2026 •

edited

Loading

Uh oh!

andrew-anyscale commented Mar 5, 2026 •

edited

Loading

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

Uh oh!

Uh oh!

aslonnie left a comment

Uh oh!

andrew-anyscale commented Mar 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

andrew-anyscale commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

andrew-anyscale commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

andrew-anyscale commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

aslonnie left a comment

Choose a reason for hiding this comment

Uh oh!

andrew-anyscale commented Mar 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

andrew-anyscale commented Mar 5, 2026 •

edited

Loading

andrew-anyscale commented Mar 5, 2026 •

edited

Loading

andrew-anyscale commented Mar 5, 2026 •

edited

Loading