[core] (cgroups 4/n) adding constraint bounds checking to the CgroupManager.#56246

Merged

edoakes merged 78 commits intomasterfrom

irabbani/cgroups-4

Sep 6, 2025

Contributor

israbbani commented Sep 4, 2025

This PR continues to implement the CgroupManager. CgroupManager will be used by the Raylet to manage the cgroup hierarchy. The implementation will be completed in subsequent PRs.

This PR stacks on #56186.

For more details about the resource isolation project see #54703.

In this PR:

CgroupManager now bound checks constraints (e.g. cpu.weight is within [1,10000].
CgroupDriver no longer bound checks constraints.

irabbani and others added 30 commits

July 24, 2025 20:39


          [core] (cgroups 1/n) Adding a sys/fs filesystem driver

05c4dbc

to perform cgroup operations.

Signed-off-by: irabbani <irabbani@anyscale.com>


          adding the copyright

645f9a0

Signed-off-by: irabbani <irabbani@anyscale.com>


          Adding a fallback for creating processes inside cgroups with fork/exec

2bb2c5b

instead of clone for older kernel headers < 5.7 (which is what we have
in CI)

Signed-off-by: irabbani <irabbani@anyscale.com>


          adding a pause in the tests to see what's up with the container

Signed-off-by: irabbani <irabbani@anyscale.com>


          Update src/ray/common/cgroup2/cgroup_driver_interface.h

85d0ebf

Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
Signed-off-by: Ibrahim Rabbani <israbbani@gmail.com>


          Comments

3a5a020

Signed-off-by: irabbani <irabbani@anyscale.com>


          Merge branch 'irabbani/cgroups-1' of github.com:ray-project/ray into …

68b0c93

…irabbani/cgroups-1


          Putting the cgroupv2 tests into a separate target

f52354b

Signed-off-by: irabbani <irabbani@anyscale.com>


          removing test sleep

148d04d

Signed-off-by: irabbani <irabbani@anyscale.com>


          Removing a docstring

d3f8b79

Signed-off-by: irabbani <irabbani@anyscale.com>


          enabling CI tests

d76ff15

Signed-off-by: irabbani <irabbani@anyscale.com>


          fixing absl imports

2798ea5

Signed-off-by: irabbani <irabbani@anyscale.com>


          commenting local

3fda505

Signed-off-by: irabbani <irabbani@anyscale.com>


          doxygen formatting

9e1e931

Signed-off-by: irabbani <irabbani@anyscale.com>


          Merge branch 'master' into irabbani/cgroups-1

f066f34


          removing integration tests

e6b4926

Signed-off-by: irabbani <irabbani@anyscale.com>


          final cleanup

f4e0cb2

Signed-off-by: irabbani <irabbani@anyscale.com>


          iwyu

544ba83

Signed-off-by: irabbani <irabbani@anyscale.com>


          Merge branch 'master' into irabbani/cgroups-1

669ba99


          we cpplintin!

2e341d6

Signed-off-by: irabbani <irabbani@anyscale.com>


          Update src/ray/common/cgroup2/sysfs_cgroup_driver.cc

9e46ce6

Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
Signed-off-by: Ibrahim Rabbani <israbbani@gmail.com>


          Apply suggestions from code review

7c745c5

Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
Signed-off-by: Ibrahim Rabbani <israbbani@gmail.com>

bug

d7eb863

Signed-off-by: irabbani <irabbani@anyscale.com>


          Merge branch 'irabbani/cgroups-1' of github.com:ray-project/ray into …

ff64534

…irabbani/cgroups-1


          [core] Integration tests for SysFsCgroupDriver.

da4b475

Signed-off-by: irabbani <irabbani@anyscale.com>


          Cleaning up cgroup_test_utils and attempting to

37e205f

fix CI.

Signed-off-by: irabbani <irabbani@anyscale.com>


          broken

7b83932

Signed-off-by: irabbani <irabbani@anyscale.com>

up

b911d25

Signed-off-by: irabbani <irabbani@anyscale.com>


          upup

63506dc

Signed-off-by: irabbani <irabbani@anyscale.com>


          Merge branch 'master' into irabbani/cgroups-2

e6f1ae9

Signed-off-by: Ibrahim Rabbani <irabbani@anyscale.com>

israbbani marked this pull request as ready for review

September 4, 2025 19:45

israbbani requested a review from a team as a code owner

September 4, 2025 19:45

israbbani mentioned this pull request

[core] (cgroups 5/n) Adding clean up methods to CgroupDriverInterface and SysFsCgroupDriver #56255

Merged


          Merge branch 'master' into irabbani/cgroups-3

israbbani changed the title ~~[core] (cgroups 4/n) adding constraint bounds checking to the CgroupManager.~~ [core] (cgroups 4/n) adding constraint bounds checking to the CgroupManager.

Ibrahim Rabbani and others added 4 commits

September 4, 2025 21:04


          Merge branch 'irabbani/cgroups-3' into irabbani/cgroups-4

fc85704


          Addressing feedback.

b5f6c5e

Signed-off-by: irabbani <irabbani@anyscale.com>


          Merge branch 'master' into irabbani/cgroups-3

03c731e


          Merge branch 'irabbani/cgroups-3' into irabbani/cgroups-4

4aeabf4

israbbani mentioned this pull request

[core] (cgroups 6/n) CgroupManager cleans up the entire cgroup hierarchy in reverse order when destroyed. #56260

Merged

Base automatically changed from irabbani/cgroups-3 to master

September 5, 2025 17:57

Ibrahim Rabbani added 3 commits

September 5, 2025 15:02


          Merge branch 'master' into irabbani/cgroups-4

49e8c75

Signed-off-by: Ibrahim Rabbani <irabbani@anyscale.com>


          fixing bad merge

fe18ef0

Signed-off-by: irabbani <irabbani@anyscale.com>


          Merge branch 'master' into irabbani/cgroups-4

edoakes approved these changes

View reviewed changes

edoakes enabled auto-merge (squash)

September 5, 2025 22:58

edoakes merged commit 704c579 into master

6 checks passed

edoakes deleted the irabbani/cgroups-4 branch

September 6, 2025 00:05

sampan-s-nayak pushed a commit to sampan-s-nayak/ray that referenced this pull request


          [core] (cgroups 4/n) adding constraint bounds checking to the CgroupM…

911f206

…anager. (ray-project#56246)

This PR continues to implement the CgroupManager. CgroupManager will be
used by the Raylet to manage the cgroup hierarchy. The implementation
will be completed in subsequent PRs.

This PR stacks on ray-project#56186.

For more details about the resource isolation project see
ray-project#54703.

In this PR:
* CgroupManager now bound checks constraints (e.g. cpu.weight is within
[1,10000].
* CgroupDriver no longer bound checks constraints.

---------

Signed-off-by: Ibrahim Rabbani <irabbani@anyscale.com>
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
Signed-off-by: sampan <sampan@anyscale.com>

edoakes added a commit that referenced this pull request


          [core] (cgroups 5/n) Adding clean up methods to CgroupDriverInterface…

9536fca

… and SysFsCgroupDriver (#56255)

This PR stacks on #56246.

For more details about the resource isolation project see
#54703.

This PR adds the DeleteCgroup method (along with unit and integration
tests) to the CgroupDriverInterface and the SysFsCgroupDriver to allow
CgroupManager to clean up the cgroup hierarchy is graceful shutdown.

---------

Signed-off-by: Ibrahim Rabbani <irabbani@anyscale.com>
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>

jugalshah291 pushed a commit to jugalshah291/ray_fork that referenced this pull request


          [core] (cgroups 4/n) adding constraint bounds checking to the CgroupM…

3a34a9a

…anager. (ray-project#56246)

This PR continues to implement the CgroupManager. CgroupManager will be
used by the Raylet to manage the cgroup hierarchy. The implementation
will be completed in subsequent PRs.

This PR stacks on ray-project#56186.

For more details about the resource isolation project see
ray-project#54703.

In this PR:
* CgroupManager now bound checks constraints (e.g. cpu.weight is within
[1,10000].
* CgroupDriver no longer bound checks constraints.

---------

Signed-off-by: Ibrahim Rabbani <irabbani@anyscale.com>
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
Signed-off-by: jugalshah291 <shah.jugal291@gmail.com>

jugalshah291 pushed a commit to jugalshah291/ray_fork that referenced this pull request


          [core] (cgroups 5/n) Adding clean up methods to CgroupDriverInterface…

78b82bb

… and SysFsCgroupDriver (ray-project#56255)

This PR stacks on ray-project#56246.

For more details about the resource isolation project see
ray-project#54703.

This PR adds the DeleteCgroup method (along with unit and integration
tests) to the CgroupDriverInterface and the SysFsCgroupDriver to allow
CgroupManager to clean up the cgroup hierarchy is graceful shutdown.

---------

Signed-off-by: Ibrahim Rabbani <irabbani@anyscale.com>
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
Signed-off-by: jugalshah291 <shah.jugal291@gmail.com>

wyhong3103 pushed a commit to wyhong3103/ray that referenced this pull request


          [core] (cgroups 4/n) adding constraint bounds checking to the CgroupM…

fca3657

…anager. (ray-project#56246)

This PR continues to implement the CgroupManager. CgroupManager will be
used by the Raylet to manage the cgroup hierarchy. The implementation
will be completed in subsequent PRs.

This PR stacks on ray-project#56186.

For more details about the resource isolation project see
ray-project#54703.

In this PR:
* CgroupManager now bound checks constraints (e.g. cpu.weight is within
[1,10000].
* CgroupDriver no longer bound checks constraints.

---------

Signed-off-by: Ibrahim Rabbani <irabbani@anyscale.com>
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
Signed-off-by: yenhong.wong <yenhong.wong@grabtaxi.com>

ZacAttack pushed a commit to ZacAttack/ray that referenced this pull request


          [core] (cgroups 4/n) adding constraint bounds checking to the CgroupM…

10b696c

…anager. (ray-project#56246)

This PR continues to implement the CgroupManager. CgroupManager will be
used by the Raylet to manage the cgroup hierarchy. The implementation
will be completed in subsequent PRs.

This PR stacks on ray-project#56186.

For more details about the resource isolation project see
ray-project#54703.

In this PR:
* CgroupManager now bound checks constraints (e.g. cpu.weight is within
[1,10000].
* CgroupDriver no longer bound checks constraints.

---------

Signed-off-by: Ibrahim Rabbani <irabbani@anyscale.com>
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
Signed-off-by: zac <zac@anyscale.com>

ZacAttack pushed a commit to ZacAttack/ray that referenced this pull request


          [core] (cgroups 5/n) Adding clean up methods to CgroupDriverInterface…

e7edc85

… and SysFsCgroupDriver (ray-project#56255)

This PR stacks on ray-project#56246.

For more details about the resource isolation project see
ray-project#54703.

This PR adds the DeleteCgroup method (along with unit and integration
tests) to the CgroupDriverInterface and the SysFsCgroupDriver to allow
CgroupManager to clean up the cgroup hierarchy is graceful shutdown.

---------

Signed-off-by: Ibrahim Rabbani <irabbani@anyscale.com>
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
Signed-off-by: zac <zac@anyscale.com>

dstrodtman pushed a commit that referenced this pull request


          [core] (cgroups 4/n) adding constraint bounds checking to the CgroupM…

c4de2f5

…anager. (#56246)

This PR continues to implement the CgroupManager. CgroupManager will be
used by the Raylet to manage the cgroup hierarchy. The implementation
will be completed in subsequent PRs.

This PR stacks on #56186.

For more details about the resource isolation project see
#54703.

In this PR:
* CgroupManager now bound checks constraints (e.g. cpu.weight is within
[1,10000].
* CgroupDriver no longer bound checks constraints.

---------

Signed-off-by: Ibrahim Rabbani <irabbani@anyscale.com>
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
Signed-off-by: Douglas Strodtman <douglas@anyscale.com>

dstrodtman pushed a commit to dstrodtman/ray that referenced this pull request


          [core] (cgroups 5/n) Adding clean up methods to CgroupDriverInterface…

d82c8da

… and SysFsCgroupDriver (ray-project#56255)

This PR stacks on ray-project#56246.

For more details about the resource isolation project see
ray-project#54703.

This PR adds the DeleteCgroup method (along with unit and integration
tests) to the CgroupDriverInterface and the SysFsCgroupDriver to allow
CgroupManager to clean up the cgroup hierarchy is graceful shutdown.

---------

Signed-off-by: Ibrahim Rabbani <irabbani@anyscale.com>
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
Signed-off-by: Douglas Strodtman <douglas@anyscale.com>

justinyeh1995 pushed a commit to justinyeh1995/ray that referenced this pull request


          [core] (cgroups 5/n) Adding clean up methods to CgroupDriverInterface…

941819e

… and SysFsCgroupDriver (ray-project#56255)

This PR stacks on ray-project#56246.

For more details about the resource isolation project see
ray-project#54703.

This PR adds the DeleteCgroup method (along with unit and integration
tests) to the CgroupDriverInterface and the SysFsCgroupDriver to allow
CgroupManager to clean up the cgroup hierarchy is graceful shutdown.

---------

Signed-off-by: Ibrahim Rabbani <irabbani@anyscale.com>
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>

landscapepainter pushed a commit to landscapepainter/ray that referenced this pull request


          [core] (cgroups 4/n) adding constraint bounds checking to the CgroupM…

0c294e2

…anager. (ray-project#56246)

This PR continues to implement the CgroupManager. CgroupManager will be
used by the Raylet to manage the cgroup hierarchy. The implementation
will be completed in subsequent PRs.

This PR stacks on ray-project#56186.

For more details about the resource isolation project see
ray-project#54703.

In this PR:
* CgroupManager now bound checks constraints (e.g. cpu.weight is within
[1,10000].
* CgroupDriver no longer bound checks constraints.

---------

Signed-off-by: Ibrahim Rabbani <irabbani@anyscale.com>
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>

landscapepainter pushed a commit to landscapepainter/ray that referenced this pull request


          [core] (cgroups 5/n) Adding clean up methods to CgroupDriverInterface…

288b006

… and SysFsCgroupDriver (ray-project#56255)

This PR stacks on ray-project#56246.

For more details about the resource isolation project see
ray-project#54703.

This PR adds the DeleteCgroup method (along with unit and integration
tests) to the CgroupDriverInterface and the SysFsCgroupDriver to allow
CgroupManager to clean up the cgroup hierarchy is graceful shutdown.

---------

Signed-off-by: Ibrahim Rabbani <irabbani@anyscale.com>
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core go