…roject#54902)

<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

<!-- Please give a short summary of the change and the problem this
solves. -->

If you don't consider budget when scaling up actor pools, then actors
can occupy all of the logical resources and starve other operators.

To fix this issue, this PR updates the autoscaler to consider the budget
as well.

## Related issue number

<!-- For example: "Closes ray-project#1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
Signed-off-by: Krishna Kalyan <krishnakalyan3@gmail.com>

…roject#54902)

<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

<!-- Please give a short summary of the change and the problem this
solves. -->

If you don't consider budget when scaling up actor pools, then actors
can occupy all of the logical resources and starve other operators.

To fix this issue, this PR updates the autoscaler to consider the budget
as well.

## Related issue number

<!-- For example: "Closes ray-project#1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
Signed-off-by: jugalshah291 <shah.jugal291@gmail.com>

…roject#54902)

<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

<!-- Please give a short summary of the change and the problem this
solves. -->

If you don't consider budget when scaling up actor pools, then actors
can occupy all of the logical resources and starve other operators.

To fix this issue, this PR updates the autoscaler to consider the budget
as well.

## Related issue number

<!-- For example: "Closes ray-project#1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
Signed-off-by: Douglas Strodtman <douglas@anyscale.com>

…h GPUs (#59366)

This PR enables the utilization-based autoscaler to work with GPUs.

**Motivation**

The current implementation only considers CPU and memory utilization
when making scaling decisions. This is based on an outdated assumption
that GPUs are only used by actor pools, and we can use actor pool
autoscaling to trigger node scale ups.

This assumption doesn't hold anymore. To fix deadlocks,
#54902 made actor pool
autoscaling respect resource budgets. As a result, the actor pool
autoscaler can't implicitly trigger node autoscaling anymore.

**Changes**

This PR extends the autoscaler to track GPU utilization alongside CPU
and memory:

1. Extract resource utilization calculation into a dedicated abstraction
(ResourceUtilizationGauge): This separates the concern of how
utilization is measured from how it's used
for scaling decisions. The abstraction makes the autoscaler more
testable and opens the door for alternative utilization strategies
(e.g., physical vs logical utilization, different averaging windows).
2. Include GPU nodes in scaling decisions: Previously, GPU nodes were
explicitly filtered out when determining what node types exist in the
cluster. Now all worker node types are considered, allowing the
autoscaler to request additional GPU nodes when needed.
3. Add GPU utilization to the scaling threshold check: The autoscaler
now triggers scale-up when any of CPU, GPU, or memory utilization
exceeds the threshold, rather than only CPU or memory.

---------

Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

…h GPUs (ray-project#59366)

This PR enables the utilization-based autoscaler to work with GPUs.

**Motivation**

The current implementation only considers CPU and memory utilization
when making scaling decisions. This is based on an outdated assumption
that GPUs are only used by actor pools, and we can use actor pool
autoscaling to trigger node scale ups.

This assumption doesn't hold anymore. To fix deadlocks,
ray-project#54902 made actor pool
autoscaling respect resource budgets. As a result, the actor pool
autoscaler can't implicitly trigger node autoscaling anymore.

**Changes**

This PR extends the autoscaler to track GPU utilization alongside CPU
and memory:

1. Extract resource utilization calculation into a dedicated abstraction
(ResourceUtilizationGauge): This separates the concern of how
utilization is measured from how it's used
for scaling decisions. The abstraction makes the autoscaler more
testable and opens the door for alternative utilization strategies
(e.g., physical vs logical utilization, different averaging windows).
2. Include GPU nodes in scaling decisions: Previously, GPU nodes were
explicitly filtered out when determining what node types exist in the
cluster. Now all worker node types are considered, allowing the
autoscaler to request additional GPU nodes when needed.
3. Add GPU utilization to the scaling threshold check: The autoscaler
now triggers scale-up when any of CPU, GPU, or memory utilization
exceeds the threshold, rather than only CPU or memory.

---------

Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Signed-off-by: peterxcli <peterxcli@gmail.com>