[Serve] Mark per-deployment autoscaling metrics push interval config deprecated by JoshKarpel · Pull Request #55102 · ray-project/ray

JoshKarpel · 2025-07-31T14:41:19Z

Why are these changes needed?

Deprecation announcements for this future PR #55016

Related issue number

Related to #45777

Checks

I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
I've run scripts/format.sh to lint the changes in this PR.
I've included any doc changes needed for https://docs.ray.io/en/master/.
- I've added any new APIs to the API Reference. For example, if I added a
  method in Tune, I've added it in doc/source/tune/api/ under the
  corresponding .rst file.
I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
Testing Strategy
- Unit tests
- Release tests
- This PR is not tested :(

Signed-off-by: Josh Karpel <josh.karpel@gmail.com>

gemini-code-assist

Summary of Changes

Hello @JoshKarpel, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request is a preparatory step for future changes in Ray Serve's autoscaling configuration. It deprecates the existing metrics_interval_s field in AutoscalingConfig and introduces new, more specific environment variables for controlling autoscaling metric push and record intervals. This change aims to provide more explicit and granular control over these critical timing parameters.

Highlights

Deprecation of metrics_interval_s: The metrics_interval_s field within Ray Serve's AutoscalingConfig (in both Python and its protobuf definition) has been marked as deprecated. This signals a shift towards environment variable-based configuration for autoscaling metric push intervals.
Introduction of New Environment Variable-backed Constants: New constants backed by environment variables have been introduced in python/ray/serve/_private/constants.py to provide more granular control over autoscaling metric push intervals. These include RAY_SERVE_REPLICA_AUTOSCALING_METRIC_PUSH_INTERVAL_S and RAY_SERVE_HANDLE_AUTOSCALING_METRIC_PUSH_INTERVAL_S.
Renaming and Externalizing Record Intervals: Existing internal constants for autoscaling metric record periods (e.g., RAY_SERVE_REPLICA_AUTOSCALING_METRIC_RECORD_PERIOD_S) have been renamed to use _INTERVAL_S (e.g., RAY_SERVE_REPLICA_AUTOSCALING_METRIC_RECORD_INTERVAL_S) and are now configurable via environment variables.
Internal Code Migration: Internal code in python/ray/serve/_private/replica.py and python/ray/serve/_private/router.py has been updated to utilize the newly introduced or renamed constants for autoscaling metric recording and pushing, ensuring consistency with the new configuration approach.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments or fill out our survey to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request effectively deprecates the per-deployment autoscaling metrics push interval configuration, replacing it with new environment variables. The changes are consistent across the codebase, including updates to configurations, constants, and documentation. I have suggestions to improve the clarity and correctness of the fallback logic for the new environment variables in python/ray/serve/_private/constants.py and to update the deprecated comments in python/ray/serve/config.py and src/ray/protobuf/serve.proto.

python/ray/serve/_private/constants.py

python/ray/serve/config.py

src/ray/protobuf/serve.proto

Signed-off-by: Josh Karpel <josh.karpel@gmail.com>

…s-pushing

Signed-off-by: Josh Karpel <josh.karpel@gmail.com>

JoshKarpel · 2025-07-31T19:23:48Z

/gemini review

gemini-code-assist

Code Review

This pull request deprecates the metrics_interval_s configuration, introduces new environment variables for configuring autoscaling metric push intervals, and provides backward compatibility for the legacy environment variable. The deprecation is communicated through code comments, docstrings, and a DeprecationWarning. The tests have been updated to reflect these changes. I've provided suggestions to enhance code readability.

python/ray/serve/_private/constants.py

…s-pushing

Signed-off-by: Josh Karpel <josh.karpel@gmail.com>

python/ray/setup-dev.py

Signed-off-by: Josh Karpel <josh.karpel@gmail.com>

zcin · 2025-08-01T20:13:25Z

python/ray/serve/config.py

        return max_replicas

+    @validator("metrics_interval_s")
+    def metrics_interval_s_deprecation_warning(cls, v: PositiveFloat) -> PositiveFloat:


@JoshKarpel do you know if users use serve.deployment(autoscaling_config={"metrics_interval_s": 10}) without using the AutoscalingConfig directly, will this print the warning? And also if they set it in a serve config and deploy it, will it print in controller logs?

Ah, good question, let me give that a try

Added these cases to the test! They do indeed raise the warning in both of those situations.

Signed-off-by: Josh Karpel <josh.karpel@gmail.com>

zcin

looks good!

python/ray/serve/api.py

python/ray/serve/_private/config.py

Signed-off-by: Josh Karpel <josh.karpel@gmail.com>

…s-pushing

JoshKarpel · 2025-08-07T01:03:50Z

Thanks @zcin !

…deprecated (#55102) ## Why are these changes needed? Deprecation announcements for this future PR #55016 ## Related issue number Related to #45777 ## Checks - [x] I've signed off every commit(by using the -s flag, i.e., `git commit -s`) in this PR. - [x] I've run `scripts/format.sh` to lint the changes in this PR. - [x] I've included any doc changes needed for https://docs.ray.io/en/master/. - [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in `doc/source/tune/api/` under the corresponding `.rst` file. - [x] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/ - Testing Strategy - [x] Unit tests - [ ] Release tests - [ ] This PR is not tested :( --------- Signed-off-by: Josh Karpel <josh.karpel@gmail.com> Signed-off-by: sampan <sampan@anyscale.com>

…deprecated (ray-project#55102) ## Why are these changes needed? Deprecation announcements for this future PR ray-project#55016 ## Related issue number Related to ray-project#45777 ## Checks - [x] I've signed off every commit(by using the -s flag, i.e., `git commit -s`) in this PR. - [x] I've run `scripts/format.sh` to lint the changes in this PR. - [x] I've included any doc changes needed for https://docs.ray.io/en/master/. - [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in `doc/source/tune/api/` under the corresponding `.rst` file. - [x] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/ - Testing Strategy - [x] Unit tests - [ ] Release tests - [ ] This PR is not tested :( --------- Signed-off-by: Josh Karpel <josh.karpel@gmail.com> Signed-off-by: Andrew Grosser <dioptre@gmail.com>

…deprecated (ray-project#55102) ## Why are these changes needed? Deprecation announcements for this future PR ray-project#55016 ## Related issue number Related to ray-project#45777 ## Checks - [x] I've signed off every commit(by using the -s flag, i.e., `git commit -s`) in this PR. - [x] I've run `scripts/format.sh` to lint the changes in this PR. - [x] I've included any doc changes needed for https://docs.ray.io/en/master/. - [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in `doc/source/tune/api/` under the corresponding `.rst` file. - [x] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/ - Testing Strategy - [x] Unit tests - [ ] Release tests - [ ] This PR is not tested :( --------- Signed-off-by: Josh Karpel <josh.karpel@gmail.com> Signed-off-by: jugalshah291 <shah.jugal291@gmail.com>

…deprecated (#55102) ## Why are these changes needed? Deprecation announcements for this future PR #55016 ## Related issue number Related to #45777 ## Checks - [x] I've signed off every commit(by using the -s flag, i.e., `git commit -s`) in this PR. - [x] I've run `scripts/format.sh` to lint the changes in this PR. - [x] I've included any doc changes needed for https://docs.ray.io/en/master/. - [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in `doc/source/tune/api/` under the corresponding `.rst` file. - [x] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/ - Testing Strategy - [x] Unit tests - [ ] Release tests - [ ] This PR is not tested :( --------- Signed-off-by: Josh Karpel <josh.karpel@gmail.com> Signed-off-by: Douglas Strodtman <douglas@anyscale.com>

deprecate per-deployment autoscaling metrics push interval config

3ea16b2

Signed-off-by: Josh Karpel <josh.karpel@gmail.com>

gemini-code-assist bot reviewed Jul 31, 2025

View reviewed changes

python/ray/serve/_private/constants.py Show resolved Hide resolved

python/ray/serve/config.py Show resolved Hide resolved

src/ray/protobuf/serve.proto Show resolved Hide resolved

JoshKarpel added 2 commits July 31, 2025 09:58

add deprecation warning on non-default value

4c8fbd3

Signed-off-by: Josh Karpel <josh.karpel@gmail.com>

use new name everywhere

cef4101

Signed-off-by: Josh Karpel <josh.karpel@gmail.com>

JoshKarpel marked this pull request as ready for review July 31, 2025 19:18

JoshKarpel requested a review from a team as a code owner July 31, 2025 19:18

JoshKarpel added 2 commits July 31, 2025 14:18

Merge branch 'master' into announce-deprecations-for-amortized-metric…

f37aaf0

…s-pushing

better env var handling

5bfc066

Signed-off-by: Josh Karpel <josh.karpel@gmail.com>

gemini-code-assist bot reviewed Jul 31, 2025

View reviewed changes

python/ray/serve/_private/constants.py Show resolved Hide resolved

ray-gardener bot added community-contribution Contributed by the community serve Ray Serve Related Issue docs An issue or change related to documentation deprecation Scheduled for deprecation labels Aug 1, 2025

JoshKarpel added 2 commits August 1, 2025 09:19

Merge branch 'master' into announce-deprecations-for-amortized-metric…

b5bf56b

…s-pushing

include fix for setup script

20ab5f0

Signed-off-by: Josh Karpel <josh.karpel@gmail.com>

JoshKarpel requested a review from a team as a code owner August 1, 2025 14:26

JoshKarpel commented Aug 1, 2025

View reviewed changes

python/ray/setup-dev.py Show resolved Hide resolved

revert setup-dev.py changes in this PR

65ce113

Signed-off-by: Josh Karpel <josh.karpel@gmail.com>

zcin reviewed Aug 1, 2025

View reviewed changes

add more cases where the deprecation warning will fire

a827b46

Signed-off-by: Josh Karpel <josh.karpel@gmail.com>

zcin reviewed Aug 4, 2025

View reviewed changes

python/ray/serve/api.py Outdated Show resolved Hide resolved

python/ray/serve/_private/config.py Outdated Show resolved Hide resolved

JoshKarpel added 2 commits August 4, 2025 11:10

Apply suggestions from code review

ea7eac5

Signed-off-by: Josh Karpel <josh.karpel@gmail.com>

Merge branch 'master' into announce-deprecations-for-amortized-metric…

46f8a49

…s-pushing

zcin approved these changes Aug 6, 2025

View reviewed changes

zcin added the go add ONLY when ready to merge, run all tests label Aug 6, 2025

zcin enabled auto-merge (squash) August 6, 2025 23:28

zcin merged commit 60d6d2d into ray-project:master Aug 7, 2025
7 checks passed

JoshKarpel deleted the announce-deprecations-for-amortized-metrics-pushing branch August 7, 2025 01:03

JoshKarpel mentioned this pull request Aug 27, 2025

[Serve] Group DeploymentHandle autoscaling metrics pushes by process #55016

Open

8 tasks

Conversation

JoshKarpel commented Jul 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why are these changes needed?

Related issue number

Checks

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JoshKarpel commented Jul 31, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

zcin Aug 1, 2025

Choose a reason for hiding this comment

Uh oh!

JoshKarpel Aug 1, 2025

Choose a reason for hiding this comment

Uh oh!

JoshKarpel Aug 1, 2025

Choose a reason for hiding this comment

Uh oh!

zcin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JoshKarpel commented Aug 7, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

JoshKarpel commented Jul 31, 2025 •

edited

Loading