call reconfigure on rank changes by abrarsheikh · Pull Request #57091 · ray-project/ray

abrarsheikh · 2025-10-01T15:44:39Z

This change ensures that reconfigure is invoked with both user_config and rank under the following conditions:

The user has implemented reconfigure, and redeploys with an updated user_config.
The user has implemented reconfigure, the replica rank changes, user has rank as param in the reconfigure method signature, and deployment_config contains a user_config.

Reconfigure is also invoked at replica startup if, user has implemented reconfigure and has provided some user_config

fixes #57048

Signed-off-by: abrar <abrar@anyscale.com>

gemini-code-assist

Code Review

This pull request introduces a useful feature to trigger reconfigure on replica rank changes, controlled by a new environment variable. The implementation is mostly solid, and the new tests cover the functionality well. However, I've identified a potential backward compatibility issue where the change to the reconfigure method's signature could break existing user deployments. I've suggested a fix to inspect the method signature to avoid this. I also found a minor inaccuracy in a test comment that I've proposed correcting for clarity.

python/ray/serve/_private/replica.py

python/ray/serve/tests/test_replica_ranks.py

kouroshHakha · 2025-10-01T15:49:06Z

python/ray/serve/_private/replica.py

        # NOTE(edoakes): there is the possibility of a race condition in user code if
        # they don't have any form of concurrency control between `reconfigure` and
        # other methods. See https://github.com/ray-project/ray/pull/42159.
+


kouroshHakha · 2025-10-01T15:50:16Z

Logic and flag makes sense. I'll let @zcin and others more familiar with the codebase to review the actual implementation.

Signed-off-by: abrar <abrar@anyscale.com>

landscapepainter

lgtm

python/ray/serve/_private/replica.py

landscapepainter · 2025-10-09T17:15:52Z

python/ray/serve/_private/replica.py

+
+        # NOTE(abrar): The only way to subscribe to rank changes is to provide some user config.
+        # We can relax this in the future as more usecases arise for rank. I am reluctant to
+        # introduce behavior change for a feature we might not need.


Suggested change

# introduce behavior change for a feature we might not need.

# introduce behavior change for a feature we might not need.

Are we explicitly mentioning this as we might remove the rank feature if not needed in the future? If so, maybe should be more explicit about how it potentially can be removed?

might remove the rank feature if not needed in the future?

Not really, i am saying that we can call reconfigure even if user_config is not provided in future.

python/ray/serve/_private/replica.py

…rank

Signed-off-by: abrar <abrar@anyscale.com>

This change ensures that `reconfigure` is invoked with both `user_config` and `rank` under the following conditions: 1. The user has implemented `reconfigure`, and redeploys with an updated `user_config`. 2. The user has implemented `reconfigure`, the replica rank changes, user has rank as param in the `reconfigure` method signature, and `deployment_config` contains a `user_config`. Reconfigure is also invoked at replica startup if, user has implemented `reconfigure` and has provided some `user_config` fixes ray-project#57048 --------- Signed-off-by: abrar <abrar@anyscale.com> Signed-off-by: Pavitra Bhalla <pavitra@rayai.com>

This change ensures that `reconfigure` is invoked with both `user_config` and `rank` under the following conditions: 1. The user has implemented `reconfigure`, and redeploys with an updated `user_config`. 2. The user has implemented `reconfigure`, the replica rank changes, user has rank as param in the `reconfigure` method signature, and `deployment_config` contains a `user_config`. Reconfigure is also invoked at replica startup if, user has implemented `reconfigure` and has provided some `user_config` fixes ray-project#57048 --------- Signed-off-by: abrar <abrar@anyscale.com> Signed-off-by: Josh Kodi <joshkodi@gmail.com>

This change ensures that `reconfigure` is invoked with both `user_config` and `rank` under the following conditions: 1. The user has implemented `reconfigure`, and redeploys with an updated `user_config`. 2. The user has implemented `reconfigure`, the replica rank changes, user has rank as param in the `reconfigure` method signature, and `deployment_config` contains a `user_config`. Reconfigure is also invoked at replica startup if, user has implemented `reconfigure` and has provided some `user_config` fixes ray-project#57048 --------- Signed-off-by: abrar <abrar@anyscale.com>

This change ensures that `reconfigure` is invoked with both `user_config` and `rank` under the following conditions: 1. The user has implemented `reconfigure`, and redeploys with an updated `user_config`. 2. The user has implemented `reconfigure`, the replica rank changes, user has rank as param in the `reconfigure` method signature, and `deployment_config` contains a `user_config`. Reconfigure is also invoked at replica startup if, user has implemented `reconfigure` and has provided some `user_config` fixes #57048 --------- Signed-off-by: abrar <abrar@anyscale.com>

This change ensures that `reconfigure` is invoked with both `user_config` and `rank` under the following conditions: 1. The user has implemented `reconfigure`, and redeploys with an updated `user_config`. 2. The user has implemented `reconfigure`, the replica rank changes, user has rank as param in the `reconfigure` method signature, and `deployment_config` contains a `user_config`. Reconfigure is also invoked at replica startup if, user has implemented `reconfigure` and has provided some `user_config` fixes ray-project#57048 --------- Signed-off-by: abrar <abrar@anyscale.com>

This change ensures that `reconfigure` is invoked with both `user_config` and `rank` under the following conditions: 1. The user has implemented `reconfigure`, and redeploys with an updated `user_config`. 2. The user has implemented `reconfigure`, the replica rank changes, user has rank as param in the `reconfigure` method signature, and `deployment_config` contains a `user_config`. Reconfigure is also invoked at replica startup if, user has implemented `reconfigure` and has provided some `user_config` fixes ray-project#57048 --------- Signed-off-by: abrar <abrar@anyscale.com> Signed-off-by: xgui <xgui@anyscale.com>

This change ensures that `reconfigure` is invoked with both `user_config` and `rank` under the following conditions: 1. The user has implemented `reconfigure`, and redeploys with an updated `user_config`. 2. The user has implemented `reconfigure`, the replica rank changes, user has rank as param in the `reconfigure` method signature, and `deployment_config` contains a `user_config`. Reconfigure is also invoked at replica startup if, user has implemented `reconfigure` and has provided some `user_config` fixes #57048 --------- Signed-off-by: abrar <abrar@anyscale.com> Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>

This change ensures that `reconfigure` is invoked with both `user_config` and `rank` under the following conditions: 1. The user has implemented `reconfigure`, and redeploys with an updated `user_config`. 2. The user has implemented `reconfigure`, the replica rank changes, user has rank as param in the `reconfigure` method signature, and `deployment_config` contains a `user_config`. Reconfigure is also invoked at replica startup if, user has implemented `reconfigure` and has provided some `user_config` fixes ray-project#57048 --------- Signed-off-by: abrar <abrar@anyscale.com>

This change ensures that `reconfigure` is invoked with both `user_config` and `rank` under the following conditions: 1. The user has implemented `reconfigure`, and redeploys with an updated `user_config`. 2. The user has implemented `reconfigure`, the replica rank changes, user has rank as param in the `reconfigure` method signature, and `deployment_config` contains a `user_config`. Reconfigure is also invoked at replica startup if, user has implemented `reconfigure` and has provided some `user_config` fixes ray-project#57048 --------- Signed-off-by: abrar <abrar@anyscale.com> Signed-off-by: Aydin Abiar <aydin@anyscale.com>

This change ensures that `reconfigure` is invoked with both `user_config` and `rank` under the following conditions: 1. The user has implemented `reconfigure`, and redeploys with an updated `user_config`. 2. The user has implemented `reconfigure`, the replica rank changes, user has rank as param in the `reconfigure` method signature, and `deployment_config` contains a `user_config`. Reconfigure is also invoked at replica startup if, user has implemented `reconfigure` and has provided some `user_config` fixes ray-project#57048 --------- Signed-off-by: abrar <abrar@anyscale.com> Signed-off-by: Future-Outlier <eric901201@gmail.com>

call reconfigure on rank changes

64baad0

Signed-off-by: abrar <abrar@anyscale.com>

abrarsheikh requested a review from a team as a code owner October 1, 2025 15:44

abrarsheikh added the go add ONLY when ready to merge, run all tests label Oct 1, 2025

gemini-code-assist bot reviewed Oct 1, 2025

View reviewed changes

python/ray/serve/_private/replica.py Show resolved Hide resolved

python/ray/serve/tests/test_replica_ranks.py Outdated Show resolved Hide resolved

kouroshHakha reviewed Oct 1, 2025

View reviewed changes

This comment was marked as outdated.

Sign in to view

abrarsheikh added 2 commits October 1, 2025 16:18

conditionally pass rank

95e8283

Signed-off-by: abrar <abrar@anyscale.com>

fix local testing

7b8662a

Signed-off-by: abrar <abrar@anyscale.com>

This comment was marked as outdated.

Sign in to view

ray-gardener bot added the serve Ray Serve Related Issue label Oct 1, 2025

abrarsheikh requested review from akyang-anyscale and landscapepainter October 1, 2025 23:08

akyang-anyscale approved these changes Oct 8, 2025

View reviewed changes

landscapepainter approved these changes Oct 9, 2025

View reviewed changes

abrarsheikh added 2 commits October 9, 2025 19:39

Merge branch 'master' of github.com:ray-project/ray into 57048-abrar-…

c612034

…rank

nits

580ab7d

Signed-off-by: abrar <abrar@anyscale.com>

zcin merged commit 6182349 into master Oct 9, 2025
5 of 6 checks passed

zcin deleted the 57048-abrar-rank branch October 9, 2025 23:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

call reconfigure on rank changes#57091

call reconfigure on rank changes#57091
zcin merged 5 commits intomasterfrom
57048-abrar-rank

abrarsheikh commented Oct 1, 2025 •

edited

Loading

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

Uh oh!

Uh oh!

kouroshHakha Oct 1, 2025

Uh oh!

kouroshHakha commented Oct 1, 2025

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

landscapepainter left a comment

Uh oh!

Uh oh!

Uh oh!

landscapepainter Oct 9, 2025

Uh oh!

abrarsheikh Oct 9, 2025

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

	# introduce behavior change for a feature we might not need.
	# introduce behavior change for a feature we might not need.

Conversation

abrarsheikh commented Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

kouroshHakha Oct 1, 2025

Choose a reason for hiding this comment

Uh oh!

kouroshHakha commented Oct 1, 2025

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

landscapepainter left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

landscapepainter Oct 9, 2025

Choose a reason for hiding this comment

Uh oh!

abrarsheikh Oct 9, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

abrarsheikh commented Oct 1, 2025 •

edited

Loading