[Serve] fix that rank logs are spammy#57831
Conversation
…rank reassignment and configuration steps in `deployment_state.py` Change-Id: Idc61828f1686086feb88119a1242dd9c57438d13 Signed-off-by: daiping8 <dai.ping88@zte.com.cn>
There was a problem hiding this comment.
Code Review
This pull request effectively addresses the issue of spammy rank-related logs by changing their level from INFO to DEBUG. The changes are applied consistently across various scenarios, such as rank assignment, reconfiguration, and reallocation. This will improve the signal-to-noise ratio in the logs during normal operation, while still providing detailed information for debugging purposes. The changes are correct and align with the stated goal. Well done.
|
i suggest doing a manual run of the app, observe the controller logs and ensure that there are no other source of logs. |
Could you please share the code to reproduce the issue? Thank you. |
look at test_replica_rank.py, the logs in the GH are from the controller. so after you run |
|
@abrarsheikh Follow your guidance! Thanks!
There are logs related to
There are not logs related to
logs related to The app code is from https://docs.ray.io/en/latest/serve/getting_started.html#running-a-ray-serve-application. |
## Description This PR changes 7 rank related INFO level logs in `python/ray/serve/_private/deployment_state.py` to DEBUG level. These logs primarily appear in the following scenarios: 1. When assigning a rank to a new replica 2. When reconfiguring a replica's rank 3. When rank reallocation is completed 4. When discontinuous ranks are detected and minimal reallocation is performed These logs will only be displayed when debugging is needed, and will not affect log output during normal operation. ## Related issues Closes ray-project#57807 The logs in the screenshot from the issue have all been adjusted to DEBUG level. Signed-off-by: daiping8 <dai.ping88@zte.com.cn>
## Description This PR changes 7 rank related INFO level logs in `python/ray/serve/_private/deployment_state.py` to DEBUG level. These logs primarily appear in the following scenarios: 1. When assigning a rank to a new replica 2. When reconfiguring a replica's rank 3. When rank reallocation is completed 4. When discontinuous ranks are detected and minimal reallocation is performed These logs will only be displayed when debugging is needed, and will not affect log output during normal operation. ## Related issues Closes ray-project#57807 The logs in the screenshot from the issue have all been adjusted to DEBUG level. Signed-off-by: daiping8 <dai.ping88@zte.com.cn> Signed-off-by: xgui <xgui@anyscale.com>
## Description This PR changes 7 rank related INFO level logs in `python/ray/serve/_private/deployment_state.py` to DEBUG level. These logs primarily appear in the following scenarios: 1. When assigning a rank to a new replica 2. When reconfiguring a replica's rank 3. When rank reallocation is completed 4. When discontinuous ranks are detected and minimal reallocation is performed These logs will only be displayed when debugging is needed, and will not affect log output during normal operation. ## Related issues Closes #57807 The logs in the screenshot from the issue have all been adjusted to DEBUG level. Signed-off-by: daiping8 <dai.ping88@zte.com.cn> Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
## Description This PR changes 7 rank related INFO level logs in `python/ray/serve/_private/deployment_state.py` to DEBUG level. These logs primarily appear in the following scenarios: 1. When assigning a rank to a new replica 2. When reconfiguring a replica's rank 3. When rank reallocation is completed 4. When discontinuous ranks are detected and minimal reallocation is performed These logs will only be displayed when debugging is needed, and will not affect log output during normal operation. ## Related issues Closes ray-project#57807 The logs in the screenshot from the issue have all been adjusted to DEBUG level. Signed-off-by: daiping8 <dai.ping88@zte.com.cn>
## Description This PR changes 7 rank related INFO level logs in `python/ray/serve/_private/deployment_state.py` to DEBUG level. These logs primarily appear in the following scenarios: 1. When assigning a rank to a new replica 2. When reconfiguring a replica's rank 3. When rank reallocation is completed 4. When discontinuous ranks are detected and minimal reallocation is performed These logs will only be displayed when debugging is needed, and will not affect log output during normal operation. ## Related issues Closes ray-project#57807 The logs in the screenshot from the issue have all been adjusted to DEBUG level. Signed-off-by: daiping8 <dai.ping88@zte.com.cn> Signed-off-by: Aydin Abiar <aydin@anyscale.com>
## Description This PR changes 7 rank related INFO level logs in `python/ray/serve/_private/deployment_state.py` to DEBUG level. These logs primarily appear in the following scenarios: 1. When assigning a rank to a new replica 2. When reconfiguring a replica's rank 3. When rank reallocation is completed 4. When discontinuous ranks are detected and minimal reallocation is performed These logs will only be displayed when debugging is needed, and will not affect log output during normal operation. ## Related issues Closes ray-project#57807 The logs in the screenshot from the issue have all been adjusted to DEBUG level. Signed-off-by: daiping8 <dai.ping88@zte.com.cn> Signed-off-by: Future-Outlier <eric901201@gmail.com>



Description
This PR changes 7 rank related INFO level logs in
python/ray/serve/_private/deployment_state.pyto DEBUG level. These logs primarily appear in the following scenarios:These logs will only be displayed when debugging is needed, and will not affect log output during normal operation.
Related issues
Closes #57807
The logs in the screenshot from the issue have all been adjusted to DEBUG level.