Skip to content

[serve.llm] Organize spread out utils.py#53722

Merged
kouroshHakha merged 1 commit intoray-project:masterfrom
nrghosh:organize-utils
Jun 13, 2025
Merged

[serve.llm] Organize spread out utils.py#53722
kouroshHakha merged 1 commit intoray-project:masterfrom
nrghosh:organize-utils

Conversation

@nrghosh
Copy link
Copy Markdown
Contributor

@nrghosh nrghosh commented Jun 10, 2025 •
edited
Loading

Refactor utils.py into to llm/_internal/common

Why are these changes needed?

Related issue number

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

@nrghosh nrghosh requested a review from a team as a code owner June 10, 2025 22:20
@nrghosh nrghosh requested a review from kouroshHakha June 10, 2025 22:21
@nrghosh nrghosh changed the title [LLM-1784] - Organize spread out utils.py (serve.llm) Organize spread out utils.py Jun 10, 2025
@nrghosh
[LLM-1784] - Organize spread out utils.py
4fa6c10 
- move to llm/_internal/common
- lint fixes

Signed-off-by: Nikhil Ghosh <nikhil@anyscale.com>
@nrghosh nrghosh added the go add ONLY when ready to merge, run all tests label Jun 10, 2025
@nrghosh nrghosh requested a review from eicherseiji June 11, 2025 18:34
@kouroshHakha kouroshHakha changed the title (serve.llm) Organize spread out utils.py [serve.llm] Organize spread out utils.py Jun 13, 2025
@kouroshHakha kouroshHakha merged commit ab1a29f into ray-project:master Jun 13, 2025
6 checks passed
@nrghosh
Copy link
Copy Markdown
Contributor Author

nrghosh commented Jun 13, 2025

Reason for not doing a full refactor / re-org of all python/ray/llm/ utils:

_internal/common is for utilities to be shared between serve and batch (ex: download_model_utils). While the other utilities are spread out and not well organized, only import_utils fits the criteria for this. Further clean-up is left for the future.

elliot-barn pushed a commit that referenced this pull request Jun 18, 2025
@nrghosh @elliot-barn
[serve.llm] Organize spread out utils.py (#53722)
e8e0bc5 
Signed-off-by: Nikhil Ghosh <nikhil@anyscale.com>
Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
elliot-barn pushed a commit that referenced this pull request Jul 2, 2025
@nrghosh @elliot-barn
[serve.llm] Organize spread out utils.py (#53722)
64e0283 
Signed-off-by: Nikhil Ghosh <nikhil@anyscale.com>
Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kouroshHakha kouroshHakha kouroshHakha approved these changes

@eicherseiji eicherseiji Awaiting requested review from eicherseiji

Assignees

No one assigned

Labels

go add ONLY when ready to merge, run all tests

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@nrghosh @kouroshHakha