[docs][serve.llm] Add Ray Serve LLM docs examples to test#54763
[docs][serve.llm] Add Ray Serve LLM docs examples to test#54763kouroshHakha merged 16 commits intoray-project:masterfrom
Conversation
There was a problem hiding this comment.
Summary of Changes
Hello @eicherseiji, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request focuses on improving the testability and maintainability of Ray Serve LLM documentation examples. It achieves this by extracting an existing LLM deployment configuration into a dedicated, testable Python script, which is then used to dynamically generate the documentation snippet. The changes also include adjustments to facilitate automated testing of this example within CI environments.
Highlights
- Test Coverage Improvement: Extracted the Qwen LLM example from the Ray Serve documentation into a new, dedicated Python file (
doc/source/serve/doc_code/qwen_example.py). This enables automated testing of the example configuration, significantly improving test coverage for the Ray Serve LLM landing page. - CI/Testing Enhancements: Implemented monkeypatching within the extracted example to ensure
serve.runoperates in a non-blocking mode and to removeaccelerator_typerequirements. This makes the example suitable for execution in CI environments without needing actual hardware accelerators, streamlining automated testing. - Documentation Synchronization: Updated the
serving-llms.rstdocumentation to use aliteralincludedirective. This change ensures that the Qwen LLM example displayed in the documentation is directly pulled from the newly created test file, guaranteeing consistency between the documentation and the testable code. - Documentation Rendering Improvement: Modified the
autopydantic.rsttemplate to enable the display of field summaries and default values for Pydantic models in the generated documentation. This enhances the clarity and detail of configuration object documentation.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
8f8136d to
454cf04
Compare
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
…e-llm-doc-test
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
…e-llm-doc-test
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
?
kouroshHakha
left a comment
There was a problem hiding this comment.
I think we should do some file reorg before merging. Let me know what you think?
doc/source/llm/examples/serve/BUILD
Outdated
There was a problem hiding this comment.
can we put all of these in doc/source/llm/doc_code/<serve_qwen>/ or sth with a better name in brackets <>?
doc/source/llm/examples/serve/BUILD
Outdated
| srcs = glob(["*.py"]), | ||
| visibility = ["//doc:__subpackages__"], | ||
| ) | ||
|
|
There was a problem hiding this comment.
could we simply add the content of this file to ray/doc/BUILD instead of creating a separate BUILD here? Basically I want to not create new opinionated paths for where the doc tests are. We can use the same convention for adding a new section in that file to introduce a section for all ray serve llm doc tests.
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
Signed-off-by: Seiji Eicher <seiji@anyscale.com> Signed-off-by: sampan <sampan@anyscale.com>
…t#54763) Signed-off-by: Seiji Eicher <seiji@anyscale.com> Signed-off-by: Andrew Grosser <dioptre@gmail.com>
…t#54763) Signed-off-by: Seiji Eicher <seiji@anyscale.com> Signed-off-by: jugalshah291 <shah.jugal291@gmail.com>
…t#54763) Signed-off-by: Seiji Eicher <seiji@anyscale.com> Signed-off-by: Douglas Strodtman <douglas@anyscale.com>
Why are these changes needed?
Improve test coverage for example config from Ray Serve LLM landing page: https://docs.ray.io/en/latest/serve/llm/serving-llms.html
Related issue number
Checks
git commit -s) in this PR.scripts/format.shto lint the changes in this PR.method in Tune, I've added it in
doc/source/tune/api/under thecorresponding
.rstfile.