[serve] Update documentation for new `DeploymentHandle` API by edoakes · Pull Request #39319 · ray-project/ray

edoakes · 2023-09-06T15:19:48Z

Why are these changes needed?

Updates the documentation for the first stage of rolling out the new DeploymentHandle API:

The new types have been added to the API reference with docstrings.
The docstrings for the existing handle types have been updated to prod people to use the new API.
I've added a note to the model composition guide and the API ref describing the changes.
The model composition guide has been updated to describe the new DeploymentHandle (including streaming support) vs. the old ones.
All examples using handles are updated to opt into the new API via handle.options(use_new_handle_api=True).

Related issue number

Checks

I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
I've run scripts/format.sh to lint the changes in this PR.
I've included any doc changes needed for https://docs.ray.io/en/master/.
- I've added any new APIs to the API Reference. For example, if I added a
  method in Tune, I've added it in doc/source/tune/api/ under the
  corresponding .rst file.
I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
Testing Strategy
- Unit tests
- Release tests
- This PR is not tested :(

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

…le-docs

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

…le-docs

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

angelinalg

Just some nits. Thanks for all of this great docs work.

In case you're wondering about my suggestions to replace via. Using via is discouraged in technical writing because it is often ambiguous and sometime too formal sounding. I didn't make suggestions for some occurrences of via because I wasn't sure what the better word woudl be. Some options are: through, with, using, in. Hope that helps.

doc/source/serve/advanced-guides/deployment-graphs.md

doc/source/serve/advanced-guides/dev-workflow.md

doc/source/serve/tutorials/aws-neuron-core-inference.md

python/ray/serve/handle.py

edoakes · 2023-09-07T01:02:43Z

Just some nits. Thanks for all of this great docs work.

In case you're wondering about my suggestions to replace via. Using via is discouraged in technical writing because it is often ambiguous and sometime too formal sounding. I didn't make suggestions for some occurrences of via because I wasn't sure what the better word woudl be. Some options are: through, with, using, in. Hope that helps.

Thanks for the added context this is helpful :)

zcin

Some non-blockers, great job!

doc/source/serve/architecture.md

doc/source/serve/key-concepts.md

doc/source/serve/architecture.md

doc/source/serve/advanced-guides/deployment-graphs.md

doc/source/serve/architecture.md

python/ray/serve/handle.py

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

Co-authored-by: angelinalg <122562471+angelinalg@users.noreply.github.com> Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

…le-docs

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

Co-authored-by: angelinalg <122562471+angelinalg@users.noreply.github.com> Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

zcin

Small note about the deployment handle explainer link, otherwise LGTM!

doc/source/serve/architecture.md

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

…le-docs

…ect#39319) Updates the documentation for the first stage of rolling out the new `DeploymentHandle` API: - The new types have been added to the API reference with docstrings. - The docstrings for the existing handle types have been updated to prod people to use the new API. - I've added a `note` to the model composition guide and the API ref describing the changes. - The model composition guide has been updated to describe the new `DeploymentHandle` (including streaming support) vs. the old ones. - All examples using handles are updated to opt into the new API via `handle.options(use_new_handle_api=True)`. Co-authored-by: angelinalg <122562471+angelinalg@users.noreply.github.com> Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

…39486) Updates the documentation for the first stage of rolling out the new `DeploymentHandle` API: - The new types have been added to the API reference with docstrings. - The docstrings for the existing handle types have been updated to prod people to use the new API. - I've added a `note` to the model composition guide and the API ref describing the changes. - The model composition guide has been updated to describe the new `DeploymentHandle` (including streaming support) vs. the old ones. - All examples using handles are updated to opt into the new API via `handle.options(use_new_handle_api=True)`. Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com> Signed-off-by: Gene Der Su <e870252314@gmail.com> Co-authored-by: angelinalg <122562471+angelinalg@users.noreply.github.com> Co-authored-by: Gene Der Su <e870252314@gmail.com>

…ect#39319) Updates the documentation for the first stage of rolling out the new `DeploymentHandle` API: - The new types have been added to the API reference with docstrings. - The docstrings for the existing handle types have been updated to prod people to use the new API. - I've added a `note` to the model composition guide and the API ref describing the changes. - The model composition guide has been updated to describe the new `DeploymentHandle` (including streaming support) vs. the old ones. - All examples using handles are updated to opt into the new API via `handle.options(use_new_handle_api=True)`. Co-authored-by: angelinalg <122562471+angelinalg@users.noreply.github.com> Signed-off-by: Jim Thompson <jimthompson5802@gmail.com>

…ect#39319) Updates the documentation for the first stage of rolling out the new `DeploymentHandle` API: - The new types have been added to the API reference with docstrings. - The docstrings for the existing handle types have been updated to prod people to use the new API. - I've added a `note` to the model composition guide and the API ref describing the changes. - The model composition guide has been updated to describe the new `DeploymentHandle` (including streaming support) vs. the old ones. - All examples using handles are updated to opt into the new API via `handle.options(use_new_handle_api=True)`. Co-authored-by: angelinalg <122562471+angelinalg@users.noreply.github.com> Signed-off-by: Victor <vctr.y.m@example.com>

edoakes requested review from a team, architkulkarni, shrekris-anyscale, sihanwang41 and zcin as code owners September 6, 2023 15:19

edoakes added 12 commits September 6, 2023 10:20

API ref

78622fd

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

update getting started

cc4dbde

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

update key-concepts

b67b7ae

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

fix

2624a86

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

update multi-app.md

84705d1

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

fix key concept links

585570c

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

update batching code

e69f136

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

fix

6b46c07

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

update more things

c99859a

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

finished going through tutorials

87faca1

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

remove dead code

727f391

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

fix

71c727e

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

edoakes mentioned this pull request Sep 6, 2023

[Serve][Doc] Add handle instruction to send multiplex request #39274

Merged

8 tasks

edoakes added 7 commits September 6, 2023 13:28

more fixes

d493ee2

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

fix

17df206

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

Merge branch 'master' of https://github.com/ray-project/ray into hand…

8425b17

…le-docs

Merge branch 'master' of https://github.com/ray-project/ray into hand…

79a729b

…le-docs

fix

f187609

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

fix

4a82efa

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

fix

c1de52a

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

edoakes requested review from a team and akshay-anyscale September 6, 2023 20:43

edoakes self-assigned this Sep 6, 2023

edoakes added 2 commits September 6, 2023 15:46

fix

6184297

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

Merge branch 'master' of https://github.com/ray-project/ray into hand…

f4d5432

…le-docs

fixes

ecaf177

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

angelinalg approved these changes Sep 6, 2023

View reviewed changes

zcin approved these changes Sep 8, 2023

View reviewed changes

doc/source/serve/architecture.md Outdated Show resolved Hide resolved

doc/source/serve/key-concepts.md Show resolved Hide resolved

doc/source/serve/key-concepts.md Outdated Show resolved Hide resolved

doc/source/serve/architecture.md Outdated Show resolved Hide resolved

shrekris-anyscale approved these changes Sep 8, 2023

View reviewed changes

doc/source/serve/advanced-guides/deployment-graphs.md Outdated Show resolved Hide resolved

doc/source/serve/architecture.md Outdated Show resolved Hide resolved

edoakes commented Sep 8, 2023

View reviewed changes

python/ray/serve/handle.py Outdated Show resolved Hide resolved

edoakes and others added 12 commits September 8, 2023 09:47

warning

eb82598

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

Apply suggestions from code review

25d8a80

Co-authored-by: angelinalg <122562471+angelinalg@users.noreply.github.com> Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

fix

5b3e7dc

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

fix

7573cb3

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

Merge branch 'master' of https://github.com/ray-project/ray into hand…

516c241

…le-docs

fix merge conflict

7447a5a

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

revert smol change

f372842

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

another smol smol change

a0f484e

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

Apply suggestions from code review

999630d

Co-authored-by: angelinalg <122562471+angelinalg@users.noreply.github.com> Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

fix

dde956b

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

fix key concepts

d6c369c

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

added new section

3935a2b

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

akshay-anyscale approved these changes Sep 8, 2023

View reviewed changes

zcin approved these changes Sep 8, 2023

View reviewed changes

doc/source/serve/architecture.md Outdated Show resolved Hide resolved

edoakes added 3 commits September 8, 2023 11:31

fix

58b006f

Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

Merge branch 'master' of https://github.com/ray-project/ray into hand…

5fac95f

…le-docs

Merge branch 'master' of https://github.com/ray-project/ray into hand…

b9c973d

…le-docs

edoakes merged commit d842d77 into ray-project:master Sep 8, 2023

edoakes mentioned this pull request Sep 8, 2023

[serve] Update documentation for new DeploymentHandle API #39486

Merged

8 tasks

Conversation

edoakes commented Sep 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why are these changes needed?

Related issue number

Checks

Uh oh!

angelinalg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

edoakes commented Sep 7, 2023

Uh oh!

zcin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

zcin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

edoakes commented Sep 6, 2023 •

edited

Loading