[Serve][1/n] Improve type annotations for DeploymentHandle, DeploymentResponse, and DeploymentResponseGenerator by abrarsheikh · Pull Request #59363 · ray-project/ray

abrarsheikh · 2025-12-10T22:33:52Z

Summary

This PR improves the generic type annotations on Ray Serve's handle classes to enable better IDE support and type inference. It also adds a type checking test file to verify the annotations work correctly.

Changes

Type Annotation Improvements (`handle.py`)

Generic return types: Updated methods to return the generic type parameter R instead of Any:
- DeploymentResponse.result() → R
- DeploymentResponse.__await__() → Generator[Any, None, R]
- DeploymentResponseGenerator.__iter__() → Iterator[R]
- DeploymentResponseGenerator.__next__() → R
- DeploymentResponseGenerator.__aiter__() → AsyncIterator[R]
- DeploymentResponseGenerator.__anext__() → R
mypy compatibility fixes:
- Added cast() for Future types in sync/async fetch methods to help mypy understand the conditional types
- Aligned _DeploymentHandleBase.options() signature with DeploymentHandle.options() to fix override error
- Refactored remote() to return directly from each branch instead of assigning different class types to a variable

Type Checking Tests (`check_handle_typing.py`)

Added a mypy test file that verifies:

Generic type R is preserved through result(), __await__, iteration methods
Generic type T is preserved through options() and method access
Placeholder tests for future mypy plugin (commented out) that will verify method return type inference

How to Test

    mypy python/ray/serve/tests/typing_files/check_handle_typing.py python/ray/serve/handle.py  \
        --follow-imports=skip \
        --ignore-missing-imports

Future Work

A mypy plugin can be implemented to infer the return type of .remote() based on which deployment method is being called:

handle: DeploymentHandle[MyDeployment]
response = handle.get_user.remote(123)  # Plugin would infer DeploymentResponse[str]
user = response.result()                 # Would be typed as str

The test file includes commented-out tests that should pass once the plugin is implemented.

Tests

From master

❯ mypy python/ray/serve/tests/typing_files/check_handle_typing.py python/ray/serve/handle.py  --follow-imports=skip --ignore-missing-imports
python/ray/serve/handle.py:10: error: Module "ray._raylet" has no attribute "ObjectRefGenerator"  [attr-defined]
python/ray/serve/handle.py:162: error: Item "None" of "Optional[Any]" has no attribute "_run_router_in_separate_loop"  [union-attr]
python/ray/serve/handle.py:210: error: Item "None" of "Optional[Any]" has no attribute "assign_request"  [union-attr]
python/ray/serve/handle.py:229: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
python/ray/serve/handle.py:292: error: Unexpected keyword argument "timeout" for "result" of "Future"  [call-arg]
/home/ubuntu/.local/lib/python3.9/site-packages/mypy/typeshed/stdlib/_asyncio.pyi: note: "result" of "Future" defined here
python/ray/serve/handle.py:319: error: Incompatible types in "await" (actual type "Union[concurrent.futures._base.Future[Any], _asyncio.Future[Any]]", expected type "Awaitable[Any]")  [misc]
python/ray/serve/handle.py:803: error: Incompatible types in assignment (expression has type "type[DeploymentResponse]", variable has type "type[DeploymentResponseGenerator]")  [assignment]
python/ray/serve/tests/typing_files/check_handle_typing.py:24: error: "DeploymentResponse" expects no type arguments, but 1 given  [type-arg]
python/ray/serve/tests/typing_files/check_handle_typing.py:24: note: Error code "type-arg" not covered by "type: ignore" comment
python/ray/serve/tests/typing_files/check_handle_typing.py:28: error: Expression is of type "Any", not "str"  [assert-type]
python/ray/serve/tests/typing_files/check_handle_typing.py:37: error: "DeploymentResponse" expects no type arguments, but 1 given  [type-arg]
python/ray/serve/tests/typing_files/check_handle_typing.py:37: note: Error code "type-arg" not covered by "type: ignore" comment
python/ray/serve/tests/typing_files/check_handle_typing.py:41: error: Expression is of type "Any", not "str"  [assert-type]
python/ray/serve/tests/typing_files/check_handle_typing.py:46: error: "DeploymentResponseGenerator" expects no type arguments, but 1 given  [type-arg]
python/ray/serve/tests/typing_files/check_handle_typing.py:46: note: Error code "type-arg" not covered by "type: ignore" comment
python/ray/serve/tests/typing_files/check_handle_typing.py:49: error: Expression is of type "Iterator[Any]", not "Iterator[int]"  [assert-type]
python/ray/serve/tests/typing_files/check_handle_typing.py:53: error: Expression is of type "Any", not "int"  [assert-type]
python/ray/serve/tests/typing_files/check_handle_typing.py:57: error: Expression is of type "Any", not "int"  [assert-type]
python/ray/serve/tests/typing_files/check_handle_typing.py:63: error: "DeploymentResponseGenerator" expects no type arguments, but 1 given  [type-arg]
python/ray/serve/tests/typing_files/check_handle_typing.py:63: note: Error code "type-arg" not covered by "type: ignore" comment
python/ray/serve/tests/typing_files/check_handle_typing.py:66: error: Expression is of type "AsyncIterator[Any]", not "AsyncIterator[int]"  [assert-type]
python/ray/serve/tests/typing_files/check_handle_typing.py:70: error: Expression is of type "Any", not "int"  [assert-type]
python/ray/serve/tests/typing_files/check_handle_typing.py:74: error: Expression is of type "Any", not "int"  [assert-type]
python/ray/serve/tests/typing_files/check_handle_typing.py:88: error: "DeploymentHandle" expects no type arguments, but 1 given  [type-arg]
python/ray/serve/tests/typing_files/check_handle_typing.py:88: note: Error code "type-arg" not covered by "type: ignore" comment
python/ray/serve/tests/typing_files/check_handle_typing.py:97: error: "DeploymentHandle" expects no type arguments, but 1 given  [type-arg]
python/ray/serve/tests/typing_files/check_handle_typing.py:104: error: "DeploymentResponse" expects no type arguments, but 1 given  [type-arg]
python/ray/serve/tests/typing_files/check_handle_typing.py:104: error: "DeploymentResponseGenerator" expects no type arguments, but 1 given  [type-arg]
python/ray/serve/tests/typing_files/check_handle_typing.py:115: error: "DeploymentHandle" expects no type arguments, but 1 given  [type-arg]
python/ray/serve/tests/typing_files/check_handle_typing.py:115: note: Error code "type-arg" not covered by "type: ignore" comment
python/ray/serve/tests/typing_files/check_handle_typing.py:119: error: Expression is of type "Any", not "DeploymentHandle"  [assert-type]
python/ray/serve/tests/typing_files/check_handle_typing.py:119: error: "DeploymentHandle" expects no type arguments, but 1 given  [type-arg]
python/ray/serve/tests/typing_files/check_handle_typing.py:151: error: "DeploymentHandle" expects no type arguments, but 1 given  [type-arg]
python/ray/serve/tests/typing_files/check_handle_typing.py:151: note: Error code "type-arg" not covered by "type: ignore" comment
python/ray/serve/tests/typing_files/check_handle_typing.py:177: error: "DeploymentHandle" expects no type arguments, but 1 given  [type-arg]
python/ray/serve/tests/typing_files/check_handle_typing.py:177: note: Error code "type-arg" not covered by "type: ignore" comment
python/ray/serve/tests/typing_files/check_handle_typing.py:200: error: "DeploymentHandle" expects no type arguments, but 1 given  [type-arg]
python/ray/serve/tests/typing_files/check_handle_typing.py:200: note: Error code "type-arg" not covered by "type: ignore" comment
python/ray/serve/tests/typing_files/check_handle_typing.py:230: error: "DeploymentHandle" expects no type arguments, but 1 given  [type-arg]
python/ray/serve/tests/typing_files/check_handle_typing.py:230: note: Error code "type-arg" not covered by "type: ignore" comment
python/ray/serve/tests/typing_files/check_handle_typing.py:262: error: "DeploymentHandle" expects no type arguments, but 1 given  [type-arg]
python/ray/serve/tests/typing_files/check_handle_typing.py:262: note: Error code "type-arg" not covered by "type: ignore" comment
Found 30 errors in 2 files (checked 2 source files)

Because of changes in this PR

❯ mypy python/ray/serve/tests/typing_files/check_handle_typing.py python/ray/serve/handle.py  --follow-imports=skip --ignore-missing-imports
python/ray/serve/handle.py:261: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
Success: no issues found in 2 source files

Signed-off-by: abrar <abrar@anyscale.com>

gemini-code-assist

Code Review

This pull request significantly improves the type annotations for Ray Serve's handle classes, enabling better IDE support and static analysis. The introduction of generics for DeploymentHandle, DeploymentResponse, and DeploymentResponseGenerator is well-executed, making the return types more precise. The changes to make the code more mypy-compatible, such as adding cast() for Future types and refactoring the remote() method, are thoughtful and effective. The addition of a dedicated type-checking test file (check_handle_typing.py) is an excellent practice that ensures these new annotations are correct and will be maintained. The code quality is high, and the changes are clear and well-documented in the pull request description. I've found one minor issue in the new test file's commented-out code.

gemini-code-assist · 2025-12-10T22:35:49Z

python/ray/serve/tests/typing_files/check_handle_typing.py

+            yield b"chunk1"
+            yield b"chunk2"
+
+    _: DeploymentHandle[MyDeployment] = None  # type: ignore[assignment]


In the commented-out tests for the mypy plugin, the DeploymentHandle variable is named _, but the following commented-out code attempts to use handle. This will cause a NameError when these tests are uncommented. To fix this, the variable should be named handle. This issue is present in all the placeholder tests for the mypy plugin.

Suggested change

_: DeploymentHandle[MyDeployment] = None # type: ignore[assignment]

handle: DeploymentHandle[MyDeployment] = None # type: ignore[assignment]

marwan116

Did a very brief review - it is very nice to see type annotations making their way to Ray Serve deployments. Let's make sure the docs are also updated as part of future work to reflect this capability.

marwan116 · 2025-12-11T16:41:54Z

python/ray/serve/tests/typing_files/check_handle_typing.py

+# out because they will fail without the plugin.
+#
+# To enable after plugin is implemented:
+# 1. Uncomment the tests below


nit: but why not use pytest annotations like xfail instead of commenting code ?

these are not run as part of pytest. these get run from mypy, which is activated from pre-commit-hook in the repo

thanks for the clarification - just found out about https://pypi.org/project/pytest-mypy/ btw - maybe worth considering ?

python/ray/serve/tests/typing_files/check_handle_typing.py

marwan116 · 2025-12-11T17:26:19Z

python/ray/serve/tests/typing_files/check_handle_typing.py

@@ -0,0 +1,277 @@
+"""Type checking tests for DeploymentHandle, DeploymentResponse, and DeploymentResponseGenerator.


would it make sense to have a test example of composition in here as well ?

abrarsheikh · 2025-12-11T17:32:50Z

Let's make sure the docs are also updated as part of future work to reflect this capability.

Good point, we will update the doc after we ship the full implementation with the mypy plugin.

…tResponse, and DeploymentResponseGenerator (ray-project#59363) Fixes ray-project#52654 ### Summary This PR improves the generic type annotations on Ray Serve's handle classes to enable better IDE support and type inference. It also adds a type checking test file to verify the annotations work correctly. ### Changes #### Type Annotation Improvements (`handle.py`) - **Generic return types**: Updated methods to return the generic type parameter `R` instead of `Any`: - `DeploymentResponse.result()` → `R` - `DeploymentResponse.__await__()` → `Generator[Any, None, R]` - `DeploymentResponseGenerator.__iter__()` → `Iterator[R]` - `DeploymentResponseGenerator.__next__()` → `R` - `DeploymentResponseGenerator.__aiter__()` → `AsyncIterator[R]` - `DeploymentResponseGenerator.__anext__()` → `R` - **mypy compatibility fixes**: - Added `cast()` for `Future` types in sync/async fetch methods to help mypy understand the conditional types - Aligned `_DeploymentHandleBase.options()` signature with `DeploymentHandle.options()` to fix override error - Refactored `remote()` to return directly from each branch instead of assigning different class types to a variable #### Type Checking Tests (`check_handle_typing.py`) Added a mypy test file that verifies: - Generic type `R` is preserved through `result()`, `__await__`, iteration methods - Generic type `T` is preserved through `options()` and method access - Placeholder tests for future mypy plugin (commented out) that will verify method return type inference ### How to Test ```bash mypy python/ray/serve/tests/typing_files/check_handle_typing.py python/ray/serve/handle.py \ --follow-imports=skip \ --ignore-missing-imports ``` ### Future Work A mypy plugin can be implemented to infer the return type of `.remote()` based on which deployment method is being called: ```python handle: DeploymentHandle[MyDeployment] response = handle.get_user.remote(123) # Plugin would infer DeploymentResponse[str] user = response.result() # Would be typed as str ``` The test file includes commented-out tests that should pass once the plugin is implemented. ## Tests From master ``` ❯ mypy python/ray/serve/tests/typing_files/check_handle_typing.py python/ray/serve/handle.py --follow-imports=skip --ignore-missing-imports python/ray/serve/handle.py:10: error: Module "ray._raylet" has no attribute "ObjectRefGenerator" [attr-defined] python/ray/serve/handle.py:162: error: Item "None" of "Optional[Any]" has no attribute "_run_router_in_separate_loop" [union-attr] python/ray/serve/handle.py:210: error: Item "None" of "Optional[Any]" has no attribute "assign_request" [union-attr] python/ray/serve/handle.py:229: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked] python/ray/serve/handle.py:292: error: Unexpected keyword argument "timeout" for "result" of "Future" [call-arg] /home/ubuntu/.local/lib/python3.9/site-packages/mypy/typeshed/stdlib/_asyncio.pyi: note: "result" of "Future" defined here python/ray/serve/handle.py:319: error: Incompatible types in "await" (actual type "Union[concurrent.futures._base.Future[Any], _asyncio.Future[Any]]", expected type "Awaitable[Any]") [misc] python/ray/serve/handle.py:803: error: Incompatible types in assignment (expression has type "type[DeploymentResponse]", variable has type "type[DeploymentResponseGenerator]") [assignment] python/ray/serve/tests/typing_files/check_handle_typing.py:24: error: "DeploymentResponse" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:24: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:28: error: Expression is of type "Any", not "str" [assert-type] python/ray/serve/tests/typing_files/check_handle_typing.py:37: error: "DeploymentResponse" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:37: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:41: error: Expression is of type "Any", not "str" [assert-type] python/ray/serve/tests/typing_files/check_handle_typing.py:46: error: "DeploymentResponseGenerator" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:46: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:49: error: Expression is of type "Iterator[Any]", not "Iterator[int]" [assert-type] python/ray/serve/tests/typing_files/check_handle_typing.py:53: error: Expression is of type "Any", not "int" [assert-type] python/ray/serve/tests/typing_files/check_handle_typing.py:57: error: Expression is of type "Any", not "int" [assert-type] python/ray/serve/tests/typing_files/check_handle_typing.py:63: error: "DeploymentResponseGenerator" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:63: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:66: error: Expression is of type "AsyncIterator[Any]", not "AsyncIterator[int]" [assert-type] python/ray/serve/tests/typing_files/check_handle_typing.py:70: error: Expression is of type "Any", not "int" [assert-type] python/ray/serve/tests/typing_files/check_handle_typing.py:74: error: Expression is of type "Any", not "int" [assert-type] python/ray/serve/tests/typing_files/check_handle_typing.py:88: error: "DeploymentHandle" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:88: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:97: error: "DeploymentHandle" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:104: error: "DeploymentResponse" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:104: error: "DeploymentResponseGenerator" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:115: error: "DeploymentHandle" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:115: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:119: error: Expression is of type "Any", not "DeploymentHandle" [assert-type] python/ray/serve/tests/typing_files/check_handle_typing.py:119: error: "DeploymentHandle" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:151: error: "DeploymentHandle" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:151: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:177: error: "DeploymentHandle" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:177: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:200: error: "DeploymentHandle" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:200: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:230: error: "DeploymentHandle" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:230: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:262: error: "DeploymentHandle" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:262: note: Error code "type-arg" not covered by "type: ignore" comment Found 30 errors in 2 files (checked 2 source files) ``` Because of changes in this PR ``` ❯ mypy python/ray/serve/tests/typing_files/check_handle_typing.py python/ray/serve/handle.py --follow-imports=skip --ignore-missing-imports python/ray/serve/handle.py:261: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked] Success: no issues found in 2 source files ``` Signed-off-by: abrar <abrar@anyscale.com> Signed-off-by: kriyanshii <kriyanshishah06@gmail.com>

…tResponse, and DeploymentResponseGenerator (ray-project#59363) Fixes ray-project#52654 ### Summary This PR improves the generic type annotations on Ray Serve's handle classes to enable better IDE support and type inference. It also adds a type checking test file to verify the annotations work correctly. ### Changes #### Type Annotation Improvements (`handle.py`) - **Generic return types**: Updated methods to return the generic type parameter `R` instead of `Any`: - `DeploymentResponse.result()` → `R` - `DeploymentResponse.__await__()` → `Generator[Any, None, R]` - `DeploymentResponseGenerator.__iter__()` → `Iterator[R]` - `DeploymentResponseGenerator.__next__()` → `R` - `DeploymentResponseGenerator.__aiter__()` → `AsyncIterator[R]` - `DeploymentResponseGenerator.__anext__()` → `R` - **mypy compatibility fixes**: - Added `cast()` for `Future` types in sync/async fetch methods to help mypy understand the conditional types - Aligned `_DeploymentHandleBase.options()` signature with `DeploymentHandle.options()` to fix override error - Refactored `remote()` to return directly from each branch instead of assigning different class types to a variable #### Type Checking Tests (`check_handle_typing.py`) Added a mypy test file that verifies: - Generic type `R` is preserved through `result()`, `__await__`, iteration methods - Generic type `T` is preserved through `options()` and method access - Placeholder tests for future mypy plugin (commented out) that will verify method return type inference ### How to Test ```bash mypy python/ray/serve/tests/typing_files/check_handle_typing.py python/ray/serve/handle.py \ --follow-imports=skip \ --ignore-missing-imports ``` ### Future Work A mypy plugin can be implemented to infer the return type of `.remote()` based on which deployment method is being called: ```python handle: DeploymentHandle[MyDeployment] response = handle.get_user.remote(123) # Plugin would infer DeploymentResponse[str] user = response.result() # Would be typed as str ``` The test file includes commented-out tests that should pass once the plugin is implemented. ## Tests From master ``` ❯ mypy python/ray/serve/tests/typing_files/check_handle_typing.py python/ray/serve/handle.py --follow-imports=skip --ignore-missing-imports python/ray/serve/handle.py:10: error: Module "ray._raylet" has no attribute "ObjectRefGenerator" [attr-defined] python/ray/serve/handle.py:162: error: Item "None" of "Optional[Any]" has no attribute "_run_router_in_separate_loop" [union-attr] python/ray/serve/handle.py:210: error: Item "None" of "Optional[Any]" has no attribute "assign_request" [union-attr] python/ray/serve/handle.py:229: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked] python/ray/serve/handle.py:292: error: Unexpected keyword argument "timeout" for "result" of "Future" [call-arg] /home/ubuntu/.local/lib/python3.9/site-packages/mypy/typeshed/stdlib/_asyncio.pyi: note: "result" of "Future" defined here python/ray/serve/handle.py:319: error: Incompatible types in "await" (actual type "Union[concurrent.futures._base.Future[Any], _asyncio.Future[Any]]", expected type "Awaitable[Any]") [misc] python/ray/serve/handle.py:803: error: Incompatible types in assignment (expression has type "type[DeploymentResponse]", variable has type "type[DeploymentResponseGenerator]") [assignment] python/ray/serve/tests/typing_files/check_handle_typing.py:24: error: "DeploymentResponse" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:24: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:28: error: Expression is of type "Any", not "str" [assert-type] python/ray/serve/tests/typing_files/check_handle_typing.py:37: error: "DeploymentResponse" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:37: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:41: error: Expression is of type "Any", not "str" [assert-type] python/ray/serve/tests/typing_files/check_handle_typing.py:46: error: "DeploymentResponseGenerator" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:46: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:49: error: Expression is of type "Iterator[Any]", not "Iterator[int]" [assert-type] python/ray/serve/tests/typing_files/check_handle_typing.py:53: error: Expression is of type "Any", not "int" [assert-type] python/ray/serve/tests/typing_files/check_handle_typing.py:57: error: Expression is of type "Any", not "int" [assert-type] python/ray/serve/tests/typing_files/check_handle_typing.py:63: error: "DeploymentResponseGenerator" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:63: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:66: error: Expression is of type "AsyncIterator[Any]", not "AsyncIterator[int]" [assert-type] python/ray/serve/tests/typing_files/check_handle_typing.py:70: error: Expression is of type "Any", not "int" [assert-type] python/ray/serve/tests/typing_files/check_handle_typing.py:74: error: Expression is of type "Any", not "int" [assert-type] python/ray/serve/tests/typing_files/check_handle_typing.py:88: error: "DeploymentHandle" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:88: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:97: error: "DeploymentHandle" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:104: error: "DeploymentResponse" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:104: error: "DeploymentResponseGenerator" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:115: error: "DeploymentHandle" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:115: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:119: error: Expression is of type "Any", not "DeploymentHandle" [assert-type] python/ray/serve/tests/typing_files/check_handle_typing.py:119: error: "DeploymentHandle" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:151: error: "DeploymentHandle" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:151: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:177: error: "DeploymentHandle" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:177: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:200: error: "DeploymentHandle" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:200: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:230: error: "DeploymentHandle" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:230: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:262: error: "DeploymentHandle" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:262: note: Error code "type-arg" not covered by "type: ignore" comment Found 30 errors in 2 files (checked 2 source files) ``` Because of changes in this PR ``` ❯ mypy python/ray/serve/tests/typing_files/check_handle_typing.py python/ray/serve/handle.py --follow-imports=skip --ignore-missing-imports python/ray/serve/handle.py:261: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked] Success: no issues found in 2 source files ``` Signed-off-by: abrar <abrar@anyscale.com>

…tResponse, and DeploymentResponseGenerator (ray-project#59363) Fixes ray-project#52654 ### Summary This PR improves the generic type annotations on Ray Serve's handle classes to enable better IDE support and type inference. It also adds a type checking test file to verify the annotations work correctly. ### Changes #### Type Annotation Improvements (`handle.py`) - **Generic return types**: Updated methods to return the generic type parameter `R` instead of `Any`: - `DeploymentResponse.result()` → `R` - `DeploymentResponse.__await__()` → `Generator[Any, None, R]` - `DeploymentResponseGenerator.__iter__()` → `Iterator[R]` - `DeploymentResponseGenerator.__next__()` → `R` - `DeploymentResponseGenerator.__aiter__()` → `AsyncIterator[R]` - `DeploymentResponseGenerator.__anext__()` → `R` - **mypy compatibility fixes**: - Added `cast()` for `Future` types in sync/async fetch methods to help mypy understand the conditional types - Aligned `_DeploymentHandleBase.options()` signature with `DeploymentHandle.options()` to fix override error - Refactored `remote()` to return directly from each branch instead of assigning different class types to a variable #### Type Checking Tests (`check_handle_typing.py`) Added a mypy test file that verifies: - Generic type `R` is preserved through `result()`, `__await__`, iteration methods - Generic type `T` is preserved through `options()` and method access - Placeholder tests for future mypy plugin (commented out) that will verify method return type inference ### How to Test ```bash mypy python/ray/serve/tests/typing_files/check_handle_typing.py python/ray/serve/handle.py \ --follow-imports=skip \ --ignore-missing-imports ``` ### Future Work A mypy plugin can be implemented to infer the return type of `.remote()` based on which deployment method is being called: ```python handle: DeploymentHandle[MyDeployment] response = handle.get_user.remote(123) # Plugin would infer DeploymentResponse[str] user = response.result() # Would be typed as str ``` The test file includes commented-out tests that should pass once the plugin is implemented. ## Tests From master ``` ❯ mypy python/ray/serve/tests/typing_files/check_handle_typing.py python/ray/serve/handle.py --follow-imports=skip --ignore-missing-imports python/ray/serve/handle.py:10: error: Module "ray._raylet" has no attribute "ObjectRefGenerator" [attr-defined] python/ray/serve/handle.py:162: error: Item "None" of "Optional[Any]" has no attribute "_run_router_in_separate_loop" [union-attr] python/ray/serve/handle.py:210: error: Item "None" of "Optional[Any]" has no attribute "assign_request" [union-attr] python/ray/serve/handle.py:229: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked] python/ray/serve/handle.py:292: error: Unexpected keyword argument "timeout" for "result" of "Future" [call-arg] /home/ubuntu/.local/lib/python3.9/site-packages/mypy/typeshed/stdlib/_asyncio.pyi: note: "result" of "Future" defined here python/ray/serve/handle.py:319: error: Incompatible types in "await" (actual type "Union[concurrent.futures._base.Future[Any], _asyncio.Future[Any]]", expected type "Awaitable[Any]") [misc] python/ray/serve/handle.py:803: error: Incompatible types in assignment (expression has type "type[DeploymentResponse]", variable has type "type[DeploymentResponseGenerator]") [assignment] python/ray/serve/tests/typing_files/check_handle_typing.py:24: error: "DeploymentResponse" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:24: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:28: error: Expression is of type "Any", not "str" [assert-type] python/ray/serve/tests/typing_files/check_handle_typing.py:37: error: "DeploymentResponse" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:37: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:41: error: Expression is of type "Any", not "str" [assert-type] python/ray/serve/tests/typing_files/check_handle_typing.py:46: error: "DeploymentResponseGenerator" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:46: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:49: error: Expression is of type "Iterator[Any]", not "Iterator[int]" [assert-type] python/ray/serve/tests/typing_files/check_handle_typing.py:53: error: Expression is of type "Any", not "int" [assert-type] python/ray/serve/tests/typing_files/check_handle_typing.py:57: error: Expression is of type "Any", not "int" [assert-type] python/ray/serve/tests/typing_files/check_handle_typing.py:63: error: "DeploymentResponseGenerator" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:63: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:66: error: Expression is of type "AsyncIterator[Any]", not "AsyncIterator[int]" [assert-type] python/ray/serve/tests/typing_files/check_handle_typing.py:70: error: Expression is of type "Any", not "int" [assert-type] python/ray/serve/tests/typing_files/check_handle_typing.py:74: error: Expression is of type "Any", not "int" [assert-type] python/ray/serve/tests/typing_files/check_handle_typing.py:88: error: "DeploymentHandle" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:88: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:97: error: "DeploymentHandle" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:104: error: "DeploymentResponse" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:104: error: "DeploymentResponseGenerator" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:115: error: "DeploymentHandle" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:115: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:119: error: Expression is of type "Any", not "DeploymentHandle" [assert-type] python/ray/serve/tests/typing_files/check_handle_typing.py:119: error: "DeploymentHandle" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:151: error: "DeploymentHandle" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:151: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:177: error: "DeploymentHandle" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:177: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:200: error: "DeploymentHandle" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:200: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:230: error: "DeploymentHandle" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:230: note: Error code "type-arg" not covered by "type: ignore" comment python/ray/serve/tests/typing_files/check_handle_typing.py:262: error: "DeploymentHandle" expects no type arguments, but 1 given [type-arg] python/ray/serve/tests/typing_files/check_handle_typing.py:262: note: Error code "type-arg" not covered by "type: ignore" comment Found 30 errors in 2 files (checked 2 source files) ``` Because of changes in this PR ``` ❯ mypy python/ray/serve/tests/typing_files/check_handle_typing.py python/ray/serve/handle.py --follow-imports=skip --ignore-missing-imports python/ray/serve/handle.py:261: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked] Success: no issues found in 2 source files ``` Signed-off-by: abrar <abrar@anyscale.com> Signed-off-by: peterxcli <peterxcli@gmail.com>

[Serve][1/n] Add generics to deployment handle

3f6b2d5

Signed-off-by: abrar <abrar@anyscale.com>

abrarsheikh requested a review from a team as a code owner December 10, 2025 22:33

abrarsheikh added the go add ONLY when ready to merge, run all tests label Dec 10, 2025

abrarsheikh requested a review from akyang-anyscale December 10, 2025 22:34

gemini-code-assist bot reviewed Dec 10, 2025

View reviewed changes

abrarsheikh requested a review from marwan116 December 10, 2025 22:36

ray-gardener bot added the serve Ray Serve Related Issue label Dec 11, 2025

SolitaryThinker approved these changes Dec 11, 2025

View reviewed changes

marwan116 reviewed Dec 11, 2025

View reviewed changes

akyang-anyscale approved these changes Dec 15, 2025

View reviewed changes

abrarsheikh merged commit d891f23 into master Dec 15, 2025
6 checks passed

abrarsheikh deleted the 52654-abrar-generics branch December 15, 2025 17:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Serve][1/n] Improve type annotations for DeploymentHandle, DeploymentResponse, and DeploymentResponseGenerator#59363

[Serve][1/n] Improve type annotations for DeploymentHandle, DeploymentResponse, and DeploymentResponseGenerator#59363
abrarsheikh merged 1 commit intomasterfrom
52654-abrar-generics

abrarsheikh commented Dec 10, 2025 •

edited

Loading

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

gemini-code-assist bot Dec 10, 2025

Uh oh!

marwan116 left a comment

Uh oh!

marwan116 Dec 11, 2025

Uh oh!

abrarsheikh Dec 11, 2025

Uh oh!

marwan116 Dec 11, 2025

Uh oh!

Uh oh!

marwan116 Dec 11, 2025

Uh oh!

abrarsheikh Dec 11, 2025

Uh oh!

abrarsheikh commented Dec 11, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

	_: DeploymentHandle[MyDeployment] = None # type: ignore[assignment]
	handle: DeploymentHandle[MyDeployment] = None # type: ignore[assignment]

		@@ -0,0 +1,277 @@
		"""Type checking tests for DeploymentHandle, DeploymentResponse, and DeploymentResponseGenerator.

Conversation

abrarsheikh commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Type Annotation Improvements (handle.py)

Type Checking Tests (check_handle_typing.py)

How to Test

Future Work

Tests

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

marwan116 left a comment

Choose a reason for hiding this comment

Uh oh!

marwan116 Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

abrarsheikh Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

marwan116 Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

marwan116 Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

abrarsheikh Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

abrarsheikh commented Dec 11, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

abrarsheikh commented Dec 10, 2025 •

edited

Loading

Type Annotation Improvements (`handle.py`)

Type Checking Tests (`check_handle_typing.py`)