[serve] combine proxy routers#48706
Merged
zcin merged 12 commits intoray-project:masterfrom Nov 13, 2024
Merged
Conversation
Signed-off-by: Cindy Zhang <cindyzyx9@gmail.com>
Signed-off-by: Cindy Zhang <cindyzyx9@gmail.com>
Signed-off-by: Cindy Zhang <cindyzyx9@gmail.com>
Signed-off-by: Cindy Zhang <cindyzyx9@gmail.com>
GeneDer
approved these changes
Nov 13, 2024
| self.http_proxy.update_routes(endpoints) | ||
| if self.grpc_proxy is not None: | ||
| self.grpc_proxy.update_routes(endpoints) | ||
| self.proxy_router.update_routes(endpoints) |
JP-sDEV
pushed a commit
to JP-sDEV/ray
that referenced
this pull request
Nov 14, 2024
## Why are these changes needed? - Combine the endpoint router and the longest prefix router. The two differed in that one used `match_route` to get a handle, and one used `get_handle_for_endpoint` to get a handle (for HTTP and gRPC respectively). We can just have a single proxy router class that have both of these methods. - This is nice in that we only have one cached set of handles. - Add a default `get_proxy_handle` function in `default_impl.py`, which can be used to control how a handle is fetched and how it is initialized. - Infer request protocol from request context, which should deterministically indicate whether the request is an http request or a gRPC request. This then removes the need to set request protocol in the handle, which makes sense since request protocol is more metadata than handle configuration. Signed-off-by: Cindy Zhang <cindyzyx9@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why are these changes needed?
match_routeto get a handle, and one usedget_handle_for_endpointto get a handle (for HTTP and gRPC respectively). We can just have a single proxy router class that have both of these methods.get_proxy_handlefunction indefault_impl.py, which can be used to control how a handle is fetched and how it is initialized.Related issue number