Define findListenerByPort to fix use_original_dst flag#493
Merged
mattklein123 merged 2 commits intoenvoyproxy:masterfrom Feb 17, 2017
Merged
Define findListenerByPort to fix use_original_dst flag#493mattklein123 merged 2 commits intoenvoyproxy:masterfrom
mattklein123 merged 2 commits intoenvoyproxy:masterfrom
Conversation
Member
mattklein123
left a comment
There was a problem hiding this comment.
small nits otherwise looks good, thanks.
| Network::Listener* ConnectionHandlerImpl::findListener(const std::string& socket_name) { | ||
| auto l = listeners_.find(socket_name); | ||
| Network::Listener* ConnectionHandlerImpl::findListenerByPort(uint32_t port) { | ||
| auto l = std::find_if( |
Member
There was a problem hiding this comment.
nit: given that the number of listeners is going to be tiny (or potentially even 1), I think using a list here is good. Can you just drop a small comment in that we are doing an O(N) scan here that won't work well for a large number of listeners and we may need to use a map in the future.
| ActiveListenerPtr l( | ||
| new ActiveListener(*this, socket, factory, bind_to_port, use_proxy_proto, use_orig_dst)); | ||
| listeners_.insert(std::make_pair(socket.localAddress()->asString(), std::move(l))); | ||
| listeners_.push_back(std::make_pair(socket.localAddress(), std::move(l))); |
Member
There was a problem hiding this comment.
nit: if you do emplace_back you can avoid std::make_pair, same below.
mattklein123
approved these changes
Feb 17, 2017
lizan
pushed a commit
to lizan/envoy
that referenced
this pull request
Jun 4, 2020
Signed-off-by: John Plevyak <jplevyak@gmail.com>
mathetake
added a commit
that referenced
this pull request
Mar 3, 2026
**Commit Message** This commit scaffolds the foundation for the Inference Extension API [1]. The design documentation was merged in #492. The controller needs to be started with `--enableInferenceExtension=true` to not break the existing controller deployment where the Inference Extension CRDs are not installed. This commit doesn't implement the actual "metrics-aware" load balancing and instead it just does the random routing out of given (resolved) endpoints. The follow up implementations will add more advanced algorithm while expanding the metrics interface that currently only provides the setter APIs. The summary of the implementation is: * Added `kind` field to AIGatewayRouteRuleBackendRef so that it can reference InferencePool. * InferencePool.Spec.Selector is allowed to specify multiple AIServiceBackend. * When building up all the extproc config via filterapi.Config, the controller reads the referenced InferencePool and its binding InferenceModels, and group them together into a single filterapi.DynamicLoadBalancing configuration. * When the extproc loads the configuration containing DynamicLoadBalancing, it will resolve all the IP addresses for hostnames belonging to the DyanmicLoadbalancing. The presence of DynamicLoadBalancing in th config forces the config watcher reload and refresh the config regardless of the updates. That way, the list of ip addresses will always be updated (eventual consistency anyways) in a non-hot path. * On the request path, the ChatCompletionProcessor will check the existence of the DynamicLoadBalancing config for the backend selected by the router. If so, it further tries to resolve the ip:port level endpoint selection. * The selected ip:port will be set to the special header that will be routed to ORIGINAL_DST. * ORIGINAL_DST cluster will be added by the EG extension sever implementation. Also, the extension server modifies some routes to properly route to that cluster. 1: https://github.com/kubernetes-sigs/gateway-api-inference-extension **Related Issues/PRs (if applicable)** Built on #492 Contributes to #423 --------- Signed-off-by: Takeshi Yoneda <t.y.mathetake@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The listener socket address and getOriginalDst address do not match (0.0.0.0 vs link-local IP), causing an issue in finding a matching listener. The fix is to search by the port value alone for IP addresses. Since Envoy binds on all localhost addresses now, the function is well-defined for IP addresses.