fix: The Gateway Inference Extension doesn't receive response bodies#56463
Merged
istio-testing merged 5 commits intoistio:experimental-gwapi-inference-extensionfrom Jun 10, 2025
Conversation
…e bodies Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
|
😊 Welcome @shmuelk! This is either your first contribution to the Istio istio repo, or it's been You can learn more about the Istio working groups, Code of Conduct, and contribution guidelines Thanks for contributing! Courtesy of your friendly welcome wagon. |
Collaborator
|
Hi @shmuelk. Thanks for your PR. I'm waiting for a istio member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
Contributor
|
/cc |
Contributor
|
@shmuelk PTAL at the CI failures... it looks like you need to run |
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Contributor
Author
|
I have tried running |
keithmattix
reviewed
Jun 10, 2025
Contributor
keithmattix
left a comment
There was a problem hiding this comment.
Looks like a nil pointer popping up in tests
| RequestBodyMode: extproc.ProcessingMode_FULL_DUPLEX_STREAMED, | ||
| RequestBodyMode: extproc.ProcessingMode_FULL_DUPLEX_STREAMED, | ||
| ResponseHeaderMode: extproc.ProcessingMode_SEND, | ||
| // GIE colelcts statistics present in the open AI standard response message |
Contributor
There was a problem hiding this comment.
Suggested change
| // GIE colelcts statistics present in the open AI standard response message | |
| // GIE collects statistics present in the open AI standard response message |
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Contributor
Author
In my attempts to run the unit tests locally they fail with issues about what I believe to eb garbage in all sorts of generated files. |
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
keithmattix
approved these changes
Jun 10, 2025
665da00
into
istio:experimental-gwapi-inference-extension
29 checks passed
aslakknutsen
pushed a commit
to aslakknutsen/istio
that referenced
this pull request
Jun 12, 2025
…stio#56463) * Set the Gateway Inference Extension EndPointPicker to receive response bodies Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Updated the test to look for extra configuration parameters Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Formatting change after running make gen Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Correct typo in comment Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Updates from running make gen Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> --------- Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
LiorLieberman
pushed a commit
to LiorLieberman/istio
that referenced
this pull request
Jun 13, 2025
…stio#56463) * Set the Gateway Inference Extension EndPointPicker to receive response bodies Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Updated the test to look for extra configuration parameters Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Formatting change after running make gen Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Correct typo in comment Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Updates from running make gen Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> --------- Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
dgn
pushed a commit
to dgn/istio
that referenced
this pull request
Jun 27, 2025
…stio#56463) * Set the Gateway Inference Extension EndPointPicker to receive response bodies Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Updated the test to look for extra configuration parameters Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Formatting change after running make gen Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Correct typo in comment Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Updates from running make gen Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> --------- Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
aslakknutsen
pushed a commit
to aslakknutsen/istio
that referenced
this pull request
Jun 30, 2025
…stio#56463) * Set the Gateway Inference Extension EndPointPicker to receive response bodies Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Updated the test to look for extra configuration parameters Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Formatting change after running make gen Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Correct typo in comment Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Updates from running make gen Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> --------- Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
aslakknutsen
pushed a commit
to aslakknutsen/istio
that referenced
this pull request
Jul 1, 2025
…stio#56463) * Set the Gateway Inference Extension EndPointPicker to receive response bodies Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Updated the test to look for extra configuration parameters Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Formatting change after running make gen Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Correct typo in comment Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Updates from running make gen Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> --------- Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
istio-testing
added a commit
that referenced
this pull request
Jul 8, 2025
* gateway-api-inference-extenstion support (#55436) * Automator: update proxy@master in istio/istio@master (#55590) * add inferencepool reconcile draft * add global extProc draft * add support for infernecePool internal sematics * attempt to add extProcPerRoute * address comments * address feedback * add push_context changes * add inferencepool status handler * more nits * add tests and cleanup * add deploymentcontroller tests * more tests * furnish tests * gofmt * add temporaray exclude * more formatting * more stuff --------- Co-authored-by: Istio Automation <istio-testing-bot@google.com> * Clusterrole and auto gen (#55644) * Automator: update proxy@master in istio/istio@master (#55590) * add auto crds gen and clusterrole --------- Co-authored-by: Istio Automation <istio-testing-bot@google.com> * fix multiple inferencepools per route (#55683) * ensure inference route name list has all the parts (#55705) * change to extproc requestBody processing mode to FULL_DUPLEX_STREAMED for Gateway Inference Extension (#56089) * fix: change to FULL_DUPLEX_STREAMED processing mode for GIE * Automator: update common-files@master in istio/istio@master (#55878) * TMP: Regen due to odd something version update something somewhere for some reason * a * Automator: update proxy@master in istio/istio@master (#56094) * Create ambient multinetwork flag (#55991) * Create new multinetwork flag Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Add new topology Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Pass Values.global.network to ztunnel Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Skip workload to workload test if workloads are in different clusters Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Fix several tests that aren't ready for multicluster Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Fix some more tests Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Fix all non-CNI/taint ambient tests Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Fix lint Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> --------- Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> --------- Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> Co-authored-by: Aslak Knutsen <aslak@4fs.no> Co-authored-by: Istio Automation <istio-testing-bot@google.com> Co-authored-by: Keith Mattix II <keithmattix@microsoft.com> * fix: The Gateway Inference Extension doesn't receive response bodies (#56463) * Set the Gateway Inference Extension EndPointPicker to receive response bodies Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Updated the test to look for extra configuration parameters Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Formatting change after running make gen Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Correct typo in comment Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Updates from running make gen Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> --------- Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * fix: update generated files after rebase The rebase on master requires a `make gen`. related to #56612 * gw-inference: support new envoy override_host lb policy (#56623) * gw-inference: support new envoy override_host lb policy * mainly formatting * cleanup and tests for inference-lb-policy (#56655) * improve inference config propagation (#56660) * add extra field for Config for inferencepool config propagation * fix tests * address feedback * dont mutate external map inside the itr * Refactor inference extension support to krt (#56656) * Refactor inference extension support to krt We do this mainly by integrating it with the rest of the gateway controller. The conversion logic now has shadow inference services as an output we can query from pushcontext or whatever else Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Create shadow service outside of the collection Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Address PR comments Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Actually run the queue Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Fixups Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Fallback to deep equal Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> --------- Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * fix(gw-inference): Add extra to the VirtualService Config clone/copy (#56775) * Add inferencepools by gateways index (#56777) Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * replace label-based gateway opt in (#56778) * replace label-based gateway opt in * replace label names * add-tests * feat(gw-inference): Add support for InferencePool reference grants (#56832) * feat(gw-inference): support InferencePool as To target for ReferenceGrants Allow cross namespace references for InferencePools. This allow HTTPRoutes to target InferencePools in another namespace. ReferenceGrants.BackendAllowed now takes a GVK to the to Object as an argument along side namespace and name. If GVK is unknown it will default to Service. * test: add resource grant for inferencepool tests * Inference pool Status handling and cross namespace references (#56792) * feat(gw-inference): Improve InferencePool.Status handling Includes tests for InferencePool.Status handling and cross namespace lookups. Test for the following basic scenarios: * Cross namespace references * Removing old state * Keeping old state from other controllers * ResolvedRef conditions * Accepted conditions A HTTPRoute can target BackendRefs in namespaces besides it self, so when we try to do a reverse lookup from InferencePool to find parent Gateways via HTTPRoute we need to list all HTTPRoutes in all Namespaces. Added InferencePool to HTTPRoute.BackendRef index for easy access. lookups. relates to #56621 * fix: safe guard possible nil access * test: add copyright and lint fixes * fix: remove default status message when we take control * fix: remove unused status types hanging around from earlier runs. We want to keep the ones we know to update state or not * fix(test): lint line to long * add failuremode support on inferencepool (#56819) * add failuremode support on inferencepool * update proxy sha * fix conversion_test.go * fix crl test * fix(gw-inference): Refactored InferencePool Collection (#56831) * fix: refactored InferenceController Refactored into 3 main pieces; 0. fetch routes via index 1. find our gateway parents 2. create shadow info 3. create status Update to latest GIE to get API changes Added support for checking the HTTPRoute Accepted status as part of InferencePool Accepted status. * fix: reuse isInferencePoolBackendRef in index * fix: use ptr.OrEmpty where applicable * run make gen * fix(gw-inference): Shadow service should only exist if it has to (#56839) The InferencePool should only create a shadow service if a HTTPRoute that has a Gateway that we control is connected. Else do nothing. Clean up Service and Status if the HTTPRoute disconnects. * Add releasenote and disable gw-inference by default (#56848) * add releasenotes and disable-by-default * add test featureflag * increase binary sizes due to krt * feedback and cleanup (#56866) * (gaie): Fixups from master PR (#56899) * Fixups Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Tackle other TODO Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> --------- Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> --------- Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> Co-authored-by: Istio Automation <istio-testing-bot@google.com> Co-authored-by: Aslak Knutsen <aslak@4fs.no> Co-authored-by: Keith Mattix II <keithmattix@microsoft.com> Co-authored-by: Shmuel Kallner <kallner@il.ibm.com> Co-authored-by: Aslak Knutsen <aslak.tux@gmail.com>
aslakknutsen
added a commit
to aslakknutsen/istio
that referenced
this pull request
Jul 22, 2025
…6845) * gateway-api-inference-extenstion support (istio#55436) * Automator: update proxy@master in istio/istio@master (istio#55590) * add inferencepool reconcile draft * add global extProc draft * add support for infernecePool internal sematics * attempt to add extProcPerRoute * address comments * address feedback * add push_context changes * add inferencepool status handler * more nits * add tests and cleanup * add deploymentcontroller tests * more tests * furnish tests * gofmt * add temporaray exclude * more formatting * more stuff --------- Co-authored-by: Istio Automation <istio-testing-bot@google.com> * Clusterrole and auto gen (istio#55644) * Automator: update proxy@master in istio/istio@master (istio#55590) * add auto crds gen and clusterrole --------- Co-authored-by: Istio Automation <istio-testing-bot@google.com> * fix multiple inferencepools per route (istio#55683) * ensure inference route name list has all the parts (istio#55705) * change to extproc requestBody processing mode to FULL_DUPLEX_STREAMED for Gateway Inference Extension (istio#56089) * fix: change to FULL_DUPLEX_STREAMED processing mode for GIE * Automator: update common-files@master in istio/istio@master (istio#55878) * TMP: Regen due to odd something version update something somewhere for some reason * a * Automator: update proxy@master in istio/istio@master (istio#56094) * Create ambient multinetwork flag (istio#55991) * Create new multinetwork flag Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Add new topology Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Pass Values.global.network to ztunnel Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Skip workload to workload test if workloads are in different clusters Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Fix several tests that aren't ready for multicluster Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Fix some more tests Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Fix all non-CNI/taint ambient tests Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Fix lint Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> --------- Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> --------- Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> Co-authored-by: Aslak Knutsen <aslak@4fs.no> Co-authored-by: Istio Automation <istio-testing-bot@google.com> Co-authored-by: Keith Mattix II <keithmattix@microsoft.com> * fix: The Gateway Inference Extension doesn't receive response bodies (istio#56463) * Set the Gateway Inference Extension EndPointPicker to receive response bodies Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Updated the test to look for extra configuration parameters Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Formatting change after running make gen Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Correct typo in comment Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Updates from running make gen Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> --------- Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * fix: update generated files after rebase The rebase on master requires a `make gen`. related to istio#56612 * gw-inference: support new envoy override_host lb policy (istio#56623) * gw-inference: support new envoy override_host lb policy * mainly formatting * cleanup and tests for inference-lb-policy (istio#56655) * improve inference config propagation (istio#56660) * add extra field for Config for inferencepool config propagation * fix tests * address feedback * dont mutate external map inside the itr * Refactor inference extension support to krt (istio#56656) * Refactor inference extension support to krt We do this mainly by integrating it with the rest of the gateway controller. The conversion logic now has shadow inference services as an output we can query from pushcontext or whatever else Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Create shadow service outside of the collection Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Address PR comments Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Actually run the queue Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Fixups Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Fallback to deep equal Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> --------- Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * fix(gw-inference): Add extra to the VirtualService Config clone/copy (istio#56775) * Add inferencepools by gateways index (istio#56777) Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * replace label-based gateway opt in (istio#56778) * replace label-based gateway opt in * replace label names * add-tests * feat(gw-inference): Add support for InferencePool reference grants (istio#56832) * feat(gw-inference): support InferencePool as To target for ReferenceGrants Allow cross namespace references for InferencePools. This allow HTTPRoutes to target InferencePools in another namespace. ReferenceGrants.BackendAllowed now takes a GVK to the to Object as an argument along side namespace and name. If GVK is unknown it will default to Service. * test: add resource grant for inferencepool tests * Inference pool Status handling and cross namespace references (istio#56792) * feat(gw-inference): Improve InferencePool.Status handling Includes tests for InferencePool.Status handling and cross namespace lookups. Test for the following basic scenarios: * Cross namespace references * Removing old state * Keeping old state from other controllers * ResolvedRef conditions * Accepted conditions A HTTPRoute can target BackendRefs in namespaces besides it self, so when we try to do a reverse lookup from InferencePool to find parent Gateways via HTTPRoute we need to list all HTTPRoutes in all Namespaces. Added InferencePool to HTTPRoute.BackendRef index for easy access. lookups. relates to istio#56621 * fix: safe guard possible nil access * test: add copyright and lint fixes * fix: remove default status message when we take control * fix: remove unused status types hanging around from earlier runs. We want to keep the ones we know to update state or not * fix(test): lint line to long * add failuremode support on inferencepool (istio#56819) * add failuremode support on inferencepool * update proxy sha * fix conversion_test.go * fix crl test * fix(gw-inference): Refactored InferencePool Collection (istio#56831) * fix: refactored InferenceController Refactored into 3 main pieces; 0. fetch routes via index 1. find our gateway parents 2. create shadow info 3. create status Update to latest GIE to get API changes Added support for checking the HTTPRoute Accepted status as part of InferencePool Accepted status. * fix: reuse isInferencePoolBackendRef in index * fix: use ptr.OrEmpty where applicable * run make gen * fix(gw-inference): Shadow service should only exist if it has to (istio#56839) The InferencePool should only create a shadow service if a HTTPRoute that has a Gateway that we control is connected. Else do nothing. Clean up Service and Status if the HTTPRoute disconnects. * Add releasenote and disable gw-inference by default (istio#56848) * add releasenotes and disable-by-default * add test featureflag * increase binary sizes due to krt * feedback and cleanup (istio#56866) * (gaie): Fixups from master PR (istio#56899) * Fixups Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Tackle other TODO Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> --------- Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> --------- Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> Co-authored-by: Istio Automation <istio-testing-bot@google.com> Co-authored-by: Aslak Knutsen <aslak@4fs.no> Co-authored-by: Keith Mattix II <keithmattix@microsoft.com> Co-authored-by: Shmuel Kallner <kallner@il.ibm.com> Co-authored-by: Aslak Knutsen <aslak.tux@gmail.com>
istio-testing
added a commit
that referenced
this pull request
Jul 23, 2025
…#57097) * gateway-api-inference-extenstion support (#55436) * Automator: update proxy@master in istio/istio@master (#55590) * add inferencepool reconcile draft * add global extProc draft * add support for infernecePool internal sematics * attempt to add extProcPerRoute * address comments * address feedback * add push_context changes * add inferencepool status handler * more nits * add tests and cleanup * add deploymentcontroller tests * more tests * furnish tests * gofmt * add temporaray exclude * more formatting * more stuff --------- * Clusterrole and auto gen (#55644) * Automator: update proxy@master in istio/istio@master (#55590) * add auto crds gen and clusterrole --------- * fix multiple inferencepools per route (#55683) * ensure inference route name list has all the parts (#55705) * change to extproc requestBody processing mode to FULL_DUPLEX_STREAMED for Gateway Inference Extension (#56089) * fix: change to FULL_DUPLEX_STREAMED processing mode for GIE * Automator: update common-files@master in istio/istio@master (#55878) * TMP: Regen due to odd something version update something somewhere for some reason * a * Automator: update proxy@master in istio/istio@master (#56094) * Create ambient multinetwork flag (#55991) * Create new multinetwork flag * Add new topology * Pass Values.global.network to ztunnel * Skip workload to workload test if workloads are in different clusters * Fix several tests that aren't ready for multicluster * Fix some more tests * Fix all non-CNI/taint ambient tests * Fix lint --------- --------- * fix: The Gateway Inference Extension doesn't receive response bodies (#56463) * Set the Gateway Inference Extension EndPointPicker to receive response bodies * Updated the test to look for extra configuration parameters * Formatting change after running make gen * Correct typo in comment * Updates from running make gen --------- * fix: update generated files after rebase The rebase on master requires a `make gen`. related to #56612 * gw-inference: support new envoy override_host lb policy (#56623) * gw-inference: support new envoy override_host lb policy * mainly formatting * cleanup and tests for inference-lb-policy (#56655) * improve inference config propagation (#56660) * add extra field for Config for inferencepool config propagation * fix tests * address feedback * dont mutate external map inside the itr * Refactor inference extension support to krt (#56656) * Refactor inference extension support to krt We do this mainly by integrating it with the rest of the gateway controller. The conversion logic now has shadow inference services as an output we can query from pushcontext or whatever else * Create shadow service outside of the collection * Address PR comments * Actually run the queue * Fixups * Fallback to deep equal --------- * fix(gw-inference): Add extra to the VirtualService Config clone/copy (#56775) * Add inferencepools by gateways index (#56777) * replace label-based gateway opt in (#56778) * replace label-based gateway opt in * replace label names * add-tests * feat(gw-inference): Add support for InferencePool reference grants (#56832) * feat(gw-inference): support InferencePool as To target for ReferenceGrants Allow cross namespace references for InferencePools. This allow HTTPRoutes to target InferencePools in another namespace. ReferenceGrants.BackendAllowed now takes a GVK to the to Object as an argument along side namespace and name. If GVK is unknown it will default to Service. * test: add resource grant for inferencepool tests * Inference pool Status handling and cross namespace references (#56792) * feat(gw-inference): Improve InferencePool.Status handling Includes tests for InferencePool.Status handling and cross namespace lookups. Test for the following basic scenarios: * Cross namespace references * Removing old state * Keeping old state from other controllers * ResolvedRef conditions * Accepted conditions A HTTPRoute can target BackendRefs in namespaces besides it self, so when we try to do a reverse lookup from InferencePool to find parent Gateways via HTTPRoute we need to list all HTTPRoutes in all Namespaces. Added InferencePool to HTTPRoute.BackendRef index for easy access. lookups. relates to #56621 * fix: safe guard possible nil access * test: add copyright and lint fixes * fix: remove default status message when we take control * fix: remove unused status types hanging around from earlier runs. We want to keep the ones we know to update state or not * fix(test): lint line to long * add failuremode support on inferencepool (#56819) * add failuremode support on inferencepool * update proxy sha * fix conversion_test.go * fix crl test * fix(gw-inference): Refactored InferencePool Collection (#56831) * fix: refactored InferenceController Refactored into 3 main pieces; 0. fetch routes via index 1. find our gateway parents 2. create shadow info 3. create status Update to latest GIE to get API changes Added support for checking the HTTPRoute Accepted status as part of InferencePool Accepted status. * fix: reuse isInferencePoolBackendRef in index * fix: use ptr.OrEmpty where applicable * run make gen * fix(gw-inference): Shadow service should only exist if it has to (#56839) The InferencePool should only create a shadow service if a HTTPRoute that has a Gateway that we control is connected. Else do nothing. Clean up Service and Status if the HTTPRoute disconnects. * Add releasenote and disable gw-inference by default (#56848) * add releasenotes and disable-by-default * add test featureflag * increase binary sizes due to krt * feedback and cleanup (#56866) * (gaie): Fixups from master PR (#56899) * Fixups * Tackle other TODO --------- --------- Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> Co-authored-by: Lior Lieberman <liorlieberman@google.com> Co-authored-by: Istio Automation <istio-testing-bot@google.com> Co-authored-by: Keith Mattix II <keithmattix@microsoft.com> Co-authored-by: Shmuel Kallner <kallner@il.ibm.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please provide a description of this PR:
The Gateway Inference Extension's EndPointPicker which runs as an External Process extension, does not receive the BodyResponse gRPC messages, due to they're not being configured to be sent. There is code that creates the configuration automatically, when an HTTPRoute CR is created that mentions an InferencePool.
NOTE: This PR adds the extra configuration and extends an existing test in this area.
This PR is against an experimental branch.