Upgrade nghttp to 1.25.0#1600
Merged
mattklein123 merged 4 commits intoenvoyproxy:masterfrom Sep 6, 2017
moderation:master
Merged
Upgrade nghttp to 1.25.0#1600mattklein123 merged 4 commits intoenvoyproxy:masterfrom moderation:master
mattklein123 merged 4 commits intoenvoyproxy:masterfrom
moderation:master
Conversation
mattklein123
approved these changes
Sep 6, 2017
mathetake
added a commit
that referenced
this pull request
Mar 3, 2026
**Description** This consolidates all the copy-pasted processors that existed per endpoint we support into one generic processor. This was made possible thanks to the series of refactoring that we landed in the past few weeks primarily for dynamic module work #90. Notably, now in order to add an endpoint, majority of the new code will be in translator (where also have shared generic interface) as well as the type definition. No longer it requires the huge copy paste of processors. **Related Issues/PRs (if applicable)** Resolves #1083 Blocker for #1429 #1584 #1592 #1594 --------- Signed-off-by: Takeshi Yoneda <t.y.mathetake@gmail.com> Co-authored-by: Ignasi Barrera <ignasi@tetrate.io>
mathetake
pushed a commit
that referenced
this pull request
Mar 3, 2026
…ranslation (#1812) **Description** Discovered a bug when sending `streaming` request to AWS Bedrock with body mutation set with `serviceTier:default` at route level. The request translation for AWS Bedrock request is incorrect due to the bug and resulting in an error from AWS Bedrock Malformed message structure for AWS Bedrock ``` { ... "messages" : [ { "role" : "user", "content" : "Explain what AWS Bedrock is in one sentence." } ], "serviceTier" : { "type" : "default" } .... } ``` AWS Bedrock expects message content structure [1] ``` { ... "messages" : [ { "role" : "user", "content" : [ { "text" : "Explain what AWS Bedrock is in one sentence." } ], } ], "serviceTier" : { "type" : "default" } } ``` Error response due to malformed request body for AWS Bedrock ``` 'Response code: 400\n' {'Date': 'Sat, 24 Jan 2026 20:59:22 GMT', 'Content-Type': 'application/json', 'Content-Length': '35', 'x-amzn-RequestId': '1140c7eb-bedd-42f7-aeed-657bd6fdfc46', 'x-amzn-ErrorType': 'SerializationException:http://internal.amazon.com/coral/com.amazon.coral.service/', 'Cache-Control': 'proxy-revalidate', 'Connection': 'keep-alive'} {"Message":"Unexpected field type"} ``` ** Root Cause ** After OpenAI request format -> Bedrock request format translation is done, current logic informs BodyMutator to use original request instead of translated request to mutate with forceBodyMutation flag set to true for initial request, this causes the undesired behavior. **My Debugging and Analysis**: in `endpointspec.go`'s ParseBody method return non-nil mutatedBody when it's streaming request and cost is configured which result in non-nil `mutatedBody` at [2] in processor_impl.go's ProcessRequestBody method, `forceBodyMutation` is set to true for initial request at [3] later cause problem in ``` if onRetry && b.originalBody != nil { // On retry, restore the original body first requestBody = b.originalBody } ``` that resets translated body(supposed to Bedrock format) back to original OpenAI format. **My proposed solution**: Remove request body restoration in body mutation. **Disclaimer**: Debug analysis and code change in processor_impl.go were done myself and test cases were added with Claude Code and tweaked manually. **Related Issues/PRs (if applicable)** Related PR: #1600 1: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ConverseStream.html 2: https://github.com/envoyproxy/ai-gateway/blob/main/internal/endpointspec/endpointspec.go#L90 3: https://github.com/envoyproxy/ai-gateway/blob/main/internal/extproc/processor_impl.go#L174-L176 --------- Signed-off-by: Xiaolin Lin <xlin158@bloomberg.net> Signed-off-by: achoo30 <achoo30@bloomberg.net> Co-authored-by: Aaron Choo <achoo30@bloomberg.net> Co-authored-by: Dan Sun <dsun20@bloomberg.net>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Upgrade to the latest version of nghttp2 - 1.25.0. I've been running this for over a week without issue (low volume light usage). Passes all tests.