Skip to content

Upgrade nghttp to 1.25.0#1600

Merged
mattklein123 merged 4 commits intoenvoyproxy:masterfrom
moderation:master
Sep 6, 2017
Merged

Upgrade nghttp to 1.25.0#1600
mattklein123 merged 4 commits intoenvoyproxy:masterfrom
moderation:master

Conversation

@moderation
Copy link
Copy Markdown
Contributor

Upgrade to the latest version of nghttp2 - 1.25.0. I've been running this for over a week without issue (low volume light usage). Passes all tests.

@mattklein123 mattklein123 reopened this Sep 6, 2017
@mattklein123 mattklein123 merged commit 10c25d7 into envoyproxy:master Sep 6, 2017
mathetake added a commit that referenced this pull request Mar 3, 2026
**Description**
This consolidates all the copy-pasted processors that existed per
endpoint we support into one generic processor. This was made possible
thanks to the series of refactoring that we landed in the past few weeks
primarily for dynamic module work #90.

Notably, now in order to add an endpoint, majority of the new code will
be in translator (where also have shared generic interface) as well as
the type definition. No longer it requires the huge copy paste of
processors.

**Related Issues/PRs (if applicable)**

Resolves #1083 
Blocker for #1429 #1584 #1592 #1594

---------

Signed-off-by: Takeshi Yoneda <t.y.mathetake@gmail.com>
Co-authored-by: Ignasi Barrera <ignasi@tetrate.io>
mathetake pushed a commit that referenced this pull request Mar 3, 2026
…ranslation (#1812)

**Description**

Discovered a bug when sending `streaming` request to AWS Bedrock with
body mutation set with `serviceTier:default` at route level. The request
translation for AWS Bedrock request is incorrect due to the bug and
resulting in an error from AWS Bedrock

Malformed message structure for AWS Bedrock
```
{
  ...
  "messages" : [ 
    {
      "role" : "user",
      "content" : "Explain what AWS Bedrock is in one sentence."
    } 
  ],
  "serviceTier" : {
    "type" : "default"
  }
  ....
}
```

AWS Bedrock expects message content structure [1]
```
{
 ...
  "messages" : [ {
    "role" : "user",
    "content" : [ {
      "text" : "Explain what AWS Bedrock is in one sentence."
    } ],
  } ],
  "serviceTier" : {
    "type" : "default"
  }
}
```

Error response due to malformed request body for AWS Bedrock
```
'Response code: 400\n'
{'Date': 'Sat, 24 Jan 2026 20:59:22 GMT', 'Content-Type': 'application/json', 'Content-Length': '35', 'x-amzn-RequestId': '1140c7eb-bedd-42f7-aeed-657bd6fdfc46', 'x-amzn-ErrorType': 'SerializationException:http://internal.amazon.com/coral/com.amazon.coral.service/', 'Cache-Control': 'proxy-revalidate', 'Connection': 'keep-alive'}

{"Message":"Unexpected field type"}
```

** Root Cause **
After OpenAI request format -> Bedrock request format translation is
done, current logic informs BodyMutator to use original request instead
of translated request to mutate with forceBodyMutation flag set to true
for initial request, this causes the undesired behavior.

**My Debugging and Analysis**:

in `endpointspec.go`'s ParseBody method return non-nil mutatedBody when
it's streaming request and cost is configured which result in non-nil
`mutatedBody` at [2]

in processor_impl.go's ProcessRequestBody method, `forceBodyMutation` is
set to true for initial request at [3] later cause problem in
```
if onRetry && b.originalBody != nil {
	// On retry, restore the original body first
	requestBody = b.originalBody
}
```
that resets translated body(supposed to Bedrock format) back to original
OpenAI format.

**My proposed solution**:
Remove request body restoration in body mutation.

**Disclaimer**:
Debug analysis and code change in processor_impl.go were done myself and
test cases were added with Claude Code and tweaked manually.

**Related Issues/PRs (if applicable)**
Related PR: #1600

1:
https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ConverseStream.html
2:
https://github.com/envoyproxy/ai-gateway/blob/main/internal/endpointspec/endpointspec.go#L90
3:
https://github.com/envoyproxy/ai-gateway/blob/main/internal/extproc/processor_impl.go#L174-L176

---------

Signed-off-by: Xiaolin Lin <xlin158@bloomberg.net>
Signed-off-by: achoo30 <achoo30@bloomberg.net>
Co-authored-by: Aaron Choo <achoo30@bloomberg.net>
Co-authored-by: Dan Sun <dsun20@bloomberg.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants