Add safety check for bedrock ConverseStream responses by liustve · Pull Request #3990 · open-telemetry/opentelemetry-python-contrib

liustve · 2025-12-01T22:57:48Z

Description

Add defensive check for Bedrock responses missing expected structure. When no messageStart event with assistant role is received, the response may lack the standard output.message format, causing the occasional KeyError exceptions.

Fixes #3958

Type of change

Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

I was unable to reproduce the exact scenario that would trigger this bug , this safety check would fix this without affecting normal instrumentation behavior.

Does This PR Require a Core Repo Change?

[x ] No.

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

Followed the style guidelines of this project
Changelogs have been updated
Unit tests have been added
Documentation has been updated

...trumentation-botocore/src/opentelemetry/instrumentation/botocore/extensions/bedrock_utils.py

…ntelemetry/instrumentation/botocore/extensions/bedrock_utils.py

CHANGELOG.md

xrmx · 2025-12-02T09:26:28Z

@liustve Please fix the CI failures and next time please run the tests locally :)

…liustve/opentelemetry-python-contrib into fix-bedrock-converse-stream-bug

xrmx · 2025-12-02T17:05:47Z

@liustve I don't think you pushed a fix for the test:

_____________ test_converse_stream_with_missing_output_in_response _____________

    def test_converse_stream_with_missing_output_in_response():
        """Test that converse stream handles malformed response missing output key."""
    
        def stream_done_callback(response, ended):
            pass
    
>       wrapper = ConverseStreamWrapper(
            stream=mock.MagicMock(),
            stream_done_callback=stream_done_callback,
            model_id="amazon.nova-micro-v1:0",
        )
E       TypeError: ConverseStreamWrapper.__init__() got an unexpected keyword argument 'model_id'

instrumentation/opentelemetry-instrumentation-botocore/tests/test_botocore_bedrock.py:3061: TypeError

liustve · 2025-12-02T17:20:07Z

@liustve I don't think you pushed a fix for the test:

_____________ test_converse_stream_with_missing_output_in_response _____________

    def test_converse_stream_with_missing_output_in_response():
        """Test that converse stream handles malformed response missing output key."""
    
        def stream_done_callback(response, ended):
            pass
    
>       wrapper = ConverseStreamWrapper(
            stream=mock.MagicMock(),
            stream_done_callback=stream_done_callback,
            model_id="amazon.nova-micro-v1:0",
        )
E       TypeError: ConverseStreamWrapper.__init__() got an unexpected keyword argument 'model_id'

instrumentation/opentelemetry-instrumentation-botocore/tests/test_botocore_bedrock.py:3061: TypeError

Apologies, had to fix a few things in my local dev environment, for some reason the tests were using another version of OTel botocore.

Also revised the test to better capture the intended behavior.

instrumentation/opentelemetry-instrumentation-botocore/tests/test_botocore_bedrock.py

…est_botocore_bedrock.py

…liustve/opentelemetry-python-contrib into fix-bedrock-converse-stream-bug

liustve · 2025-12-02T22:43:58Z

@xrmx added another defensive check for orig_message["content"] as well. Just in case.

*Description of changes:* Ports the changes from open-telemetry/opentelemetry-python-contrib#3990 Add defensive check for Bedrock responses missing expected structure. When no messageStart event with assistant role is received, the response may lack the standard output.message format, causing the occasional KeyError exceptions. For testing, I was unable to reproduce the exact scenario that would trigger this bug, however, this safety check would fix this issue without affecting normal instrumentation behavior. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…servability#547) *Description of changes:* Ports the changes from open-telemetry/opentelemetry-python-contrib#3990 Add defensive check for Bedrock responses missing expected structure. When no messageStart event with assistant role is received, the response may lack the standard output.message format, causing the occasional KeyError exceptions. For testing, I was unable to reproduce the exact scenario that would trigger this bug, however, this safety check would fix this issue without affecting normal instrumentation behavior. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…#3990) * add defensive check for from_converse response * add defensive check for from_converse response * Update instrumentation/opentelemetry-instrumentation-botocore/src/opentelemetry/instrumentation/botocore/extensions/bedrock_utils.py * Update CHANGELOG.md * add comment for malformed repsonses in from_converse * merge suggested changes * fix botocore converse stream test * lint fix * Update instrumentation/opentelemetry-instrumentation-botocore/tests/test_botocore_bedrock.py * add defensive check just in case content doesn't exist in orig_message * lint fix * new line --------- Co-authored-by: Riccardo Magliocchetti <riccardo.magliocchetti@gmail.com>

Automated update of OpenTelemetry dependencies. **Build Status:** ❌ [failure](https://github.com/aws-observability/aws-otel-python-instrumentation/actions/runs/21465140126) **Updated versions:** - [OpenTelemetry Python](https://github.com/open-telemetry/opentelemetry-python/releases/tag/v1.39.1): 1.39.1 - [OpenTelemetry Contrib](https://github.com/open-telemetry/opentelemetry-python-contrib/releases/tag/v0.60b1): 0.60b1 - [opentelemetry-sdk-extension-aws](https://pypi.org/project/opentelemetry-sdk-extension-aws/2.1.0/): 2.1.0 - [opentelemetry-propagator-aws-xray](https://pypi.org/project/opentelemetry-propagator-aws-xray/1.0.2/): 1.0.2 **Upstream releases with breaking changes:** Note: the mechanism to detect upstream breaking changes is not perfect. Be sure to check all new releases and understand if any additional changes need to be addressed. **opentelemetry-python:** - [Version 1.35.0/0.56b0](https://github.com/open-telemetry/opentelemetry-python/releases/tag/v1.35.0) **opentelemetry-python-contrib:** - [Version 1.34.0/0.55b0](https://github.com/open-telemetry/opentelemetry-python-contrib/releases/tag/v0.55b0) *Description of changes:* - Un-reverts changes done in this PR: #531 - Removed patches for Bedrock following the changes applied in these PRs: open-telemetry/opentelemetry-python-contrib#3544, open-telemetry/opentelemetry-python-contrib#3548, open-telemetry/opentelemetry-python-contrib#3875, open-telemetry/opentelemetry-python-contrib#3990 - Removes patches for Secrets Manager, SNS, and Step Functions following the changes applied in these PRs: open-telemetry/opentelemetry-python-contrib#3734, open-telemetry/opentelemetry-python-contrib#3737, open-telemetry/opentelemetry-python-contrib#3765, - Removes patches for Starlette following the changes applied in this PR: open-telemetry/opentelemetry-python-contrib#3456 - Changes imports and implementation of `OTLPAwsLogExporter`, `AwsCloudWatchOtlpBatchLogRecordProcessor`, and `CompactConsoleLogExporter` following these PRs: open-telemetry/opentelemetry-python#4580, open-telemetry/opentelemetry-python#4535, open-telemetry/opentelemetry-python#4562, open-telemetry/opentelemetry-python#4647, open-telemetry/opentelemetry-python#4676 - Removes a few AWS semantic conventions from `_aws_attribute_keys ` and replaces them with equivalent ones from upstream following the changes in this PR: open-telemetry/opentelemetry-python#4791 - Fix Lambda instrumentation test to set `AWS_LAMBDA_FUNCTION_NAME` env var following changes in: open-telemetry/opentelemetry-python-contrib#3183 - Adds a few more contract tests to verify upstream's botocore instrumentation library By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. --------- Co-authored-by: github-actions <github-actions@github.com> Co-authored-by: Thomas Pierce <thp@amazon.com> Co-authored-by: Steve Liu <liustve@amazon.com>

…#3990) * add defensive check for from_converse response * add defensive check for from_converse response * Update instrumentation/opentelemetry-instrumentation-botocore/src/opentelemetry/instrumentation/botocore/extensions/bedrock_utils.py * Update CHANGELOG.md * add comment for malformed repsonses in from_converse * merge suggested changes * fix botocore converse stream test * lint fix * Update instrumentation/opentelemetry-instrumentation-botocore/tests/test_botocore_bedrock.py * add defensive check just in case content doesn't exist in orig_message * lint fix * new line --------- Co-authored-by: Riccardo Magliocchetti <riccardo.magliocchetti@gmail.com>

add defensive check for from_converse response

429f624

liustve requested a review from a team as a code owner December 1, 2025 22:57

liustve force-pushed the fix-bedrock-converse-stream-bug branch from 65e10cb to fde933a Compare December 1, 2025 23:00

liustve mentioned this pull request Dec 1, 2025

bedrock_utils.py throwing an error on #3958

Closed

add defensive check for from_converse response

0861f9f

liustve force-pushed the fix-bedrock-converse-stream-bug branch from 05bcdf9 to 0861f9f Compare December 1, 2025 23:03

xrmx reviewed Dec 2, 2025

View reviewed changes

...trumentation-botocore/src/opentelemetry/instrumentation/botocore/extensions/bedrock_utils.py Show resolved Hide resolved

Update instrumentation/opentelemetry-instrumentation-botocore/src/ope…

24ed6f7

…ntelemetry/instrumentation/botocore/extensions/bedrock_utils.py

xrmx approved these changes Dec 2, 2025

View reviewed changes

xrmx reviewed Dec 2, 2025

View reviewed changes

CHANGELOG.md Outdated Show resolved Hide resolved

Update CHANGELOG.md

753b370

xrmx added this to Python PR digest Dec 2, 2025

xrmx moved this to Approved PRs that need fixes in Python PR digest Dec 2, 2025

liustve added 3 commits December 2, 2025 08:47

add comment for malformed repsonses in from_converse

c348072

Merge branch 'fix-bedrock-converse-stream-bug' of https://github.com/…

79e4f77

…liustve/opentelemetry-python-contrib into fix-bedrock-converse-stream-bug

merge suggested changes

cf6012a

liustve added 2 commits December 2, 2025 09:14

fix botocore converse stream test

ed477de

lint fix

568b6bc

xrmx reviewed Dec 2, 2025

View reviewed changes

instrumentation/opentelemetry-instrumentation-botocore/tests/test_botocore_bedrock.py Show resolved Hide resolved

Update instrumentation/opentelemetry-instrumentation-botocore/tests/t…

ace58d2

…est_botocore_bedrock.py

liustve mentioned this pull request Dec 2, 2025

[PATCH] Add safety check for bedrock ConverseStream responses aws-observability/aws-otel-python-instrumentation#547

Merged

liustve added 4 commits December 2, 2025 14:40

add defensive check just in case content doesn't exist in orig_message

b019d66

Merge branch 'fix-bedrock-converse-stream-bug' of https://github.com/…

7e00629

…liustve/opentelemetry-python-contrib into fix-bedrock-converse-stream-bug

lint fix

331e4e9

new line

663c016

xrmx merged commit be8dfdc into open-telemetry:main Dec 3, 2025
647 checks passed

github-project-automation bot moved this from Approved PRs that need fixes to Done in Python PR digest Dec 3, 2025

Cirilla-zmh mentioned this pull request Dec 4, 2025

[chore] Sync OpenTelemetry upstream main. alibaba/loongsuite-python-agent#78

Merged

Sami-Tanquary mentioned this pull request Jan 15, 2026

[BUG] AttributeError in AgentCoreMemorySessionManager when created_message is None aws/bedrock-agentcore-sdk-python#215

Closed

This was referenced Jan 30, 2026

Bump OTel dependencies to 1.39.1/0.60b1 aws-observability/aws-otel-python-instrumentation#598

Closed

Nightly dependency update: OpenTelemetry 1.39.1/0.60b1 aws-observability/aws-otel-python-instrumentation#596

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add safety check for bedrock ConverseStream responses#3990

Add safety check for bedrock ConverseStream responses#3990
xrmx merged 14 commits intoopen-telemetry:mainfrom
liustve:fix-bedrock-converse-stream-bug

liustve commented Dec 1, 2025 •

edited by xrmx

Loading

Uh oh!

Uh oh!

Uh oh!

xrmx commented Dec 2, 2025

Uh oh!

xrmx commented Dec 2, 2025

Uh oh!

liustve commented Dec 2, 2025

Uh oh!

Uh oh!

liustve commented Dec 2, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

liustve commented Dec 1, 2025 • edited by xrmx Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

How Has This Been Tested?

Does This PR Require a Core Repo Change?

Checklist:

Uh oh!

Uh oh!

Uh oh!

xrmx commented Dec 2, 2025

Uh oh!

xrmx commented Dec 2, 2025

Uh oh!

liustve commented Dec 2, 2025

Uh oh!

Uh oh!

liustve commented Dec 2, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

liustve commented Dec 1, 2025 •

edited by xrmx

Loading