feat(typescript-anthropic): add streaming support by rollyjoel · Pull Request #20384 · mlflow/mlflow

rollyjoel · 2026-01-27T22:07:47Z

Related Issues/PRs

#20382

What changes are proposed in this pull request?

Add tracing support for the messages.stream() method in the MLflow Anthropic TypeScript integration.

The stream method returns a MessageStream object that emits events as the response streams in. This implementation traces streaming by:

Wrapping the returned MessageStream object in a Proxy
Intercepting finalMessage() calls to create a span around the full streaming operation
Intercepting the async iterator (Symbol.asyncIterator) to trace for await loops
Extracting token usage from the final message after streaming completes
Setting appropriate span attributes (MESSAGE_FORMAT: 'anthropic', token usage, etc.)
libs/typescript/integrations/anthropic/src/index.ts - Main implementation
libs/typescript/integrations/anthropic/tests/index.test.ts - Tests
libs/typescript/integrations/anthropic/tests/mockAnthropicServer.ts - Mock SSE endpoint

How is this PR tested?

Existing unit/integration tests
New unit/integration tests
Manual tests

Does this PR require documentation update?

Release Notes

Is this a user-facing change?

No. You can skip the rest of this section.
Yes. Give a description of this change to be included in the release notes for MLflow users.

Add support for @anthropic-ai/sdk's streaming responses

What component(s), interfaces, languages, and integrations does this PR affect?

Components

How should the PR be classified in the release notes? Choose one:

rn/none - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section
rn/breaking-change - The PR will be mentioned in the "Breaking Changes" section
rn/feature - A new user-facing feature worth mentioning in the release notes
rn/bug-fix - A user-facing bug fix worth mentioning in the release notes
rn/documentation - A user-facing documentation change worth mentioning in the release notes

Should this PR be included in the next patch release?

Yes should be selected for bug fixes, documentation updates, and other small changes. No should be selected for new features and larger changes. If you're unsure about the release classification of this PR, leave this unchecked to let the maintainers decide.

What is a minor/patch release?

Minor release: a release that increments the second part of the version number (e.g., 1.2.0 -> 1.3.0).
Bug fixes, doc updates and new features usually go into minor releases.
Patch release: a release that increments the third part of the version number (e.g., 1.2.0 -> 1.2.1).
Bug fixes and doc updates usually go into patch releases.

Yes (this PR will be cherry-picked and included in the next patch release)
No (this PR will be included in the next minor release)

github-actions · 2026-01-27T22:10:38Z

🛠 DevTools 🛠

Install mlflow from this PR

# mlflow
pip install git+https://github.com/mlflow/mlflow.git@refs/pull/20384/merge
# mlflow-skinny
pip install git+https://github.com/mlflow/mlflow.git@refs/pull/20384/merge#subdirectory=libs/skinny

For Databricks, use the following command:

%sh curl -LsSf https://raw.githubusercontent.com/mlflow/mlflow/HEAD/dev/install-skinny.sh | sh -s pull/20384/merge

Copilot

Pull request overview

This pull request adds streaming support for the MLflow Anthropic TypeScript integration, addressing issue #20382. The implementation enables tracing for the messages.stream() method by wrapping the returned MessageStream object with a Proxy that intercepts both async iteration and finalMessage() calls.

Changes:

Added tracing support for Anthropic's messages.stream() method
Implemented Proxy-based wrapping for MessageStream with dual patterns: async iteration and finalMessage()
Added mock SSE endpoint and comprehensive test coverage for streaming scenarios

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File	Description
libs/typescript/integrations/anthropic/src/index.ts	Core streaming tracing implementation with Proxy-based MessageStream wrapping and async generator support
libs/typescript/integrations/anthropic/tests/index.test.ts	Added three comprehensive tests for streaming: async iteration, finalMessage(), and parent span integration
libs/typescript/integrations/anthropic/tests/mockAnthropicServer.ts	Added SSE streaming endpoint mock with proper event sequence simulation
docs/docs/genai/tracing/integrations/listing/anthropic.mdx	Updated feature matrix to indicate streaming support

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

libs/typescript/integrations/anthropic/src/index.ts

libs/typescript/integrations/anthropic/tests/index.test.ts

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

libs/typescript/integrations/anthropic/src/index.ts

libs/typescript/integrations/anthropic/tests/index.test.ts

libs/typescript/integrations/anthropic/src/index.ts

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

libs/typescript/integrations/anthropic/src/index.ts

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

github-actions · 2026-01-29T06:31:59Z

Documentation preview for 58e9a55 is available at:

https://pr-20384--mlflow-docs-preview.netlify.app/docs/latest/

Changed Pages (1)

genai/tracing/integrations/listing/anthropic (modified)

More info

Ignore this comment if this PR does not change the documentation.
The preview is updated when a new commit is pushed to this PR.
This comment was created by this workflow run.
The documentation was built by this workflow run.

rollyjoel · 2026-01-29T12:47:58Z

Thank you to whomever approved the action runs! I've fixed the lint/prettier errors and would love another run!

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

B-Step62

LGTM! Left a minor comment.

B-Step62 · 2026-01-30T04:52:55Z

libs/typescript/integrations/anthropic/src/index.ts

+                  span.setAttribute(SpanAttributeKey.TOKEN_USAGE, usage);
+                }
+              } catch (error) {
+                console.debug('Error extracting token usage', error);


Let's remove console.debug usage

seems to be consistent with how the other integrations handle failure to extract token messages. If you do want me to break the pattern, should I just catch (error) {}?

also, apologies I neglected to run lint 🤦 fixed inline with the other integrations and pushed up another commit (I squashed all the previous feedback commits)

Oh my bad, I wasn't aware we use that extensibly in other flavors. Let's keep it as-is then:)

Add tracing support for the `messages.stream()` method in the MLflow Anthropic TypeScript integration. The `stream` method returns a `MessageStream` object that emits events as the response streams in. This implementation traces streaming by: - Wrapping the returned `MessageStream` object in a Proxy - Intercepting `finalMessage()` calls to create a span around the full streaming operation - Intercepting the async iterator (`Symbol.asyncIterator`) to trace `for await` loops - Extracting token usage from the final message after streaming completes - Setting appropriate span attributes (`MESSAGE_FORMAT: 'anthropic'`, token usage, etc.) - `libs/typescript/integrations/anthropic/src/index.ts` - Main implementation - `libs/typescript/integrations/anthropic/tests/index.test.ts` - Tests - `libs/typescript/integrations/anthropic/tests/mockAnthropicServer.ts` - Mock SSE endpoint Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Joel Dodge <joeldodge@gmail.com>

Signed-off-by: Joel Dodge <joeldodge@gmail.com>

rollyjoel force-pushed the streaming branch from 3557778 to bcbc3dc Compare January 27, 2026 22:08

rollyjoel marked this pull request as ready for review January 27, 2026 22:10

Copilot AI review requested due to automatic review settings January 27, 2026 22:10

github-actions bot added area/tracing MLflow Tracing and its integrations rn/feature Mention under Features in Changelogs. labels Jan 27, 2026

Copilot started reviewing on behalf of rollyjoel January 27, 2026 22:10 View session

rollyjoel marked this pull request as draft January 27, 2026 22:11

Copilot AI reviewed Jan 27, 2026

View reviewed changes

rollyjoel force-pushed the streaming branch from 7d07ee1 to 5fd52b0 Compare January 27, 2026 23:03

rollyjoel requested a review from Copilot January 27, 2026 23:05

Copilot started reviewing on behalf of rollyjoel January 27, 2026 23:05 View session

Copilot AI reviewed Jan 27, 2026

View reviewed changes

libs/typescript/integrations/anthropic/src/index.ts Outdated Show resolved Hide resolved

libs/typescript/integrations/anthropic/tests/index.test.ts Show resolved Hide resolved

libs/typescript/integrations/anthropic/src/index.ts Show resolved Hide resolved

rollyjoel force-pushed the streaming branch from b158598 to 7938a1b Compare January 27, 2026 23:19

rollyjoel requested a review from Copilot January 27, 2026 23:21

Copilot started reviewing on behalf of rollyjoel January 27, 2026 23:21 View session

Copilot AI reviewed Jan 27, 2026

View reviewed changes

libs/typescript/integrations/anthropic/src/index.ts Show resolved Hide resolved

libs/typescript/integrations/anthropic/src/index.ts Show resolved Hide resolved

rollyjoel force-pushed the streaming branch from bf14991 to 797335e Compare January 27, 2026 23:47

rollyjoel requested a review from Copilot January 27, 2026 23:49

Copilot started reviewing on behalf of rollyjoel January 27, 2026 23:50 View session

Copilot AI reviewed Jan 27, 2026

View reviewed changes

rollyjoel force-pushed the streaming branch from 797335e to 653812c Compare January 28, 2026 00:00

rollyjoel marked this pull request as ready for review January 28, 2026 00:01

rollyjoel mentioned this pull request Jan 28, 2026

[FR] mlflow-anthropic typescript integration streaming support #20382

Closed

14 tasks

rollyjoel force-pushed the streaming branch from 653812c to d394b79 Compare January 29, 2026 12:42

github-actions bot added the v3.9.1 label Jan 29, 2026

rollyjoel requested a review from Copilot January 29, 2026 12:44

Copilot started reviewing on behalf of rollyjoel January 29, 2026 12:44 View session

Copilot AI reviewed Jan 29, 2026

View reviewed changes

B-Step62 approved these changes Jan 30, 2026

View reviewed changes

github-actions bot assigned B-Step62 Jan 30, 2026

rollyjoel force-pushed the streaming branch 2 times, most recently from 04f76f1 to ea8796d Compare January 31, 2026 18:32

rollyjoel mentioned this pull request Jan 31, 2026

chore(typescript-core): bump @opentelemetry/sdk-node version #20363

Merged

29 tasks

rollyjoel requested a review from B-Step62 January 31, 2026 18:34

rollyjoel and others added 2 commits February 2, 2026 17:24

fix lint

58e9a55

Signed-off-by: Joel Dodge <joeldodge@gmail.com>

rollyjoel force-pushed the streaming branch from ea8796d to 58e9a55 Compare February 3, 2026 00:25

B-Step62 enabled auto-merge February 3, 2026 09:56

B-Step62 added this pull request to the merge queue Feb 3, 2026

Merged via the queue into mlflow:master with commit 55dd48e Feb 3, 2026
47 checks passed

Conversation

rollyjoel commented Jan 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Related Issues/PRs

What changes are proposed in this pull request?

How is this PR tested?

Does this PR require documentation update?

Release Notes

Is this a user-facing change?

What component(s), interfaces, languages, and integrations does this PR affect?

How should the PR be classified in the release notes? Choose one:

Should this PR be included in the next patch release?

Uh oh!

github-actions bot commented Jan 27, 2026

Install mlflow from this PR

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

github-actions bot commented Jan 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rollyjoel commented Jan 29, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

B-Step62 left a comment

Choose a reason for hiding this comment

Uh oh!

B-Step62 Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

rollyjoel Jan 31, 2026

Choose a reason for hiding this comment

Uh oh!

B-Step62 Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rollyjoel commented Jan 27, 2026 •

edited

Loading

github-actions bot commented Jan 29, 2026 •

edited

Loading