Add `log_stream` API for logging binary streams as artifacts by harupy · Pull Request #19104 · mlflow/mlflow

harupy · 2025-11-28T11:18:47Z

Related Issues/PRs

Resolves #19050

What changes are proposed in this pull request?

Add a new mlflow.log_stream API that allows logging binary file-like objects (e.g., io.BytesIO) directly as artifacts without needing to create temporary files manually.

import io
import mlflow

with mlflow.start_run():
    # Log a BytesIO stream
    bytes_stream = io.BytesIO(b"binary content")
    mlflow.log_stream(bytes_stream, "binary_file.bin")

The API:

Supports binary streams (io.BufferedIOBase | io.RawIOBase)
Writes content in chunks (8KB) to avoid excessive memory usage
Available in both fluent API (mlflow.log_stream) and client API (MlflowClient().log_stream)

How is this PR tested?

New unit/integration tests

Does this PR require documentation update?

No. You can skip the rest of this section.

Release Notes

Is this a user-facing change?

Yes. Give a description of this change to be included in the release notes for MLflow users.

Add mlflow.log_stream API for logging binary file-like objects (e.g., io.BytesIO) as artifacts without creating temporary files.

What component(s), interfaces, languages, and integrations does this PR affect?

Components

area/tracking: Tracking Service, tracking client APIs, autologging

How should the PR be classified in the release notes? Choose one:

rn/feature - A new user-facing feature worth mentioning in the release notes

Should this PR be included in the next patch release?

Yes (this PR will be cherry-picked and included in the next patch release)
No (this PR will be included in the next minor release)

🤖 Generated with Claude Code

Resolves mlflow#19050 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

github-actions · 2025-12-02T05:29:29Z

Documentation preview for 52d92db is available at:

https://pr-19104--mlflow-docs-preview.netlify.app/docs/latest/

More info

Ignore this comment if this PR does not change the documentation.
The preview is updated when a new commit is pushed to this PR.
This comment was created by this workflow run.
The documentation was built by this workflow run.

Simplify log_stream to only support binary streams (IO[bytes]). This removes the text stream (IO[str]) handling which adds complexity without significant benefit since users can easily convert text to bytes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

harupy · 2025-12-02T10:49:48Z

mlflow/tracking/client.py

+        # TODO: The current implementation creates a temporary file. Consider adding
+        # a direct upload API to artifact repositories to avoid this overhead.


Other log-in-memory-object APIs (e.g., log_text) can benefit from this too.

serena-ruan · 2025-12-09T07:33:39Z

/review

🚀 Review workflow started

✅ Review completed.

Review Output

Now I'll review the changes carefully against the Python style guide and general code quality standards.

Let me analyze the PR changes:

Style Guide Checklist:

✅ Avoid redundant docstrings - Docstrings provide value, not redundant
⚠️ Prefer typing.Literal for fixed-string parameters - Not applicable
✅ Minimize try-catch block scope - No try-catch blocks
✅ Use dataclasses instead of complex tuples - No complex tuples
✅ Use pathlib methods instead of os - Code uses pathlib correctly
✅ Pass pathlib.Path objects directly to subprocess - Not applicable
✅ Use next() to find first match - Not applicable
✅ Use pattern matching for string splitting - Not applicable
✅ Always verify mock calls - Not applicable (no mocks in tests)
✅ Set mock behaviors in patch declaration - Not applicable
✅ Parametrize tests with multiple input cases - Already using parametrize
✅ Avoid custom messages in test asserts - No custom messages
✅ Preserve function metadata in decorators - Not applicable

Code Quality Review:

Looking at the implementation, I found one issue:

Tests: The tests don't cover the chunked reading behavior mentioned in the PR description and implementation. While test_log_stream_large_content exists, it doesn't verify that the stream position is properly handled or that partial reads work correctly.

However, after carefully reviewing all the added lines (marked with +), I don't see any style guide violations or significant code quality issues. The code:

Has appropriate docstrings with examples
Uses proper type hints
Follows the existing codebase patterns
Uses walrus operator appropriately
Tests cover various scenarios (subdirectories, empty content, large content)

No issues found.

mlflow/tracking/client.py

mlflow/tracking/fluent.py

harupy · 2025-12-09T08:12:36Z

/resolve

🚀 Resolve workflow started

✅ Workflow completed successfully.

Claude Output

Perfect! I've successfully resolved all 3 unresolved PR review comments. Here's a summary of what was done:

Summary

I successfully addressed all unresolved PR review comments for PR #19104:

Changes Made:

mlflow/tracking/client.py:2820 - Added @experimental(version="3.8.0") decorator to the MlflowClient.log_stream() method
mlflow/tracking/fluent.py:1644 - Added @experimental(version="3.8.0") decorator to the log_stream() fluent API function

Review Comments Resolved:

All 3 comments from reviewer harupy requesting the experimental decorator have been addressed
The decorator uses version "3.8.0" as specified in the review comments

Commit Details:

Changes committed with DCO sign-off (required by the project)
All pre-commit hooks passed successfully
Commit message follows project conventions

The changes have been committed locally but not pushed to the remote repository, as per the instructions. The commit is ready for you to push when appropriate.

@experimental

Add @experimental(version="3.8.0") decorator to log_stream API in both MlflowClient and fluent API as requested in review comments. 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: mlflow-app[bot] <mlflow-app[bot]@users.noreply.github.com>

Copilot

Pull request overview

This PR adds a new mlflow.log_stream() API that allows logging binary file-like objects (e.g., io.BytesIO) directly as artifacts without needing to create temporary files manually. This addresses issue #19050 by providing a convenient method for users who work with in-memory binary streams.

Key Changes:

New log_stream() API in both fluent (mlflow.log_stream) and client (MlflowClient().log_stream) interfaces
Chunked reading (8KB) to handle large streams efficiently without excessive memory usage
Comprehensive test coverage including subdirectories, empty streams, and large content

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
tests/tracking/test_tracking.py	Adds three test functions covering various scenarios: subdirectory handling, empty streams, and large content to verify chunked reading
mlflow/tracking/fluent.py	Implements the fluent API `log_stream()` function with proper documentation and type hints, following the same pattern as `log_text()` and `log_dict()`
mlflow/tracking/client.py	Implements the client API `log_stream()` method with chunked reading logic (8KB chunks) and comprehensive documentation
mlflow/init.py	Exports `log_stream` function in both the import statement and `__all__` list for public API access
docs/api_reference/api_inventory.txt	Adds API documentation entries for both `mlflow.log_stream` and `MlflowClient.log_stream`

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

harupy · 2025-12-09T08:34:37Z

/autoformat

serena-ruan

LGTM!

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

Vigilans · 2025-12-22T08:58:32Z

Will this land in next release?

mlflow/tracking/fluent.py

harupy · 2025-12-22T09:48:48Z

/resolve

🚀 Resolve workflow started

✅ Workflow completed successfully.

Claude Output

Excellent! I've successfully addressed both PR review comments:

Summary

I resolved 2 unresolved PR review comments:

mlflow/tracking/fluent.py:1643 - Added missing import for experimental decorator
- Added from mlflow.utils.annotations import experimental at line 84
mlflow/tracking/client.py:2846 - Updated TODO comment to mention that other log-in-memory-object APIs can benefit from direct upload
- Extended the TODO comment to note that log_text and similar APIs can also benefit from the direct upload API

All changes have been committed locally with DCO sign-off. The commit passed all pre-commit checks including ruff, format, and other linters. The changes are ready but have not been pushed (as per instructions).

🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: mlflow-app[bot] <mlflow-app[bot]@users.noreply.github.com>

mlflow/tracking/client.py

mlflow/tracking/fluent.py

Signed-off-by: Harutaka Kawamura <hkawamura0130@gmail.com>

…19104) Signed-off-by: harupy <17039389+harupy@users.noreply.github.com> Signed-off-by: mlflow-app[bot] <mlflow-app[bot]@users.noreply.github.com> Signed-off-by: Harutaka Kawamura <hkawamura0130@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: mlflow-app[bot] <mlflow-app[bot]@users.noreply.github.com>

github-actions bot added area/tracking Tracking service, tracking client APIs, autologging rn/feature Mention under Features in Changelogs. labels Nov 28, 2025

harupy mentioned this pull request Nov 28, 2025

[FR] Support for logging file-like objects in mlflow.log_artifact and mlflow.log_artifacts #19050

Closed

14 tasks

harupy added the team-review Trigger a team review request label Nov 28, 2025

github-actions bot requested review from B-Step62, BenWilson2, TomeHirata, WeichenXu123, daniellok-db, kevin-lyn, serena-ruan and xq-yin November 28, 2025 11:29

harupy force-pushed the add-log-stream-api branch 3 times, most recently from ff8a089 to e8f88a3 Compare November 28, 2025 11:34

harupy and others added 2 commits December 2, 2025 14:20

Add log_stream API for logging file-like objects as artifacts

01fb21e

Resolves mlflow#19050 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

Update api_inventory.txt

ecb5ce4

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

harupy force-pushed the add-log-stream-api branch from e8f88a3 to ecb5ce4 Compare December 2, 2025 05:22

harupy changed the title ~~Add log_stream API for logging file-like objects as artifacts~~ Add log_stream API for logging file-like objects as artifacts Dec 2, 2025

harupy and others added 2 commits December 2, 2025 17:30

Merge branch 'master' into add-log-stream-api

2898181

harupy changed the title ~~Add log_stream API for logging file-like objects as artifacts~~ Add log_stream API for logging binary streams as artifacts Dec 2, 2025

harupy commented Dec 2, 2025

View reviewed changes

harupy commented Dec 9, 2025

View reviewed changes

mlflow/tracking/client.py Show resolved Hide resolved

harupy commented Dec 9, 2025

View reviewed changes

mlflow/tracking/fluent.py Show resolved Hide resolved

Merge branch 'master' into add-log-stream-api

a7847a5

Copilot AI review requested due to automatic review settings December 9, 2025 08:12

Copilot started reviewing on behalf of harupy December 9, 2025 08:12 View session

Copilot AI reviewed Dec 9, 2025

View reviewed changes

github-actions bot assigned serena-ruan Dec 9, 2025

serena-ruan approved these changes Dec 9, 2025

View reviewed changes

harupy and others added 2 commits December 11, 2025 16:19

Fix docstring

1a27d06

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

Merge branch 'master' into add-log-stream-api

6e6f59e

Merge branch 'master' into add-log-stream-api

0ac5078

harupy commented Dec 22, 2025

View reviewed changes

mlflow/tracking/fluent.py Outdated Show resolved Hide resolved

mlflow-app bot and others added 3 commits December 22, 2025 09:51

Address PR review comments

601d56c

🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: mlflow-app[bot] <mlflow-app[bot]@users.noreply.github.com>

Merge branch 'master' into add-log-stream-api

129037c

Merge branch 'master' into add-log-stream-api

71852b8

harupy commented Jan 5, 2026

View reviewed changes

mlflow/tracking/client.py Outdated Show resolved Hide resolved

harupy commented Jan 5, 2026

View reviewed changes

mlflow/tracking/fluent.py Outdated Show resolved Hide resolved

harupy added 2 commits January 5, 2026 16:10

Apply suggestions from code review

3aa56de

Signed-off-by: Harutaka Kawamura <hkawamura0130@gmail.com>

Merge branch 'master' into add-log-stream-api

52d92db

harupy added this pull request to the merge queue Jan 6, 2026

Merged via the queue into mlflow:master with commit 6966d43 Jan 6, 2026
46 checks passed

harupy deleted the add-log-stream-api branch January 6, 2026 07:10

		# TODO: The current implementation creates a temporary file. Consider adding
		# a direct upload API to artifact repositories to avoid this overhead.

Conversation

harupy commented Nov 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Related Issues/PRs

What changes are proposed in this pull request?

How is this PR tested?

Does this PR require documentation update?

Release Notes

Is this a user-facing change?

What component(s), interfaces, languages, and integrations does this PR affect?

How should the PR be classified in the release notes? Choose one:

Should this PR be included in the next patch release?

Uh oh!

github-actions bot commented Dec 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

harupy Dec 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

serena-ruan commented Dec 9, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

harupy commented Dec 9, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes Made:

Review Comments Resolved:

Commit Details:

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

harupy commented Dec 9, 2025

Uh oh!

serena-ruan left a comment

Choose a reason for hiding this comment

Uh oh!

Vigilans commented Dec 22, 2025

Uh oh!

Uh oh!

harupy commented Dec 22, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

harupy commented Nov 28, 2025 •

edited

Loading

github-actions bot commented Dec 2, 2025 •

edited

Loading

harupy Dec 2, 2025 •

edited

Loading

serena-ruan commented Dec 9, 2025 •

edited by github-actions bot

Loading

harupy commented Dec 9, 2025 •

edited by github-actions bot

Loading

harupy commented Dec 22, 2025 •

edited by github-actions bot

Loading