Skip to content

Read script output/error streams as they become available#14695

Merged
Carreau merged 8 commits intoipython:mainfrom
krassowski:fix-line-streaming
Jan 31, 2025
Merged

Read script output/error streams as they become available#14695
Carreau merged 8 commits intoipython:mainfrom
krassowski:fix-line-streaming

Conversation

@krassowski
Copy link
Copy Markdown
Member

Fixes #14648

@krassowski krassowski marked this pull request as ready for review January 29, 2025 16:22
@krassowski
Copy link
Copy Markdown
Member Author

The test is flaky on Windows, I will rework it..

@krassowski krassowski marked this pull request as draft January 29, 2025 16:50
@krassowski krassowski marked this pull request as ready for review January 30, 2025 10:40
async def _readchunk(stream):
try:
return await stream.readuntil(b"\n")
return await stream.read(100)
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will consume any up to 100 bytes, as soon as at least 1 byte is available. The 100 number is arbitrary, but my hope is it will roughly correspond to an average line length so that it should not change how small/frequent chunks are compared to before (but of course if someone streamed lines of 1000 characters, now those would be returned in 10 chunks instead of 1).

https://docs.python.org/3/library/asyncio-stream.html#asyncio.StreamReader.read

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Jupyter shebang cells won't show partially written lines

2 participants