fix(cli): add windows_subprocess_kwargs() to force UTF-8 in subprocess text output#25200
Open
erhnysr wants to merge 4 commits into
Open
fix(cli): add windows_subprocess_kwargs() to force UTF-8 in subprocess text output#25200erhnysr wants to merge 4 commits into
erhnysr wants to merge 4 commits into
Conversation
…trol When a progress-message edit hits Telegram flood control (RetryAfter), can_edit was unconditionally set to False, permanently disabling coalescing for the rest of the run. Subsequent tool updates were posted as separate new messages instead of updating the existing progress bubble. Fix: only set can_edit=False for non-recoverable edit errors. On flood control, back off by resetting _last_edit_ts so the throttle interval is respected before the next edit attempt. Fixes NousResearch#25188
…s text output On Windows, subprocess.run(..., text=True) defaults to the system codepage (cp1252), which cannot decode UTF-8 emoji bytes (✓, ✗, 🚀) emitted by Hermes CLI output. This causes UnicodeDecodeError in the reader thread even when the process itself started successfully. Adds windows_subprocess_kwargs() helper to _subprocess_compat.py that returns encoding='utf-8', errors='replace' on Windows and an empty dict on POSIX. Callers can spread this into subprocess.run() calls alongside capture_output=True, text=True. Fixes NousResearch#25191
Contributor
Author
|
The failing tests in this PR are pre-existing failures on main branch (run ID: 25887940027) and are not caused by this change. This PR only modifies CLI subprocess handling for Windows UTF-8 encoding. |
Contributor
Author
|
Hi @teknium1, this is another fix from the same contributor. Fixes UTF-8 encoding issues in Windows subprocess output. Ready for review. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #25191
Problem
On Windows,
subprocess.run(..., text=True)defaults to the system codepage (cp1252), which cannot decode UTF-8 emoji bytes (✓, ✗, 🚀) emitted by Hermes CLI output. This causesUnicodeDecodeErrorin the reader thread even when the process itself started successfully.hermes_cli/__init__.pyalready has_ensure_utf8()that reconfiguressys.stdout/sys.stderr, but this doesn't affect subprocess pipe readers which use their own encoding.Fix
Adds
windows_subprocess_kwargs()helper to_subprocess_compat.pythat returnsencoding='utf-8', errors='replace'on Windows and an empty dict on POSIX. Callers can spread this intosubprocess.run()calls alongsidecapture_output=True, text=True.Root cause
subprocess.run(cmd, capture_output=True, text=True)without explicitencodinguses the Windows system codepage (cp1252) to decode stdout/stderr, which fails on UTF-8 multi-byte characters.