fix(tui): use base64 encoding for PowerShell clipboard writes to preserve UTF-8#35124
Closed
annguyenNous wants to merge 1 commit into
Closed
fix(tui): use base64 encoding for PowerShell clipboard writes to preserve UTF-8#35124annguyenNous wants to merge 1 commit into
annguyenNous wants to merge 1 commit into
Conversation
…erve UTF-8 When writing text to the clipboard via PowerShell (WSL2 and native Windows), the previous implementation piped text through stdin using `Set-Clipboard -Value $input`. PowerShell reads stdin using the Windows system's default ANSI code page (e.g. CP936 for Chinese Windows), causing all non-ASCII characters (CJK, emoji, accented) to become garbled. Fix: encode the text as base64 in Node.js and pass it as a command argument. PowerShell decodes it from base64 using explicit UTF-8, bypassing the code page issue entirely. Fixes NousResearch#35107
Collaborator
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When using
/copyin TUI on WSL2, non-ASCII text (CJK, emoji, accented characters) becomes garbled when pasted on the Windows side.Root cause:
writeClipboardText()pipes text topowershell.exe Set-Clipboard -Value $inputvia stdin. PowerShell reads stdin using the Windows system's default ANSI code page (e.g. CP936 for Chinese Windows), not UTF-8. This reinterprets UTF-8 bytes as ANSI, producing garbled output.Example:
你好世界→浣犲ソ涓栫晫Fix
Encode text as base64 in Node.js and pass it as a PowerShell command argument instead of piping through stdin. PowerShell explicitly decodes from base64 using UTF-8:
This bypasses the ANSI code page entirely since the text is passed as a command argument (which PowerShell handles in UTF-8) rather than through stdin.
Before vs After
你好世界，测试中文浣犲ソ涓栫晫锛屾祴璇曚腑鏂你好世界，测试中文✅café résumécaf茅 r茅sum茅café résumé✅🎉 Hello🎉 Hello✅Changes
ui-tui/src/lib/clipboard.ts: PowerShell clipboard writes use base64-encoded UTF-8 via-Commandargument instead of stdin piping. Other backends (pbcopy, wl-copy, xclip, xsel) continue using stdin as before.ui-tui/src/__tests__/clipboard.test.ts: Updated existing tests to verify base64 encoding, added CJK-specific regression test.Tests
All 19 clipboard tests pass (3 new assertions + 1 new test for CJK text).
Fixes #35107