bpo-42318: Fix support of non-BMP characters in Tkinter on macOS #23281

serhiy-storchaka · 2020-11-14T11:47:52Z

https://bugs.python.org/issue42318

ronaldoussoren

The PR looks good to me.

I've tested the PR with the copy of Tk 8.6.10 that I reported the issue for, and with this patch the problem is gone. I also get correct syntax colouring in edit windows.

ronaldoussoren · 2020-11-15T11:32:25Z

Modules/_tkinter.c

+        return NULL;
+    }
+    Py_UCS4 ch;
+    for (j = i; i < len; i++, u[j++] = ch) {


I had to reread this loop a couple of times before I understood how the contents of "u" is updated. My C is getting a bit rusty ;-)

The code looks fine though.

miss-islington · 2020-11-15T16:17:02Z

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.8, 3.9.
🐍🍒⛏🤖

miss-islington · 2020-11-15T16:17:22Z

Sorry, @serhiy-storchaka, I could not cleanly backport this to 3.9 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker a26215db11cfcf7b5f55cab9e91396761a0e0bcf 3.9

miss-islington · 2020-11-15T16:17:24Z

Sorry @serhiy-storchaka, I had trouble checking out the 3.8 backport branch.
Please backport using cherry_picker on command line.
cherry_picker a26215db11cfcf7b5f55cab9e91396761a0e0bcf 3.8

pythonGH-23281). (cherry picked from commit a26215d) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

bedevere-bot · 2020-12-15T16:55:28Z

GH-23784 is a backport of this pull request to the 3.9 branch.

GH-23281). (GH-23784) (cherry picked from commit a26215d)

pythonGH-23281). (pythonGH-23784) (cherry picked from commit a26215d) (cherry picked from commit 28bf6ab) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

bedevere-bot · 2020-12-15T18:57:58Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL8 LTO + PGO 3.9 has failed when building commit 28bf6ab.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/436/builds/422) and take a look at the build logs.
Check if the failure is related to this commit (28bf6ab) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/436/builds/422

Summary of the results of the build (if available):

== Tests result: ENV CHANGED ==

409 tests OK.

10 slowest tests:

test_concurrent_futures: 2 min 40 sec
test_multiprocessing_spawn: 1 min 10 sec
test_multiprocessing_forkserver: 1 min
test_multiprocessing_fork: 56.5 sec
test_asyncio: 48.3 sec
test_signal: 47.1 sec
test_io: 34.7 sec
test_imaplib: 33.5 sec
test_pydoc: 31.1 sec
test_subprocess: 29.6 sec

1 test altered the execution environment:
test_asyncio

15 tests skipped:
test_devpoll test_gdb test_ioctl test_kqueue test_msilib test_nis
test_ossaudiodev test_startfile test_tix test_tk test_ttk_guionly
test_winconsoleio test_winreg test_winsound test_zipfile64

Total duration: 4 min 47 sec

Click to see traceback logs

Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.9.edelsohn-rhel8-z.lto-pgo/build/Lib/asyncio/sslproto.py", line 321, in __del__
    self.close()
  File "/home/dje/cpython-buildarea/3.9.edelsohn-rhel8-z.lto-pgo/build/Lib/asyncio/sslproto.py", line 316, in close
    self._ssl_protocol._start_shutdown()
  File "/home/dje/cpython-buildarea/3.9.edelsohn-rhel8-z.lto-pgo/build/Lib/asyncio/sslproto.py", line 590, in _start_shutdown
    self._abort()
  File "/home/dje/cpython-buildarea/3.9.edelsohn-rhel8-z.lto-pgo/build/Lib/asyncio/sslproto.py", line 731, in _abort
    self._transport.abort()
  File "/home/dje/cpython-buildarea/3.9.edelsohn-rhel8-z.lto-pgo/build/Lib/asyncio/selector_events.py", line 680, in abort
    self._force_close(None)
  File "/home/dje/cpython-buildarea/3.9.edelsohn-rhel8-z.lto-pgo/build/Lib/asyncio/selector_events.py", line 731, in _force_close
    self._loop.call_soon(self._call_connection_lost, exc)
  File "/home/dje/cpython-buildarea/3.9.edelsohn-rhel8-z.lto-pgo/build/Lib/asyncio/base_events.py", line 746, in call_soon
    self._check_closed()
  File "/home/dje/cpython-buildarea/3.9.edelsohn-rhel8-z.lto-pgo/build/Lib/asyncio/base_events.py", line 510, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

GH-23281). (GH-23784) (GH-23787) (cherry picked from commit a26215d) (cherry picked from commit 28bf6ab)

…honGH-23281)

serhiy-storchaka added type-bug An unexpected behavior, bug, or error needs backport to 3.8 needs backport to 3.9 labels Nov 14, 2020

serhiy-storchaka requested a review from ronaldoussoren Nov 14, 2020

the-knights-who-say-ni added the CLA signed label Nov 14, 2020

bedevere-bot added the awaiting core review label Nov 14, 2020

bpo-42318: Fix support of non-BMP characters in Tkinter on macOS

6290d11

serhiy-storchaka changed the title ~~Fix support of non-BMP characters in Tkinter on macOS~~ bpo-42318: Fix support of non-BMP characters in Tkinter on macOS Nov 14, 2020

serhiy-storchaka force-pushed the tkinter-macos-nonbmp branch from a7166d4 to 6290d11 Compare Nov 14, 2020

Fix tests on non-UTF-8 locales.

9ee01c6

ronaldoussoren approved these changes Nov 15, 2020

View changes

bedevere-bot added awaiting merge and removed awaiting core review labels Nov 15, 2020

serhiy-storchaka merged commit a26215d into python:master Nov 15, 2020
10 checks passed

bedevere-bot removed the awaiting merge label Nov 15, 2020

serhiy-storchaka deleted the tkinter-macos-nonbmp branch Nov 15, 2020

miss-islington assigned serhiy-storchaka Nov 15, 2020

bedevere-bot removed the needs backport to 3.9 label Dec 15, 2020

serhiy-storchaka added a commit that referenced this pull request Dec 15, 2020

[3.9] bpo-42318: Fix support of non-BMP characters in Tkinter on macOS (

28bf6ab

GH-23281). (GH-23784) (cherry picked from commit a26215d)

serhiy-storchaka pushed a commit that referenced this pull request Dec 25, 2020

[3.8] bpo-42318: Fix support of non-BMP characters in Tkinter on macOS (

4d840e4

GH-23281). (GH-23784) (GH-23787) (cherry picked from commit a26215d) (cherry picked from commit 28bf6ab)

serhiy-storchaka removed the needs backport to 3.8 label Dec 31, 2020

serhiy-storchaka removed their assignment Dec 31, 2020

adorilson pushed a commit to adorilson/cpython that referenced this pull request Mar 13, 2021

bpo-42318: Fix support of non-BMP characters in Tkinter on macOS (pyt…

6be06f9

…honGH-23281)

bpo-42318: Fix support of non-BMP characters in Tkinter on macOS #23281

bpo-42318: Fix support of non-BMP characters in Tkinter on macOS #23281

serhiy-storchaka commented Nov 14, 2020 •

edited by bedevere-bot

ronaldoussoren left a comment

ronaldoussoren Nov 15, 2020

miss-islington commented Nov 15, 2020

miss-islington commented Nov 15, 2020

miss-islington commented Nov 15, 2020

bedevere-bot commented Dec 15, 2020

bedevere-bot commented Dec 15, 2020

bpo-42318: Fix support of non-BMP characters in Tkinter on macOS #23281

bpo-42318: Fix support of non-BMP characters in Tkinter on macOS #23281

Conversation

serhiy-storchaka commented Nov 14, 2020 • edited by bedevere-bot

ronaldoussoren left a comment

ronaldoussoren Nov 15, 2020

Choose a reason for hiding this comment

miss-islington commented Nov 15, 2020

miss-islington commented Nov 15, 2020

miss-islington commented Nov 15, 2020

bedevere-bot commented Dec 15, 2020

bedevere-bot commented Dec 15, 2020

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

serhiy-storchaka commented Nov 14, 2020 •

edited by bedevere-bot