Using tkinter.filedialog crashes on macOS 12.0 Beta (21A5294g) on M1 when the open file dialog window is created. Full crash below:

2021-08-04 07:19:04.239 Python[40251:323363] *** Assertion failure in -[NSOpenPanel beginServicePanel:asyncExHandler:], NSVBOpenAndSavePanels.m:1910
2021-08-04 07:19:04.241 Python[40251:323363] -[NSSavePanel beginWithCompletionHandler:]_block_invoke caught non-fatal NSInternalInconsistencyException '<NSOpenPanel: 0x1206062d0> is attempting to advance this Open/Save panel to run phase while another self.advanceToRunPhaseCompletionHandler is in waiting for a previous attempt. An Open/Save panel cannot start to advance more than once.' with user dictionary {
    NSAssertFile = "NSVBOpenAndSavePanels.m";
    NSAssertLine = 1910;
} and backtrace (
	0   CoreFoundation                      0x00000001a9d47150 __exceptionPreprocess + 240
	1   libobjc.A.dylib                     0x00000001a9a986e8 objc_exception_throw + 60
	2   Foundation                          0x00000001aac3b4a4 -[NSCalendarDate initWithCoder:] + 0
	3   AppKit                              0x00000001ad1f02b0 -[NSSavePanel beginServicePanel:asyncExHandler:] + 512
	4   AppKit                              0x00000001ad1f1708 -[NSSavePanel runModal] + 332
	5   libtk8.6.dylib                      0x00000001013d8c18 showOpenSavePanel + 360
	6   libtk8.6.dylib                      0x00000001013d99e4 Tk_ChooseDirectoryObjCmd + 992
	7   libtcl8.6.dylib                     0x00000001011cbafc TclNRRunCallbacks + 80
	8   _tkinter.cpython-39-darwin.so       0x0000000100c111a4 Tkapp_Call + 400
	9   Python                              0x0000000100d66a40 cfunction_call + 96
	10  Python                              0x0000000100d184e0 _PyObject_Call + 128
	11  Python                              0x0000000100e10150 _PyEval_EvalFrameDefault + 40288
	12  Python                              0x0000000100e053f0 _PyEval_EvalCode + 444
	13  Python                              0x0000000100d1877c _PyFunction_Vectorcall + 364
	14  Python                              0x0000000100e12590 call_function + 128
	15  Python                              0x0000000100e0ff08 _PyEval_EvalFrameDefault + 39704
	16  Python                              0x0000000100e053f0 _PyEval_EvalCode + 444
	17  Python                              0x0000000100d1877c _PyFunction_Vectorcall + 364
	18  Python                              0x0000000100e12590 call_function + 128
	19  Python                              0x0000000100e0ff84 _PyEval_EvalFrameDefault + 39828
	20  Python                              0x0000000100e053f0 _PyEval_EvalCode + 444
	21  Python                              0x0000000100e5cce4 run_eval_code_obj + 136
	22  Python                              0x0000000100e5cbf8 run_mod + 112
	23  Python                              0x0000000100e5a434 pyrun_file + 168
	24  Python                              0x0000000100e59d58 pyrun_simple_file + 276
	25  Python                              0x0000000100e59c04 PyRun_SimpleFileExFlags + 80
	26  Python                              0x0000000100e79d2c pymain_run_file + 320
	27  Python                              0x0000000100e7947c Py_RunMain + 916
	28  Python                              0x0000000100e7a6c4 pymain_main + 36
	29  Python                              0x0000000100e7a93c Py_BytesMain + 40
	30  dyld                                0x00000001007990fc start + 520
)
.

Thanks for reporting issue and for including the backtrace. I presume you used the Universal 2 installer, given that you are running an M1 mac? 

Kevin, do you have access to the macOS 12 beta to help test whether this is a Tkinter or Tk bug? (I assume the latter, as it is likely Apple have changed the API again)

Thanks for the report. I verified that the crash also occurs on an Intel Mac VM with Monterey beta 4 using the current python.org universal2 installers for 3.9.6 and 3.10.0rc1; they both have a private copy of Tk 8.6.11; so the problem does not appear to be Apple Silicon (M1) related. For what it's worth, the legacy python.org 3.9.6 Intel-only installer, which uses a copy of Tk 8.6.8, does not crash the Monterey beta 4. And no such failures are observed using any of those installers on the current Big Sur 11.5.1 release.

I used brew.sh to install Python, which I think uses the universal installer (although I can’t check, reinstalling macOS after hours of failing to downgrade back to Big Sur). I can’t remember the exact error, but it seemed to be from Apple’s end since it complained about not having access to the save/open dialog box. After I reinstall macOS, if you need further testing, i’m open to trying.
On Aug 4, 2021, 2:53 PM -0400, E. Paine <report@bugs.python.org>, wrote:
>
> E. Paine <xepaine13@gmail.com> added the comment:
>
> Thanks for reporting issue and for including the backtrace. I presume you used the Universal 2 installer, given that you are running an M1 mac?
>
> Kevin, do you have access to the macOS 12 beta to help test whether this is a Tkinter or Tk bug? (I assume the latter, as it is likely Apple have changed the API again)
>
> ----------
> components: +macOS
> nosy: +epaine, ned.deily, ronaldoussoren, serhiy.storchaka, wordtech
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue44828>
> _______________________________________

> I used brew.sh to install Python, which I think uses the universal installer

As far as I know, Homebrew builds its own Python and Tk.

> After I reinstall macOS, if you need further testing, i’m open to trying.

Thanks. I think the next step would be to try to reproduce just using Tcl and Tk as this most likely is an issue in Tk rather than Python.  It's also quite possible that the underlying problem will be fixed in a future Monterey beta. Alas, I don't have time to look further into this in the immediate future.  Perhaps someone can follow up directly with the Tk folks.

I built Tcl and Tk 8.6 on Monterey beta (21A5294g) and I see this
traceback in the Wish file dialog demo.  Note that this is *not* an
error.  The file dialog works fine.  This is a non-fatal
NSInternalInconsistencyException which prints a traceback to stderr.
It occurs within a call to [NSSavePanel runModal] which is entirely
Apple code.  The runModal method is not deprecated.  Most likely, when
Apple releases Monterey the assert command will have been removed,
possibly replaced by a call to NSLog which writes information into a
log file rather than print it to stderr.  (NSLog itself prints to
stderr when debugging is enabled.)  Of course macOS apps run without
stderr, so you will only see this traceback when running a python
program from the terminal.  (Any linux user who runs linux apps from
the terminal is used to seeing lots of stderr messages in stderr.  Try
running okular from a terminal sometime.  So we shouldn't complain too
much.)

Anyway, this looks harmless and the messages will almost surely go
away in the production version.  However, there are several new
deprecations related to specifying preferred file types in a file
dialog which I will address in Tk before Monterey is released.

Beta releases are betas, even for macOS.

- Marc

On Thu, Aug 5, 2021 at 1:23 PM Ned Deily <report@bugs.python.org> wrote:
>
>
> Ned Deily <nad@python.org> added the comment:
>
> > I used brew.sh to install Python, which I think uses the universal installer
>
> As far as I know, Homebrew builds its own Python and Tk.
>
> > After I reinstall macOS, if you need further testing, i’m open to trying.
>
> Thanks. I think the next step would be to try to reproduce just using Tcl and Tk as this most likely is an issue in Tk rather than Python.  It's also quite possible that the underlying problem will be fixed in a future Monterey beta. Alas, I don't have time to look further into this in the immediate future.  Perhaps someone can follow up directly with the Tk folks.
>
> ----------
> nosy: +culler
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue44828>
> _______________________________________

I should have mentioned that I tested on Intel hardware, not M1.  I do not have access to an M1 Apple at this time.

Thanks for looking into this, Marc.

I am also experiencing this issue.

I have an M1 Macbook Air running MaOS Monterey Version 12.0 Beta (21A5304g) with a fresh install of Python 3.9.6. 

When attempting to save or open anything from either the editor or console window I get the following error message "The Save file operation failed. The save file operation failed to connect to the open and save panel service.”

I have granted the IDEL full drive access which hasn't resolved the issue. The dialog prevents any savings from occurring making the IDEL useless. 

I have submitted this issue through Apple's Feedback tool as well.

Can also confirm this issue, tested on M1 and Intel Mac and both has the same error when using Monterey 12.0 Beta 21A5506j. 

On the same computer, using Big Sur, saves the file successfully and no error present. Seems like a Monterey issue, not dependent on specific hardware. (Intel/M1)

Tested on fresh Python 3.9.6 (tk version 8.6) Installation on Big Sur and Monterey Beta (Build 21A5506j) on a M1 Macbook Pro and Early 2015 13" Macbook Pro.

Same issue. Running Monterey beta. Experiencing crash.

Is there any plan for a fix or is the plan to wait for Monterey to release?

Unfortunately, I am still seeing this failure in Monterey beta 9.  However, we are no longer alone.  Here is a report of the same issue in Android Studio:

https://stackoverflow.com/questions/69068842/android-studio-open-file-operation-failed-the-open-file-operation-failed-to-co

This is reportedly fixed in the Adroid Studio beta, so maybe someone will be able to figure out what Google did to workaround the problem.

I was able to fix this problem for Tk on Monterey beta [21A5543b].
The fix has been committed to the tip of the core-8-6-branch in the
Tk fossil repository.

Here is a synopsis.  Tk used to open the file dialog by calling
[NSApp runModalForWindow:panel].  Starting with the release of 10.14
this call would produce a warning message on stderr saying that
[NSOpenPanel runModal] should be used instead.  But on systems older
than 10.14, the runModal method would fail because the completion handler
would not get called.  Now, with 12.0 (at least in the beta) calling
[NSOpenPanel runModal] produces an error dialog saying "The open file
operation failed to connect to the open and save panel service" and
this would be accompanied by a traceback printed on stderr for an
assertion error.  (It was not a "crash" but the file selection would
fail.)  However, it turns out that calling [NSApp runModalForWindow:panel]
no longer produces the warning in 12.0, and it works correctly.

So my workaround was to add conditional code that only calls the runModal
method on 10.14 and 10.15 and calls runModalForWindow on other versions
of macOS.  I tested on 10.15 and it works there as well.

Needless to say, none of these changes are documented by Apple.

Marc, thanks for the update.

Will there be a Tcl/Tk release soonish that includes this fix? 

Alternatively, is there patch that we can apply to the latest release when building the copy of Tk included with the installers?

Hi Ronald,
There is no calendar scheduling for Tk releases.  Don Porter decides when they happen.  But I think we are due for another one soonish.  In case it doesn't happen before the next Python release I will attach the patch file for commit a32262e9.  (Note that this is subject to change - I still need to test on 10.14.) I believe that this patch is completely self-contained and can be applied to the 8.6.11 release.

(Hint: Tk uses fossil as its SCM system.  On a fossil timeline, clicking any two nodes generates their diff.  The Tk timeline is at https://core.tcl-lang.org/tk/timeline)

Please help me.  Am new to Python, and don't know enough to post here, but I will try.  Have written a couple of programs that use tkinter, especially tkinter.filedialog.askopenfilenames, and as everyone else mine has quit working since Monterey. I have a macOS Monterey version beta        ( 21A5543b ).  Am using Python3 v3.10.0.  Have tried using v3.11.0a1, but could not even compile, says that import ( PIL ) does not exist.  Put back v 3.10.0 and am back to no filedialog.  Is Apple attempting to do away with the file API.  Just asking. Thank you.

No, Apple is not going to do away with their NSOpenPanel.  There is always some churn when they release a new OS.  Subtle changes to APIs can occur with no warning and no documentation.  Sometimes they are bugs. Sometimes they disappear when the OS is released.  Sometimes they are permanent.  I would not recommend using a beta version of the OS to develop your tkinter app.

Thank you very
Guy DeStefano

On Mon, Oct 11, 2021 at 2:10 PM Marc Culler <report@bugs.python.org> wrote:

>
> Marc Culler <marc.culler@gmail.com> added the comment:
>
> No, Apple is not going to do away with their NSOpenPanel.  There is always
> some churn when they release a new OS.  Subtle changes to APIs can occur
> with no warning and no documentation.  Sometimes they are bugs. Sometimes
> they disappear when the OS is released.  Sometimes they are permanent.  I
> would not recommend using a beta version of the OS to develop your tkinter
> app.
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue44828>
> _______________________________________
>

Thank you very much for the reply. Sorry for previous text.
Guy DeStefano

On Mon, Oct 11, 2021 at 2:10 PM Marc Culler <report@bugs.python.org> wrote:

>
> Marc Culler <marc.culler@gmail.com> added the comment:
>
> No, Apple is not going to do away with their NSOpenPanel.  There is always
> some churn when they release a new OS.  Subtle changes to APIs can occur
> with no warning and no documentation.  Sometimes they are bugs. Sometimes
> they disappear when the OS is released.  Sometimes they are permanent.  I
> would not recommend using a beta version of the OS to develop your tkinter
> app.
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue44828>
> _______________________________________
>

@Guy, thanks for your interest but in the future please don't use the issue tracker as a help forum. There are lots of places to ask about such matters; https://www.python.org/about/help/ has a good list of resources and, among the Python-specific mailing lists at https://www.python.org/about/help/, there are ones specifically for Python usage on macOS (https://mail.python.org/mailman/listinfo/pythonmac-sig) and Tkinter usage (https://mail.python.org/mailman/listinfo/tkinter-discuss).

I appreciate the information, In the future I will do as is stated.  Thanks
for the reply.
Guy DeStefano

On Mon, Oct 11, 2021 at 2:24 PM Ned Deily <report@bugs.python.org> wrote:

>
> Ned Deily <nad@python.org> added the comment:
>
> @Guy, thanks for your interest but in the future please don't use the
> issue tracker as a help forum. There are lots of places to ask about such
> matters; https://www.python.org/about/help/ has a good list of resources
> and, among the Python-specific mailing lists at
> https://www.python.org/about/help/, there are ones specifically for
> Python usage on macOS (
> https://mail.python.org/mailman/listinfo/pythonmac-sig) and Tkinter usage
> (https://mail.python.org/mailman/listinfo/tkinter-discuss).
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue44828>
> _______________________________________
>

This issue is happening for me in with my installation of python 3.10 running the release version of mac os monterey 12.0.1 running on an intel mac, I noticed this behavior trying to open or save files in IDLE.

Marc, thanks for providing the patch for Tk. There are some issues with it, though, at least when used with IDLE. I built a Python 3.10.0+ universal2 installer, like what we provide on python.org, with just the updated Tk and tested it on macOS 10.9, 10.13, 10.14, 10.15, 11 and 12.

First, and most important, there seems to be a typo in the fix that causes the dialog panel to break on macOS 10.14: 

-	if ([NSApp macOSVersion] > 101400) {
+	if ( osVersion >= 101400 && osVersion < 120000) {

That ">=" should be just an ">" as before, I think. At least, making that change unbreaks 10.14.

Second, while the filedialog panel now works on macOS 12 (Monterey) with the patch, there are some side effects, at least when using CMD-S (Save) in a new IDLE edit window. After pressing CMD-S, some small object appears on the screen very briefly (too quickly for me to recognize what it is) and then the expected Save filedialog panel appears; however, unlike on all the other operating system levels tested, the keyboard focus is not on the filename text field in the panel. So when the user starts typing the file name, nothing happens until they do something, like clicking on the file name field, to manually move the keyboard focus to that field. I think most users would find that confusing. I don't know if there's any way to change that, either in Tk, tkinter, or IDLE. But it does seem to be a regression. The same binaries work fine on all of the other macOS versions I tested.

Any ideas?

(Although it's not an IDLE bug, I've added IDLE to the components list since it is impacted by this issue.)

Thanks, Ned, for finding my mistake which you generously called a typo.
I have fixed the inequality in the Tk fossil repository.  Incidentally, there is now a core-8-6-12-rc branch of Tk, also containing the fix.  So it should not be too long before 8.6.12 is released.

I will look at the code and check whether I see the UFO that you report.  As you know, all that Tk does is to open an NSOpenPanel.  Once that is open it is completely handled by the OS.  Tk sees no events from the modal interaction and does not have any way of controlling any aspect of the dialog.  So the only hope would be that it is possible to set some configuration option before opening the NSOpenPanel which would somehow cause the OS to assign focus to the correct widget.  I will try to investigate what options are available.

Hi Ned,  I think this problem is fixed now in the tip of the Tk macosx_filedialog branch.  I am attaching the tkMacOSXDialog.c file from that branch.  I think you should be able to just replace the version in 8.6.11 and be able to build a working version.  I have tested on macOS 10.14, 10.15, 11 and 12.  Can you please test on your systems?

If it looks OK I will merge the changes into 8.6.12-rc so they will appear in the 8.6.12 release when that happens.  It should be fairly soon.

Thanks for the quick response, Marc. The results of testing were mixed. The good news: the new version did seem to solve the Monterey problem of the "UFO" and the loss of keyboard focus. However, the new version causes new problems on other releases.

- On 10.14 and earlier systems, I observed the following new behavior in IDLE: on doing a Save from an Edit Window (either with CMD-S or click on the Save menu item), the Save file pane appears as expected but, after entering the file name and clicking Save on the pane, the Save file pane is not dismissed although the file is actually saved under the expected file name. Clicking on the pane's Save button again brings up a "file already exists, do you want to overwrite" message and after clicking OK to that, both the "overwrite" pane and the underlying Save pane are dismissed.

- On 10.15 only, I stumbled across this behavior (and verified it did not occur with the unpatched 8.6.11 nor does it seem to occur anywhere but on 10.15): in a new IDLE edit window, type some text, then Save to a file (CMD-S). Now alter the text and then use Save As (CMD-SHIFT-S) to attempt to save to another file name. Upon entering CMD-SHIFT-S (or clicking on the Save As... menu item), Tk crashes. I've attached a few crash dumps.

Thanks for doing all the testing, Ned.  I guess that the path forward is now clear.  I will revert to the 8.6.11 code for 10.15 and earlier and use the new code for 11 and later.  Of course the 8.6.11 code already has two cases, for 10.14 and earlier and 10.15 and later.  So I will have some code cleanup to do.

The UFO, by the way, was a standalone file dialog which was first being rendered as a separate window and then immediately attached to the parent window as a sheet.  This was easier to see on a slower machine, like a VMWare VM which does not have graphics acceleration.

Evidently the big change for 10.15 was that the file dialog was changed so that it became managed by a separate process, rather than just borrowing the current process's event queue.  The reason, supposedly, was to make it so that an app running in a sandbox could still read and write the user's files.  It seems that Apple had a hard time making this work, and the UFO as well as the assertion error suggest to me that they still haven't figured it out.  So I thnk we can expect more trouble with the next OS.  At least we have a year before that adventure starts.

Needless to say, having some documentation from Apple would be much more pleasant than the trial and error method that we are forced to use now.  But we know better than to expect that.

Hmmm, the 10.15 segfault seems to occur when writing the filename into the entry widget on the dialog.  So maybe it is actually an issue with reference counting an NSString.  I will have to look at that more carefully.

Hi Ned.  Here is one more attempt, hopefully the final one.  I tested with IDLE on 10.14 (VM), 10.15(hard), 11(hard), 12(VM).  I used Python 3.10.0.  I replaced libtk8.6.dylib with the Tk lib compiled from the tip of the macosx_filedialog branch.  I could not reproduce any of the issues that you reported.  (But I hope you will retry as well, since just replacing the tkMacOSXDialog.c file in 8.6.11 is not the same as using the soon-to-be 8.6.12 release candidate.)

I will attach the tkMacOSXDialog.c that I used.

> Here is one more attempt, hopefully the final one.

I think we have a winner! The issues I've seen all seem to be resolved in this latest round. Thank you, Marc! For the record, my testing for this has been very rudimentary manual testing so I'm certainly not saying there aren't other problems that we haven't run into yet but I'm reasonably confident that, with this Tk fix applied, IDLE behavior with python.org Pythons should be similar on macOS 12 Monterey to macOS 11 Big Sur.

I'm going to merge this Tk patch into our macOS installer build scripts and plan to release it in 3.9.8 which is scheduled for next week. And I'm working with the rest of the release team on how best to provide this fix for 3.10 since 3.10.1 is not planned for another month. I'll also try to test with the Tk 8.6.12 branch soon.

That is great news!  I will now merge the changes into the Tk core-8-6-branch and core-8-6-12-rc branches.

New changeset be8318be05e1a874215fa75b8845ede74b2c69b6 by Ned Deily in branch 'main':
bpo-44828: Avoid tkinter file dialog failure on macOS 12 Monterey (GH-29276)
https://github.com/python/cpython/commit/be8318be05e1a874215fa75b8845ede74b2c69b6

New changeset 54579087c69f95531cbe7a97401c67f104a3e52f by Miss Islington (bot) in branch '3.10':
bpo-44828: Avoid tkinter file dialog failure on macOS 12 Monterey (GH-29276)
https://github.com/python/cpython/commit/54579087c69f95531cbe7a97401c67f104a3e52f

New changeset 8e5e74e3049875e9d834fe4408263676fe21e890 by Miss Islington (bot) in branch '3.9':
bpo-44828: Avoid tkinter file dialog failure on macOS 12 Monterey (GH-29276)
https://github.com/python/cpython/commit/8e5e74e3049875e9d834fe4408263676fe21e890

New changeset f19c1a115f782036edac306de0f3f9968c1e1fd6 by Miss Islington (bot) in branch '3.8':
bpo-44828: Avoid tkinter file dialog failure on macOS 12 Monterey (GH-29276) (GH-29279)
https://github.com/python/cpython/commit/f19c1a115f782036edac306de0f3f9968c1e1fd6