Skip to content

let examples have a chance to exit normally#1605

Merged
j9ac9k merged 5 commits intopyqtgraph:masterfrom
pijyoi:clean_shutdown
Mar 7, 2021
Merged

let examples have a chance to exit normally#1605
j9ac9k merged 5 commits intopyqtgraph:masterfrom
pijyoi:clean_shutdown

Conversation

@pijyoi
Copy link
Copy Markdown
Contributor

@pijyoi pijyoi commented Feb 23, 2021

PR to demonstrate segfaults occurring for many examples if they are allowed to end normally.

@pijyoi
Copy link
Copy Markdown
Contributor Author

pijyoi commented Feb 23, 2021

So ColorButton.py seems to be fixed by removing the reference to the child widget from the main script.
https://doc.qt.io/qt-5/qmainwindow.html#setCentralWidget notes: QMainWindow takes ownership of the widget pointer and deletes it at the appropriate time.

@pijyoi
Copy link
Copy Markdown
Contributor Author

pijyoi commented Feb 23, 2021

Segfault in JoystickButton.py is fixed by wrapping things up into a class. So at least one class of segfaults could be fixed in a similar way.

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Feb 23, 2021

the macOS Failure...

Python 3.8.7 (default, Jan 25 2021, 16:16:23) 
[Clang 11.0.0 (clang-1100.0.33.17)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> >>> ... ... ... ... ... ... ... ... ... ... ... ... ... ... Unable to create basic Accelerated OpenGL renderer.
Unable to create basic Accelerated OpenGL renderer.
Core Image is now using the software OpenGL renderer. This will be slow.

I've seen this periodically, I have no clue how to fix this. One potential action is to migrate our macOS CI platform to Big Sur (macOS 11) from Catalina (10.15); there may be other downsides with that, and that does not ensure this issue won't be resolved.

Thanks for tackling this @pijyoi Please let me know if there is a specific test you'd like me to try and patch up. I have access to macOS and Windows platforms, I can get access to an ubuntu platform too if needed.

@pijyoi
Copy link
Copy Markdown
Contributor Author

pijyoi commented Feb 24, 2021

Console.py creates a cyclic self reference, preventing cleanup. Was introduced in 8aec44d as a feature, so this fix would break that usage.

Thanks for tackling this @pijyoi Please let me know if there is a specific test you'd like me to try and patch up. I have access to macOS and Windows platforms, I can get access to an ubuntu platform too if needed.

It's unlikely that I would be able to see this through. Perhaps create a "needs help" issue?
Rewriting the examples that segfault would be a huge effort and it's likely that some bugs or change in behavior would be introduced, particularly if the functionality involved isn't even used by the developer in their own programs.

From what I can see,

  1. the segfaults affect Python 3.8, PySide2, Windows and Linux.
  2. PyQt doesn't segfault, probably because of this mitigation: https://www.riverbankcomputing.com/static/Docs/PyQt5/gotchas.html#crashes-on-exit
  3. all the 3D Graphics examples segfault.
  4. Linux has a lot more segfaults
    • The Console.py commit fixes the segfault for Windows but not for Linux, so some other thing is going on for Linux

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Feb 24, 2021 via email

@pijyoi
Copy link
Copy Markdown
Contributor Author

pijyoi commented Feb 25, 2021

The segfaults were git bisected to a change made in #1302. Inheriting from QApplication, that is.
You need a new-ish Python to trigger this segfault.
Using Ubuntu 20.04 system Python 3.8.5 will not trigger the segfaults. The CI uses Python 3.8.7 and does trigger the segfaults.

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Feb 25, 2021

Gotta love it when a PR you created/merged causes breakage that's discovered later...

Is it just the inheritance that's causing the segfault, or is it something we're doing within that inheritance that's driving it.

For example, is there still a segfault if we just have?

class App(QtGui.QApplication):

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

I can look into this later too, just might take me a day or two... Thanks for the tip with checking for newer versions of python.

@pijyoi
Copy link
Copy Markdown
Contributor Author

pijyoi commented Feb 26, 2021

For example, is there still a segfault if we just have?

class App(QtGui.QApplication):
   def __init__(self, *args, **kwargs):
       super().__init__(*args, **kwargs)

I didn't try it but it seems unnecessary to sub-class QApplication which is a singleton. In my last commit, I connected the signal to a standalone function.

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Feb 26, 2021

I didn't try it but it seems unnecessary to sub-class QApplication which is a singleton. In my last commit, I connected the signal to a standalone function.

🤯 ...it didn't dawn on me that QApplication is indeed just a singleton and thus shouldn't need to subclass; ...you're absolutely right.

@pijyoi pijyoi marked this pull request as ready for review February 26, 2021 06:17
@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Feb 26, 2021

That deduplication commit... why didn't I think of that when I wrote it, ...instead of copy-pasting....

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Feb 26, 2021

@pijyoi this PR looks great to me. I'll give time for some other folks to look at this before I merge; but this should make a significant improvement to the reliability ... the QTimer.singleShot(1000, pg.Qt.QtWidgets.QApplication.quit) is really slick.

@pijyoi
Copy link
Copy Markdown
Contributor Author

pijyoi commented Feb 26, 2021

Some segfaults now on macOS PyQt: ScatterPlot.py, VideoSpeedTest.py, ViewBoxFeatures.py
Not altogether random, I have seen it happen before a few times for the last two.

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Feb 26, 2021

The ViewboxFeatures.py failures on macOS I'm fairly sure are unrelated to this PR; but some CI shenanigans.

>>> >>> ... ... ... ... ... ... ... ... ... ... ... ... ... Unable to create basic Accelerated OpenGL renderer.
Unable to create basic Accelerated OpenGL renderer.
Core Image is now using the software OpenGL renderer. This will be slow.

I haven't figured out how to handle this issue and despite seeing the same error in other repositories, I have yet to see a fix. Only idea I have is rolling our macOS test image from 10.15, to 11.0. Another option is to run pytest -n 2 instead of -n auto; but no guarantee that will work.

I'll try and investigate the issue on Windows in a bit.

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Feb 26, 2021

Can't replicate the windows failure either; that's annoying... for the fun of it, I'll re-run the CI and see what happens.

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Feb 26, 2021

Well on rerun, everything passes.... I feel so conflicted about this.

EDIT: will re-run again momentarily

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Feb 27, 2021

I'm going to re-run the CI pipeline a few times to see if we can get a list of items that are segfaulting.

I am suspicious that the items that are segfaulting have QTimer's that are connected to an update method (not sure what the best way to address this if this is indeed the issue).

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Feb 27, 2021

I've re-ran the CI suite almost 10 times now... The failures I've seen are:

  • VideoSpeedTest.py x2 (both on Windows - PyQt5 5.15/Python 3.8)
  • PanningPlot.py (Windows - PyQt5 5.15/Python 3.8)
  • test_interpolateArray_order0 worked crashed during this test
  • ViewBoxFeatures.py macOS PyQt5 5.15/Python 38 and Windows - PyQt6 - Python 3.9
  • ScatterPlot Ubuntu PyQt6/Python 3.9
  • ScrollingPlots.py - macOS PySide6/Python 3.9
  • test_ref_cycles.py::test_ImageView xdist worker crashed (also got weird exception from the coverage tool)

@pijyoi I would recommend not installing pytest-xdist and pytest-cov, removing the -n auto, and --cov calls to minimize other factors that might be influencing these tests. Let me know if you'd like me to make these changes on your branch.

@pijyoi
Copy link
Copy Markdown
Contributor Author

pijyoi commented Feb 27, 2021 via email

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Feb 28, 2021

I branched off of your branch, and am testing some stuff out on my fork; I'm developing confidence that I've fixed the segfault with VideoSpeedTest.py by adding the following method:

    def closeEvent(self, event):
        self.timer.disconnect(self.update)
        return super().closeEvent(event)

Edit: when re-enabling this method, appears that the .disconnect() method is really picky about the arguments it takes; periodically complaining about me giving it the wrong argument types; regardless of what combination I do it in...

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Feb 28, 2021

I think I may have fixed up example/ScatterPlot.py but I think the "fix" is a workaround for a potential bug in pyqtgraph...

The example hasn't failed on me since I made this change.

w1.setParent(view)
w2.setParent(view)
w3.setParent(view)
w4.setParent(view)
s1.setParent(view)
s2.setParent(view)
s3.setParent(view)
s4.setParent(view)

Turns out, that view.addPlot does not set the parent of the newly created plot. Before those lines, if you query w1.parent() , you would get None which is likely not what we want if we want a clean tear-down.

EDIT: I suspect this same issue is what occasionally makes ViewBoxFeatures.py segfault as well.

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Feb 28, 2021

What I don't know how to do easily is disconnect the timer on examples like scrollingPlots.py when the window is closing, without subclassing...

I suppose we could add a signal (sigClosing) to GraphicsWidget that is emitted when the close event is received.

EDIT: I'm now experimenting with setting the parent object of the timer to the graphics window we're displaying, hopefully that will work.

@pijyoi
Copy link
Copy Markdown
Contributor Author

pijyoi commented Feb 28, 2021

The following line from https://doc.qt.io/qt-5/qtimer.html makes me uneasy.
"As a special case, a QTimer with a timeout of 0 will time out as soon as possible, though the ordering between zero timers and other sources of events is unspecified"

It's like saying that it's possible for your idle timer to fire after some gui objects have been torn down.

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Feb 28, 2021

The following line from https://doc.qt.io/qt-5/qtimer.html makes me uneasy.
"As a special case, a QTimer with a timeout of 0 will time out as soon as possible, though the ordering between zero timers and other sources of events is unspecified"

It's like saying that it's possible for your idle timer to fire after some gui objects have been torn down.

I suspect that's exactly what it's saying. I suppose we should change our all our instances of QTimer(0) to QTimer(1); given it's in ms I think that should still be fast enough.

@pijyoi
Copy link
Copy Markdown
Contributor Author

pijyoi commented Feb 28, 2021

https://wiki.qt.io/Timers#Windows
Windows implementation is quite different from the others.

@pijyoi
Copy link
Copy Markdown
Contributor Author

pijyoi commented Mar 1, 2021

Would the fix here to modify examples such that they have a method that is run after import that starts things off?

The keyword here being "method", i.e. nothing related to Qt instantiated in module namespace. So you would need to wrap everything up in a class.

Alternatively, I was trying to cause destruction of the imported module with the del sys.modules['{1}'] line in test_examples.py, but I don't think it's working.

In the following pair of scripts, __del__ doesn't seem to get executed. So if someone knows how to do it, please chime in.
Update: The following will trigger __del__ but requires a gc.collect()

# file MyClass.py
class MyClass:
    def __init__(self):
        pass

    def __del__(self):
        print('__del__')

win = MyClass()
# file import_MyClass.py
import sys
import gc
import MyClass

print('A')
del sys.modules['MyClass']
del MyClass
gc.collect()
print('B')

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Mar 1, 2021

You may need to also do a call the collect function of th garbage collector, but that has other unfortunate side effects.

@pijyoi
Copy link
Copy Markdown
Contributor Author

pijyoi commented Mar 1, 2021

the latest commit causes failures for PyQt Linux and macOS specifically for the 2 examples that make use of QTimer.singleShot()

  • ImageItem
  • PColorMeshItem

This seems to show that PyQt binding executes the event loop upon application shutdown.

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Mar 1, 2021

hi @pijyoi thanks so much for sticking with this... I'm trying to follow along, and when you think you have this in a "ready to merge" state, I'll likely have more questions.

The one thing that sticks out at me about the diff as it stands right now are these lines:

if hasattr({1}, 'MainWindow'):
    pg.mkQApp()
    win = {1}.MainWindow()

Is there something special about QMainWindow class vs. a generic QWidget here?

@pijyoi
Copy link
Copy Markdown
Contributor Author

pijyoi commented Mar 1, 2021

Is there something special about QMainWindow class vs. a generic QWidget here?

the idea was that examples that have sub-classed QMainWindow are likely those which have wrapped everything up nicely and do not instantiate things on import.

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Mar 1, 2021

Is there something special about QMainWindow class vs. a generic QWidget here?

the idea was that examples that have sub-classed QMainWindow are likely those which have wrapped everything up nicely and do not instantiate things on import.

For the record, I have no problem modifying the examples so they are all bundled into encompassing classes, especially as that is how Qt is generally used anyway (but I fully recognize that is well outside the scope of this PR).

@pijyoi
Copy link
Copy Markdown
Contributor Author

pijyoi commented Mar 1, 2021

To summarize, there are 2 main fixes going on here:

  1. Many examples segfault on PySide2 5.15 / Python 3.8.7 when QApplication was sub-classed
    o this is visible by the user running the examples
  2. Examples were tested by importing, which makes their order of destruction unpredictable. PyQt is affected more by this.
    o this is only visible by running pytest, although not producible locally.

I have verified that the fix for (2) does not fix (1)

cleanup imported module

1) instantiate MainWindow class if present
2) delete attributes from imported module when done
@pijyoi
Copy link
Copy Markdown
Contributor Author

pijyoi commented Mar 1, 2021

To not detract from the root issues described above, I have dropped from this PR the commits that change some examples to QMainWindow classes.
The changes made to VideoSpeedTest.py can be resubmitted in a separate PR.

when you think you have this in a "ready to merge" state, I'll likely have more questions.

Should be ready for review. The "Disable coverage and xdist for troubleshooting" commit can be dropped later.

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Mar 1, 2021

@pijyoi This PR looks good to me; thank you so much for tracking these issues down. I don't know of any of the maintainers that think we have some great knowledge in the lifecycle of QObjects, so I'm inclined to say functionally things should remain as they are (minus my PR altering main.yml). Only changes I would request is adding a comments next to

timer.setSingleShot(True) 
# not using `timer.singleShot() because of persistence see PR #1605

and

App = QtWidgets.QApplication
# subclassing QApplication causes segfaults on python 3.8.7+

(or something to that effect).

I would remove my commit from your branch (it was put there to make tests more deterministic), but I have a reluctance to force-push on someone else's branch, so I'll leave that to you.

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Mar 1, 2021

I think this is ready to merge.

@ixjlyons @outofculture @ksunden you all have any other input?

@pijyoi
Copy link
Copy Markdown
Contributor Author

pijyoi commented Mar 2, 2021

I just saw this on a separate fork, macOS / Python 3.8 / PyQt5.15
Screenshot at 2021-03-02 17-00-57
Again, it looks like PyQt5 is processing events after app.exec_() has returned and after we have deleted the namespace variables (including in this case QtCore).

Even prior to the commits in this PR, I had noticed something similar on the CI only for macOS. This PR only confirms that it happens.

@pijyoi
Copy link
Copy Markdown
Contributor Author

pijyoi commented Mar 2, 2021

I found the other case I mentioned, macOS / Python 3.9 / PyQt6
https://github.com/pijyoi/pyqtgraph/runs/1853387168?check_suite_focus=true
Screenshot at 2021-03-02 17-47-40
In this case, it is a QDialog rejected callback that is being executed even though nobody could have been clicking "cancel"...

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Mar 2, 2021

I have seen output like this that got printed after the test results summary was shown on the console, was a bit confused as to what was going on. I wonder if this is a case of not having a good QObject parent <-> child relationship chain and things aren't closing out as they should on exit.

In the cases I saw it, it didn't actually show any test failures...

@pijyoi
Copy link
Copy Markdown
Contributor Author

pijyoi commented Mar 3, 2021

Would it be preferable to split this PR into 2? The first for the user-visible segfaults and the second for the CI-only segfaults (which are triggered by trying to let the examples end normally)

The first should fix #1599 (comment) (Could you also verify that?)

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Mar 3, 2021

I'm fine keeping the PR together; I'll verify this fixes the issue I commented on before merging.

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Mar 7, 2021

Meant to merge this earlier this week, apologies for the wait. Thanks so much for this @pijyoi !

@j9ac9k j9ac9k merged commit c5a1174 into pyqtgraph:master Mar 7, 2021
@pijyoi pijyoi deleted the clean_shutdown branch March 9, 2021 02:53
@j9ac9k j9ac9k mentioned this pull request Jul 19, 2021
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants