Skip to content

Remove workaround for memory leak in QImage#1223

Merged
j9ac9k merged 1 commit intopyqtgraph:developfrom
j9ac9k:fix-724-remove-qimage-workaround
Jun 6, 2020
Merged

Remove workaround for memory leak in QImage#1223
j9ac9k merged 1 commit intopyqtgraph:developfrom
j9ac9k:fix-724-remove-qimage-workaround

Conversation

@j9ac9k
Copy link
Copy Markdown
Member

@j9ac9k j9ac9k commented Jun 4, 2020

Fixes #724 - PySide2 < 5.12 should not be considered stable, and due to its age, pyqtgraph should not try and compensate with it by embedding workarounds.

@j9ac9k j9ac9k requested a review from campagnola June 4, 2020 18:13
@2xB
Copy link
Copy Markdown
Contributor

2xB commented Jun 4, 2020

Could the if block also be deleted, leaving only the code from the current else block?

@j9ac9k
Copy link
Copy Markdown
Member Author

j9ac9k commented Jun 4, 2020

Could the if block also be deleted, leaving only the code from the current else block?

Suppose only one way to find out... let me make the change and see what the CI says....

@j9ac9k
Copy link
Copy Markdown
Member Author

j9ac9k commented Jun 4, 2020

@2xB looks like some failures are starting to trickle in through the CI, specifically on the PySide2 builds (I'll let it finish out to see if there is more of a pattern besides pyside1/2 fail, pyqt4/5 pass).

@pijyoi
Copy link
Copy Markdown
Contributor

pijyoi commented Jun 4, 2020

PySide2 doesn't support "QImage(imgData.ctypes.data, ...)"

Both PySide2 and PyQt5 support "QImage(imgData, ...)", i.e. just use the numpy buffer directly.

Don't know about PySide1 and PyQt4 though.

@j9ac9k
Copy link
Copy Markdown
Member Author

j9ac9k commented Jun 4, 2020

PySide2 doesn't support "QImage(imgData.ctypes.data, ...)"

Both PySide2 and PyQt5 support "QImage(imgData, ...)", i.e. just use the numpy buffer directly.

Don't know about PySide1 and PyQt4 though.

Thanks for pointing that out @pijyoi let's try it out!

@2xB
Copy link
Copy Markdown
Contributor

2xB commented Jun 4, 2020

Good to know!
The imgData.ctypes.data was introduced in adda8ae in which Luke writes:

This fixes memory leaks with PyQt 4.10 except when using makeQImage(copy=False).
Tested on 4.9.3 and 4.10.2; need to be tested against other versions.

Well - It would look even more compact otherwise.

This experimenting shows that the fix for old PyQt4 versions in the except clause is Python2 specific though:

>               img = QtGui.QImage(buffer(imgData), imgData.shape[1], imgData.shape[0], imgFormat)
E               NameError: name 'buffer' is not defined

@j9ac9k
Copy link
Copy Markdown
Member Author

j9ac9k commented Jun 4, 2020

can't kill Py2/Qt4 support soon enough...

@j9ac9k
Copy link
Copy Markdown
Member Author

j9ac9k commented Jun 4, 2020

Looks like this is failing for all PyQt4/5 versions (still CI runs are going). I think I'll revert back to how it was and leave it at that, or is there another change I should try?

@2xB
Copy link
Copy Markdown
Contributor

2xB commented Jun 4, 2020

Okay, so technically this works, but image comparison tests in PyQt fail though...

@j9ac9k
Copy link
Copy Markdown
Member Author

j9ac9k commented Jun 5, 2020

Okay, so technically this works, but image comparison tests in PyQt fail though...

Ahh, I should have looked at the nature of the failures... you caught me being lazy ... you think it's worth updating the images being compared?

@j9ac9k j9ac9k force-pushed the fix-724-remove-qimage-workaround branch from 361edce to b0f148d Compare June 5, 2020 00:08
@j9ac9k
Copy link
Copy Markdown
Member Author

j9ac9k commented Jun 5, 2020

have to call it a night; I reverted to the state where CI passed across the board.

@j9ac9k
Copy link
Copy Markdown
Member Author

j9ac9k commented Jun 6, 2020

I'm going to merge this as it stands, I suspect after we limit the number of supported libraries we're going to be taking a closer look at all these workarounds for the different bindings.

@j9ac9k j9ac9k merged commit d282f8a into pyqtgraph:develop Jun 6, 2020
@2xB
Copy link
Copy Markdown
Contributor

2xB commented Jun 6, 2020

I am sorry for somehow missing your question. I think your plan sounds good.
Also, I think this shows that we should start to require notes like # Using imgData.ctypes.data needed to fix memory leak in 4.10 for makeQImage(copy=True) in the source code for obscure fixes, having this information only in commit messages is obviously impractical.

@j9ac9k
Copy link
Copy Markdown
Member Author

j9ac9k commented Jun 6, 2020

@2xB I think that's a great idea (making better use of comments to describe workarounds). I'll try and be better about it in the future.

@j9ac9k j9ac9k deleted the fix-724-remove-qimage-workaround branch June 6, 2020 18:36
@pijyoi
Copy link
Copy Markdown
Contributor

pijyoi commented Jan 15, 2021

Both PySide2 and PyQt5 support "QImage(imgData, ...)", i.e. just use the numpy buffer directly.

I found some old comments in my code regarding PyQt5:
"If QImage is created with a python object (including ndarray and ctypes object), then it will make a copy of the data when you call setColorTable()"

So looks like PySide and PyQt codepaths can't be safely unified here as there may be some unknown/unwanted side effects.

@j9ac9k
Copy link
Copy Markdown
Member Author

j9ac9k commented Jan 28, 2021

Both PySide2 and PyQt5 support "QImage(imgData, ...)", i.e. just use the numpy buffer directly.

I found some old comments in my code regarding PyQt5:
"If QImage is created with a python object (including ndarray and ctypes object), then it will make a copy of the data when you call setColorTable()"

So looks like PySide and PyQt codepaths can't be safely unified here as there may be some unknown/unwanted side effects.

Do you think it would be worth testing again to see if this is an issue with Qt6 (or PyQt5 5.15)?

@pijyoi
Copy link
Copy Markdown
Contributor

pijyoi commented Jan 28, 2021

Do you think it would be worth testing again to see if this is an issue with Qt6 (or PyQt5 5.15)?

I retested PyQt5 5.15 while adding PyQt6 support and documented it more thoroughly in the comments.
https://github.com/pyqtgraph/pyqtgraph/blob/master/pyqtgraph/functions.py#L1259-L1279

In short, PyQt5 5.15 needs its own special case.

The copying effect in PyQt5 can be demonstrated in the following code..
https://github.com/pyqtgraph/pyqtgraph/blob/master/pyqtgraph/widgets/RemoteGraphicsView.py#L230-L241
If you modify the PyQt5 code to be the same as PyQt6, no plot will result. i.e. calling (non-const) QImage::fill() triggers the copy-on-write mechanism and further renders are to this new memory rather than to the shared memory.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PySide QImage workaround should only be used where bug is present

3 participants