Skip to content

Remove references to self from lambdas#1598

Merged
j9ac9k merged 3 commits intopyqtgraph:masterfrom
outofculture:selfless-lambdas
Feb 23, 2021
Merged

Remove references to self from lambdas#1598
j9ac9k merged 3 commits intopyqtgraph:masterfrom
outofculture:selfless-lambdas

Conversation

@outofculture
Copy link
Copy Markdown
Contributor

Lambdas used as signal responders which reference self prevent GC of said object. Instance methods do not suffer the same problem. I wasn't sure if the QTimer one needed to be fixed, but it doesn't hurt to be safe.

I couldn't solve pyqtgraph.WidgetGroup.WidgetGroup.mkChangeCallback, though. That case can't be rewritten as an instance method, and it needs to remain associated with self, so it can't be a static function. Maybe it would be sufficient to add the appropriate disconnect somehow? But what would that look like? If anyone has ideas, I'm willing to take another stab at this one.

@pijyoi
Copy link
Copy Markdown
Contributor

pijyoi commented Feb 21, 2021

Could most of these new methods be replaced by a use of functools.partial?

For Parameters, a change in prototype of emitStateChanged

def emitStateChanged(self, param, data, changeDesc):
    ...

self.sigOptionsChanged.connect(partial(self.emitStateChanged, changeDesc='options'))

@pijyoi
Copy link
Copy Markdown
Contributor

pijyoi commented Feb 22, 2021

For mkChangeCallback, could the single call to self.mkChangeCallback(w) be converted to partial(self.widgetChanged, w=w)?

@outofculture
Copy link
Copy Markdown
Contributor Author

Okay, yeah, partials would be more elegant and solve the mkChangeCallback problem. I don't know if partials suffer from the same problem as lambdas; they might preserve the reference to self somehow. I was just working off of recommendations on stack overflow. I need a test that I can prove it one way or the other. I'll try to build that, and then use it to prove out whether partials would work.

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Feb 22, 2021

Okay, yeah, partials would be more elegant and solve the mkChangeCallback problem. I don't know if partials suffer from the same problem as lambdas; they might preserve the reference to self somehow. I was just working off of recommendations on stack overflow. I need a test that I can prove it one way or the other. I'll try to build that, and then use it to prove out whether partials would work.

Feel free to use PR #646 with a PyQt5/6 as an example.

Change:

        self.vb.sigLogChanged.connect(self._logChanged)

back to a lambda and test_ref_cycles will fail with this exact issue with C++ object already deleted...

@outofculture
Copy link
Copy Markdown
Contributor Author

@j9ac9k way to pick the most challenging to convert to a partial! 😁

Playing around with that, I was able to prove that partials still trigger the error. Thanks, everyone, for the ideas, but it looks like partials aren't going to be helpful here. It looks like PyQt* is doing extra work to make sure instance methods get disconnected appropriately. I could use functools.partialmethod to declare these one-liner methods, but we wouldn't be much better off for it, and it still doesn't help with mkChangeCallback.

@pijyoi
Copy link
Copy Markdown
Contributor

pijyoi commented Feb 22, 2021 via email

@outofculture
Copy link
Copy Markdown
Contributor Author

The first. The following fails the test: self.vb.sigLogChanged.connect(partial(self.setLogMode, x=1, y=1)) with the same failure as a lambda. I just hard-coded the args, but I made sure that hard-coding them in the _logChanged method didn't also produce the same error.

@pijyoi
Copy link
Copy Markdown
Contributor

pijyoi commented Feb 22, 2021

https://stackoverflow.com/questions/47941743/lifetime-of-object-in-lambda-connected-to-pyqtsignal/47945948#47945948

This here says that instance methods are treated specially. Mentions that lambda, partial and others don't get this special treatment. ☹

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Feb 22, 2021

@pijyoi thanks for sharing that link, the addition of "static methods" is something I would never have thought to be applicable. I am now taking a hard look at some of my projects involving pyqtgraph...

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Feb 22, 2021

2 Questions...

  1. Do we want to make some of these methods "private" by setting the _ before the method name?
  2. Not related to this PR, but is there a reason we don't use the @Slot decorator?

Regarding the private methods, I'm just bringing this up; I'm not advocating for a change to be made here...

@outofculture
Copy link
Copy Markdown
Contributor Author

Ooo, now that you mention it, I generally like private-ing methods that aren't useful as a public interface, yeah. This is a big library, and not having to worry about maintaining every little utility function is of great value to the maintainers. I'm changing it!

No opinion on the @Slot decorator. I'll read up on it, though.

@outofculture
Copy link
Copy Markdown
Contributor Author

Just a note that I looked at this some more (with help from @j9ac9k) and I failed to be able to disconnect lambdas as part of object deletion. self.destroyed.connect(self._onDestroy) does not ever call self._onDestroy(), at least not in the #646 test I was using. I even tried putting the _onDestroy onto a different object, under the suspicion that PyQt was disconnecting all instance methods associated with the to-be-deleted object before emitting its destroyed signal.

Any other ideas for improving this PR?

@dgoeries
Copy link
Copy Markdown
Contributor

Using the @slot decorator is beneficial.

Connecting a signal to a decorated Python method also has the advantage of reducing the amount of memory used and is slightly faster.

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Feb 23, 2021

@dgoeries I've seen this talked about before, but I've never been able to test this. Every time I try and get specific with my slots...

@Slot(tuple)  # using tuple as an example, I haven't done exhaustive testing, but this holds for many cases...
def someSlot(self, arg1):
    sender = self.sender()
    isinstance(sender, QObject)  # this returns False because sender is None in this case
    ...

but when I do...

@Slot(object)
def someSlot(self, arg1):
    sender = self.sender()
    isinstance(sender, QObject)  # this returns True
    ...

So if I'm just decorating my slots to pass object, is there really any performance boost?

This is outside the scope of this PR, but I am eager to know if this works and if we can measure the performance benefit.

In the meantime, I think I'm going to merge this as is. Thanks @outofculture !

@j9ac9k j9ac9k merged commit cd73aee into pyqtgraph:master Feb 23, 2021
@outofculture outofculture deleted the selfless-lambdas branch May 6, 2021 00:38
@j9ac9k j9ac9k mentioned this pull request Dec 11, 2021
13 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants