Skip to content

9972: support yield on coroutines within inlineCallbacks functions#12239

Merged
adiroiban merged 3 commits intotwisted:trunkfrom
p12tic:9972-inlinecallbacks-coroutines
Jul 22, 2024
Merged

9972: support yield on coroutines within inlineCallbacks functions#12239
adiroiban merged 3 commits intotwisted:trunkfrom
p12tic:9972-inlinecallbacks-coroutines

Conversation

@p12tic
Copy link
Contributor

@p12tic p12tic commented Jul 17, 2024

Scope and purpose

Fixes #9972.

The change is very simple - it essentially embeds Deferred.fromCoroutine() into within inlineCallbacks().

This allows piecewise porting of inlineCallbacks-based codebases to coroutines. As long as addCallback(), addErrback() and friends are not used on results of inlineCallbacks-decorated functions, the conversion process is very simple.

@codspeed-hq
Copy link

codspeed-hq bot commented Jul 17, 2024

CodSpeed Performance Report

Merging #12239 will not alter performance

Comparing p12tic:9972-inlinecallbacks-coroutines (d9fa675) with trunk (668ce3c)

Summary

✅ 16 untouched benchmarks

@p12tic p12tic force-pushed the 9972-inlinecallbacks-coroutines branch 2 times, most recently from 5178a49 to a002fe7 Compare July 17, 2024 01:45
@p12tic
Copy link
Contributor Author

p12tic commented Jul 17, 2024

please review

@chevah-robot chevah-robot requested a review from a team July 17, 2024 01:54
@p12tic p12tic force-pushed the 9972-inlinecallbacks-coroutines branch from a002fe7 to 2ad8ce9 Compare July 17, 2024 02:08
Copy link
Member

@adiroiban adiroiban left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Many thanks for working on this.

Only minor comment.

Please review before the merge.

Thanks again!

it is possible to rewrite functions using C{inlineCallbacks} to C{async def}
in piecewise manner and be mostly compatible to existing code.

The rewrite process is simply replacing C{inlineCallbacks} decorator with
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor comment.

I am not sure if we need this paragraph.
I would say that this info can be part of defer-intro.rst

Otherwise we end up with duplicated documentation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that docstrings and defer-intro.rst serve different purposes. The former is a reference, a person knows what he wants to do and just comes to look for details. The latter is tutorial - a person does not know how to achieve something, thus one reads tutorial start to end to get general understanding.

In this particular case I think we should mention that there's a better way to do things in both places. I agree that this introduces some duplication, but it serves a practical purpose that developers reading the documentation are less confused. I think this saves time overall if we think about both developers of this project and its users.

Finally, reducing duplication serves the purpose of reducing the chances of errors, when only one repetition is updated. I think this is less of concern in this case, because the next change is likely to be removal of inlineCallbacks some years into the future.

yield getDivisionFailureCoro("OMG")
except ZeroDivisionError as e:
self.assertEqual(str(e), "OMG")
returnValue("WOOSH")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use the standard return statement ?

Suggested change
returnValue("WOOSH")
return "WOOSH"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This tests that returnValue still works. Once we actually remove the feature we can simplify the tests too.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My worry is that returValue now raises a deprecation warning and these new tests are not asserting or clearing that warning.

This is not a big issue now... but it adds some tech debt.

In an ideal case, we will run each test while capturing any warning and failing if there are unhandled / un-asserted warnings.

Unfortunately, trial has no support for automatic catching of the warnings that are generated during a test run... and this is why the test are not yet failing.


I understand that the current tests have a wider coverage ... so we can keep this code.


I was thinking that since returnValue is now a thing of the past, we can stop worrying about it and don't bother too much with covering returnValue in new tests.

If returnValue don't work with coroutine, that should be fine... as the original code written using returnValue is old code from the time when coroutines were not supported.

p12tic added 2 commits July 22, 2024 03:52
This essentially embeds Deferred.fromCoroutine() into within
inlineCallbacks().

This allows piecewise porting of inlineCallbacks-based codebases to
coroutines. As long as addCallback(), addErrback() and friends are not
used on results of inlineCallbacks-decorated functions, the conversion
process is very simple.
@p12tic p12tic force-pushed the 9972-inlinecallbacks-coroutines branch from 209e92d to 21e56ca Compare July 22, 2024 00:54
@p12tic
Copy link
Contributor Author

p12tic commented Jul 22, 2024

@adiroiban Thanks for review, I've addressed all of your comments - either with a fix or a comment why I think the fix is not necessary.

@adiroiban adiroiban merged commit 11b7596 into twisted:trunk Jul 22, 2024
@adiroiban
Copy link
Member

Thanks for the update. I have merged this.

I left some feedback wheter we should continue to worry about returnValue for new features.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support @inlineCallbacks on async def functions

3 participants