Loop Factory parameter for asyncio mark#1164
Conversation
|
Can't seem to figure out where I should inject this new function I made. When I would go to run tests they would fail. |
|
Testing with different event loops is definitely something we want to support in pytest-asyncio. Currently, there's an event_loop_policy fixture for this purpose. Can you explain why the existing approach is insufficient for your use cases? |
@seifertm I would love to, as of right now eventloop policies are deprecated and planned for removal in 3.16 my goal is to provide a workaround that doesn't involve silencing warnings and allows for the EventLoopPolicy to remain temporarly but redirect users to using a non-deprecated approach sooner than later. I'm trying to catch up with these changes by allowing for a different fixture method to be used I would like to save this library from simply having to keep a warning silent which I would argue is a bad approach to solving a problem. Sooner it's changed the better. |
|
I agree that the existing approach with the I'd like to avoid configuring the loop type through a fixture, though, for two reasons:
I'm currently leaning towards a configuration setting in pyproject.toml dor a global parametrization of loop types. I don't really like it, so I'm open for alternative suggestions :) It definitely needs some more thought. Besides parametrizing globally, we also want users to be able to override the loop type for specific tests (see #1032 #1101). Users should be able to pass a @pytest.mark.asyncio(loop_factory=uvloop.new_event_loop)
def test_run_with_uvloop():
...If you're interested in working on the |
That would be a great solution actually. Thanks for helping me come up with a compromise, I'll see what I can do to make that happen. |
|
Let me revert my changes and then I'll make new ones. |
|
Guess the only thing I will have to figure out is where loop-factory should get executed I wanted to make it so that both Sequences of factories and singular factories could be accepted. |
|
Now I think about it I am having trouble finding where to inject loop factories. I'll upload what I have so far and take a temporary break. But I'm trying to make it so that loop_factory can pass multiple factories and run them all. |
|
This puzzle turned out to be bigger than I thought let me push what I have now so far... |
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
…e a comment explaining it.
seifertm
left a comment
There was a problem hiding this comment.
Thanks for the effort. I rebased the PR.
I think this is a good start, but there are some edge cases that we didn't think of at first (see my previous PR comment). I added some tests. I think their code explains the issues best.
| print(f"FACTORY {factory}") | ||
| return factory | ||
| else: | ||
| return request.obj.__dict__.get("_loop_factory", None) # type: ignore[attr-defined] |
There was a problem hiding this comment.
My understanding is that the else branch is supposed to retrieve the loop factory from a fixture that's requested by the test. This seems reasonable, but not quite enough. It's possible that the fixture pulls in another fixture transitively, which can also have a loop factory.
Therefore, we need to walk the request's _iter_chain to find loop factories in transitive fixtures.
That still leaves the problem that a test can request multiple fixtures, each of which can define their own loop_factory and pull in other fixtures transitively. Effectively, that means we would have to walk the whole fixture dependency tree upwards. All of that makes me question if we should allow fixtures to set the loop factory.
It gets even more complicated than that: Let's assume all tests and fixtures run with loop_scope="module". If two tests in the same module define different loop factories, it's a user error and we have to bail out. However, get_loop_factory can't really identify the issue, because the function is executed as part of the fixture setup for an individual test. It has no knowledge whatsoever about the loop_factories defined in other tests.
dfa3a50 to
e1af98a
Compare
|
I had another look at the edge cases and how to address them. To be honest, I don't think it's reasonable to implement the proposed API. The The problem is exacerbated by the fact that tests may pull in fixtures, which can have their own loop_factory arguments and each fixture may depend on other fixtures. All in all, I think the amount of complexity isn't worth it. I suggest we proceed with a simpler type of parametrization, for example specifying a single set of loop factories that applies to all tests. The parametrization could potentially be limited to certain pytest node IDs. Any thoughts on this @Vizonex ? |
I would think that an approach similar to anyio would be the way to go in the future. Or I would be down for a rewrite of the backend if that helps with the organization of everything. |
…184) * test(claude_session): remove orphaned fixtures test left by #174 #174 (aa815ee) intentionally deleted mcp/claude-session/fixtures/ (the MalformedWriteToolInput model + its fixture + manifest.json) but left tests/claude_session/test_fixtures.py asserting that now-missing directory -- silently red on main since. Remove the dead test, its dir-local conftest, and the now-empty test package. * fix(tests): upgrade pytest-asyncio 1.3.0 -> 1.4.0 (event-loop leak) 1.3.0's _temporary_event_loop_policy captured the current loop via asyncio.get_event_loop() to restore per async test; on Python 3.13 that fabricates an unclosed _UnixSelectorEventLoop + AF_UNIX socketpair, which filterwarnings=["error"] escalated into a non-deterministic PytestUnraisableExceptionWarning landing on a random victim test. 1.4.0 drops the loop capture/restore (upstream pytest-dev/pytest-asyncio#1164). Verified: the previously non-deterministic cc_lib+crb+hooks combo now passes 429/429 across repeated runs. * ci: run the full suite (minus browser-only selenium) on every PR The pytest-asyncio 1.4.0 loop-leak fix + orphan-test cleanup make the non-browser suite deterministic, so CI now guards all 830 of those tests instead of just tests/linters/. tests/selenium_browser needs a real browser the ubuntu runner lacks; it gets a dedicated browser job in a follow-up. * ci: install zsh for claude-remote-bash executor tests The crb executor runs commands in /bin/zsh (its configured shell); ubuntu runners don't ship zsh, so 12 test_executor.py tests failed with FileNotFoundError. Same shape as libsnappy: a real crb runtime dependency the runner must provide.
This MR contains the following updates: | Package | Type | Update | Change | OpenSSF | |---|---|---|---|---| | [black](https://github.com/psf/black) ([changelog](https://github.com/psf/black/blob/main/CHANGES.md)) | dev | minor | `26.3.1` → `26.5.1` | [](https://securityscorecards.dev/viewer/?uri=github.com/psf/black) | | [pytest-asyncio](https://github.com/pytest-dev/pytest-asyncio) ([changelog](https://pytest-asyncio.readthedocs.io/en/stable/reference/changelog.html)) | test | minor | `1.3.0` → `1.4.0` | [](https://securityscorecards.dev/viewer/?uri=github.com/pytest-dev/pytest-asyncio) | | [python-gitlab](https://github.com/python-gitlab/python-gitlab) ([changelog](https://github.com/python-gitlab/python-gitlab/blob/main/CHANGELOG.md)) | dev | minor | `8.3.0` → `8.4.0` | [](https://securityscorecards.dev/viewer/?uri=github.com/python-gitlab/python-gitlab) | --- ### Release Notes <details> <summary>psf/black (black)</summary> ### [`v26.5.1`](https://github.com/psf/black/blob/HEAD/CHANGES.md#Version-2651) [Compare Source](psf/black@26.5.0...26.5.1) ##### Stable style - Fix unstable formatting of annotated assignments whose subscript annotation contains an inline comment (e.g. `x: list[ # pyright: ignore[...]`) ([#​5130](psf/black#5130)) - Preserve inline comments (including `# type: ignore`) immediately before a `# fmt: skip` line, avoiding AST equivalence failures ([#​5139](psf/black#5139)) ##### Packaging - Correct the version in the published executables ([#​5137](psf/black#5137)) ##### Documentation - Add Neovim integration guide covering conform.nvim, ALE, and simple command approaches ([#​5124](psf/black#5124)) ### [`v26.5.0`](https://github.com/psf/black/blob/HEAD/CHANGES.md#Version-2650) [Compare Source](psf/black@26.3.1...26.5.0) ##### Highlights - Add support for unpacking in comprehensions (PEP 798) and for lazy imports (PEP 810), both new syntactic features in Python 3.15 ([#​5048](psf/black#5048)) - Python 3.15 is now supported. Compiled wheels are not yet provided for Python 3.15, so performance may be slower than on existing Python versions. Wheels will be provided once Python 3.15 is later in its release cycle. ([#​5127](psf/black#5127)) ##### Stable style - Fix `# fmt: skip` being ignored in nested `if` expressions with parenthesized `in` clauses ([#​4903](psf/black#4903)) - Add syntactic support for Python 3.15 ([#​5048](psf/black#5048)) - Fix crash when an f-string follows a `# fmt: off` comment inside brackets ([#​5097](psf/black#5097)) - Preserve multiline compound statement headers when `# fmt: skip` is placed on the colon line ([#​5117](psf/black#5117)) ##### Preview style - Improve heuristics around whether blank lines should appear before, within and after groups of same-name decorated functions (such as `@overload` groups) in `.pyi` stub files ([#​5021](psf/black#5021)) - Fix blank lines being removed between a function and a decorated class in `.pyi` stub files ([#​5092](psf/black#5092)) - Prevent string merger from creating unsplittable long lines when a pragma comment (e.g. `# type: ignore`) follows the closing bracket ([#​5096](psf/black#5096)) ##### Packaging - Run CI on 3.15 ([#​5127](psf/black#5127)) ##### Output - Improve parse error readability by showing multi-line output with an error pointer. ([#​5068](psf/black#5068)) - Add `SourceASTParseError` to distinguish source parse failures from internal safety errors, improving error reporting when Black's lenient parser accepts input that `ast.parse()` rejects ([#​5080](psf/black#5080)) ##### *Blackd* - Return HTTP 400 (Bad Request) for source parse failures instead of HTTP 500, keeping HTTP 500 only for genuine internal safety errors ([#​5080](psf/black#5080)) ##### Integrations - Added documentation for doctest formatting tools and updated the integrations index to match ([#​4916](psf/black#4916)) ##### Documentation - Use "Version X.Y.Z" headings in changelog for stable permalink anchors on ReadTheDocs ([#​5063](psf/black#5063)) - Note in the editor integrations that the SublimeText `sublack` plugin is archived and unmaintained ([#​5082](psf/black#5082)) </details> <details> <summary>pytest-dev/pytest-asyncio (pytest-asyncio)</summary> ### [`v1.4.0`](https://github.com/pytest-dev/pytest-asyncio/releases/tag/v1.4.0): pytest-asyncio v1.4.0 [Compare Source](pytest-dev/pytest-asyncio@v1.3.0...v1.4.0) ### [1.4.0](https://github.com/pytest-dev/pytest-asyncio/tree/1.4.0) - 2026-05-26 #### Deprecated - Overriding the *event\_loop\_policy* fixture is deprecated. Use the `pytest_asyncio_loop_factories` hook instead. ([#​1419](pytest-dev/pytest-asyncio#1419)) #### Added - Added the `pytest_asyncio_loop_factories` hook to parametrize asyncio tests with custom event loop factories. The hook returns a mapping of factory names to loop factories, and `pytest.mark.asyncio(loop_factories=[...])` selects a subset of configured factories per test. When a single factory is configured, test names are unchanged. Synchronous `@pytest_asyncio.fixture` functions now see the correct event loop when custom loop factories are configured, even when test code disrupts the current event loop (e.g., via `asyncio.run()` or `asyncio.set_event_loop(None)`). ([#​1164](pytest-dev/pytest-asyncio#1164)) #### Changed - Improved the readability of the warning message that is displayed when `asyncio_default_fixture_loop_scope` is unset ([#​1298](pytest-dev/pytest-asyncio#1298)) - Only import `asyncio.AbstractEventLoopPolicy` for type checking to avoid raising a DeprecationWarning. ([#​1394](pytest-dev/pytest-asyncio#1394)) - Updated minimum supported pytest version to v8.4.0. ([#​1397](pytest-dev/pytest-asyncio#1397)) #### Fixed - Fixed a `ResourceWarning: unclosed event loop` warning that could occur when a synchronous test called `asyncio.run()` or otherwise unset the current event loop after pytest-asyncio had run an async test or fixture. ([#​724](pytest-dev/pytest-asyncio#724)) #### Notes for Downstream Packagers - Added dependency on `sphinx-tabs >= 3.5` to organize documentation examples into tabs. ([#​1395](pytest-dev/pytest-asyncio#1395)) </details> <details> <summary>python-gitlab/python-gitlab (python-gitlab)</summary> ### [`v8.4.0`](https://github.com/python-gitlab/python-gitlab/blob/HEAD/CHANGELOG.md#v840-2026-05-28) [Compare Source](python-gitlab/python-gitlab@v8.3.0...v8.4.0) ##### Features - **const**: Add new Security Manager role ([`3738bb2`](python-gitlab/python-gitlab@3738bb2)) ##### Testing - **const**: Add tests for AccessLevel ([`2ab6d9f`](python-gitlab/python-gitlab@2ab6d9f)) </details> --- - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box --- This MR has been generated by [Mend Renovate](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4yMDUuMiIsInVwZGF0ZWRJblZlciI6IjQzLjIwNS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJkZXBlbmRlbmNpZXMiLCJyZW5vdmF0ZSJdfQ==--> See merge request swiss-armed-forces/cyber-command/cea/loom!526 Co-authored-by: Loom MR Pipeline Trigger <group_103951964_bot_9504bb8dead6d4e406ad817a607f24be@noreply.gitlab.com>
After mentally thinking about this back and forth. I decided to step to the plate and try and fix this problem once and for all.
The aiohttp developers have told me they had a bit of intrest utilizing this library and so would I with my plans to migrate from unittest to pytest for winloop and a new project coming that aims to upgrade & migrate pycares to cython, which I've added my own deadline that I would move winloop to pytest by 2026. And yes aiothreading which aims to be an alternative to aiomultiprocess for users with slower hardware or providing safer solutions for heavier loads.
While talking with the aiohttp devs on the matrix about the problem. I decided to see if I could have a crack at providing a way to run eventloops without needing to use deprecated setups. Unfortunately I'm extremely picky about deprecated setups This might be a good thing however because the sooner someone does something to solve the problem the better.