Skip to content

[ty] Type inference for @asynccontextmanager #21876

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
sharkdp merged 2 commits into from
Dec 9, 2025
Merged

[ty] Type inference for @asynccontextmanager #21876

sharkdp merged 2 commits into from
Dec 9, 2025

Conversation

@sharkdp
Copy link
Contributor

@sharkdp sharkdp commented Dec 9, 2025 •
edited
Loading

Summary

This PR adds special handling for asynccontextmanager calls as a temporary solution for astral-sh/ty#1804. We will be able to remove this soon once we have support for generic protocols in the solver.

closes astral-sh/ty#1804

Ecosystem

+ tests/test_downloadermiddleware.py:305:56: error[invalid-argument-type] Argument to bound method `download` is incorrect: Expected `Spider`, found `Unknown | Spider | None`
+ tests/test_downloadermiddleware.py:305:56: warning[possibly-missing-attribute] Attribute `spider` may be missing on object of type `Crawler | None`

These look like true positives

+ pymongo/asynchronous/database.py:1021:35: error[invalid-assignment] Object of type `(AsyncClientSession & ~AlwaysTruthy & ~AlwaysFalsy) | (_ServerMode & ~AlwaysFalsy) | Unknown | Primary` is not assignable to `_ServerMode | None`
+ pymongo/asynchronous/database.py:1025:17: error[invalid-argument-type] Argument to bound method `_conn_for_reads` is incorrect: Expected `_ServerMode`, found `_ServerMode | None`

Known problems or true positives, just caused by the new type for session

- src/integrations/prefect-sqlalchemy/prefect_sqlalchemy/database.py:269:16: error[invalid-return-type] Return type does not match returned value: expected `Connection | AsyncConnection`, found `_GeneratorContextManager[Unknown, None, None] | _AsyncGeneratorContextManager[Unknown, None] | Connection | AsyncConnection`
+ src/integrations/prefect-sqlalchemy/prefect_sqlalchemy/database.py:269:16: error[invalid-return-type] Return type does not match returned value: expected `Connection | AsyncConnection`, found `_GeneratorContextManager[Unknown, None, None] | _AsyncGeneratorContextManager[AsyncConnection, None] | Connection | AsyncConnection`

Just a more concrete type

- src/prefect/flow_engine.py:1277:24: error[missing-argument] No argument provided for required parameter `cls`
- src/prefect/server/api/server.py:696:49: error[missing-argument] No argument provided for required parameter `cls`
- src/prefect/task_engine.py:1426:24: error[missing-argument] No argument provided for required parameter `cls`

Good

Test Plan

  • Adapted and newly added Markdown tests
  • Tested on internal codebase

@sharkdp sharkdp added ty Multi-file analysis & type inference ecosystem-analyzer labels Dec 9, 2025
sharkdp
sharkdp commented Dec 9, 2025
View reviewed changes
crates/ty_python_semantic/src/types/call/bind.rs Outdated
Comment on lines 4663 to 4665
if yield_ty.is_divergent() {
return Some(yield_ty);
}
Copy link
Contributor Author

@sharkdp sharkdp Dec 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This took me a while to figure out. Looks related to astral-sh/ty#1821 / astral-sh/ty#1821. If we don't do this, we end up with unions of the actual type that we want to have and Divergent

@astral-sh-bot
Copy link

astral-sh-bot bot commented Dec 9, 2025 •
edited
Loading

Diagnostic diff on typing conformance tests

No changes detected when running ty on typing conformance tests ✅

@astral-sh-bot
Copy link

astral-sh-bot bot commented Dec 9, 2025 •
edited
Loading

mypy_primer results

Changes were detected when running on open source projects 
beartype (https://github.com/beartype/beartype)
- beartype/claw/_package/clawpkgtrie.py:66:29: warning[unsupported-base] Unsupported class base with type `<class 'dict[str, PackagesTrieBlacklist]'> | <class 'dict[str, Divergent]'>`
- beartype/claw/_package/clawpkgtrie.py:247:29: warning[unsupported-base] Unsupported class base with type `<class 'dict[str, PackagesTrieWhitelist]'> | <class 'dict[str, Divergent]'>`
- Found 494 diagnostics
+ Found 492 diagnostics

scrapy (https://github.com/scrapy/scrapy)
+ tests/test_downloadermiddleware.py:305:56: error[invalid-argument-type] Argument to bound method `download` is incorrect: Expected `Spider`, found `Unknown | Spider | None`
+ tests/test_downloadermiddleware.py:305:56: warning[possibly-missing-attribute] Attribute `spider` may be missing on object of type `Crawler | None`
- Found 1735 diagnostics
+ Found 1737 diagnostics

mongo-python-driver (https://github.com/mongodb/mongo-python-driver)
+ pymongo/asynchronous/database.py:1021:35: error[invalid-assignment] Object of type `(AsyncClientSession & ~AlwaysTruthy & ~AlwaysFalsy) | (_ServerMode & ~AlwaysFalsy) | Unknown | Primary` is not assignable to `_ServerMode | None`
+ pymongo/asynchronous/database.py:1025:17: error[invalid-argument-type] Argument to bound method `_conn_for_reads` is incorrect: Expected `_ServerMode`, found `_ServerMode | None`
- Found 460 diagnostics
+ Found 462 diagnostics

trio (https://github.com/python-trio/trio)
+ src/trio/_channel.py:610:12: error[invalid-return-type] Return type does not match returned value: expected `(**P@as_safe_channel) -> AbstractAsyncContextManager[ReceiveChannel[T@as_safe_channel], bool | None]`, found `((self, *args: P@as_safe_channel.args, **kwargs: P@as_safe_channel.kwargs) -> _AsyncGeneratorContextManager[Unknown, None]) | Unknown`
- Found 494 diagnostics
+ Found 495 diagnostics

scikit-build-core (https://github.com/scikit-build/scikit-build-core)
+ src/scikit_build_core/_logging.py:153:13: warning[unsupported-base] Unsupported class base with type `<class 'Mapping[str, Style]'> | <class 'Mapping[str, Divergent]'>`
- Found 41 diagnostics
+ Found 42 diagnostics

prefect (https://github.com/PrefectHQ/prefect)
- src/integrations/prefect-sqlalchemy/prefect_sqlalchemy/database.py:269:16: error[invalid-return-type] Return type does not match returned value: expected `Connection | AsyncConnection`, found `_GeneratorContextManager[Unknown, None, None] | _AsyncGeneratorContextManager[Unknown, None] | Connection | AsyncConnection`
+ src/integrations/prefect-sqlalchemy/prefect_sqlalchemy/database.py:269:16: error[invalid-return-type] Return type does not match returned value: expected `Connection | AsyncConnection`, found `_GeneratorContextManager[Unknown, None, None] | _AsyncGeneratorContextManager[AsyncConnection, None] | Connection | AsyncConnection`
- src/prefect/flow_engine.py:1277:24: error[missing-argument] No argument provided for required parameter `cls`
- src/prefect/server/api/server.py:696:49: error[missing-argument] No argument provided for required parameter `cls`
- src/prefect/task_engine.py:1426:24: error[missing-argument] No argument provided for required parameter `cls`
- Found 5480 diagnostics
+ Found 5477 diagnostics

No memory usage changes detected ✅

@astral-sh-bot
Copy link

astral-sh-bot bot commented Dec 9, 2025 •
edited
Loading

ecosystem-analyzer results

Lint rule Added Removed Changed
invalid-argument-type 2 0 0
invalid-return-type 1 0 1
invalid-assignment 1 0 0
possibly-missing-attribute 1 0 0
unsupported-base 0 1 0
Total 5 1 1

Full report with detailed diff (timing results)

@sharkdp sharkdp force-pushed the david/fix-1804 branch 2 times, most recently from 810e257 to f02359c Compare December 9, 2025 20:40
sharkdp
sharkdp commented Dec 9, 2025
View reviewed changes
scripts/check_ecosystem.py

@asynccontextmanager
async def clone(self: Self, checkout_dir: Path) -> AsyncIterator[Path]:
async def clone(self: Self, checkout_dir: Path) -> AsyncIterator[str]:
Copy link
Contributor Author

@sharkdp sharkdp Dec 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh look, we found a bug.

@sharkdp sharkdp marked this pull request as ready for review December 9, 2025 20:56
@astral-sh-bot
Copy link

astral-sh-bot bot commented Dec 9, 2025 •
edited
Loading

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

Formatter (stable)

✅ ecosystem check detected no format changes.

Formatter (preview)

✅ ecosystem check detected no format changes.

carljm
carljm approved these changes Dec 9, 2025
View reviewed changes
Copy link
Contributor

@carljm carljm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a reasonable temporary patch for a common case to me!

dcreager
dcreager approved these changes Dec 9, 2025
View reviewed changes
Copy link
Member

@dcreager dcreager left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 thank you!

@sharkdp
Copy link
Contributor Author

sharkdp commented Dec 9, 2025

Merging despite the failing benchmark run (timeout). Benchmarks were running on a previous version of this PR and I only made a completely trivial change since then.

@sharkdp sharkdp merged commit a9be810 into main Dec 9, 2025
39 of 40 checks passed
@sharkdp sharkdp deleted the david/fix-1804 branch December 9, 2025 21:49
dcreager added a commit that referenced this pull request Dec 10, 2025
@dcreager
Merge remote-tracking branch 'origin/main' into dcreager/callable-return
4f7ad7b 
* origin/main: (33 commits)
  [ty] Simplify union lower bounds and intersection upper bounds in constraint sets (#21871)
  [ty] Collapse `never` paths in constraint set BDDs (#21880)
  Fix leading comment formatting for lambdas with multiple parameters (#21879)
  [ty] Type inference for `@asynccontextmanager` (#21876)
  Fix comment placement in lambda parameters (#21868)
  [`pylint`] Detect subclasses of builtin exceptions (`PLW0133`) (#21382)
  Fix stack overflow with recursive generic protocols (depth limit) (#21858)
  New diagnostics for unused range suppressions (#21783)
  [ty] Use default settings in completion tests
  [ty] Infer type variables within generic unions  (#21862)
  [ty] Fix overload filtering to prefer more "precise" match (#21859)
  [ty] Stabilize auto-import
  [ty] Fix reveal-type E2E test (#21865)
  [ty] Use concise message for LSP clients not supporting related diagnostic information (#21850)
  Include more details in Tokens 'offset is inside token' panic message (#21860)
  apply range suppressions to filter diagnostics (#21623)
  [ty] followup: add-import action for `reveal_type` too (#21668)
  [ty] Enrich function argument auto-complete suggestions with annotated types
  [ty] Add autocomplete suggestions for function arguments
  [`flake8-bugbear`] Accept immutable slice default arguments (`B008`) (#21823)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dcreager dcreager dcreager approved these changes

@carljm carljm carljm approved these changes

@AlexWaygood AlexWaygood Awaiting requested review from AlexWaygood AlexWaygood is a code owner

Assignees

No one assigned

Labels

ecosystem-analyzer ty Multi-file analysis & type inference

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

async with usage with asynccontextmanager-decorated function results in Unknown

4 participants

@sharkdp @dcreager @carljm