Skip to content

[ty] Preserve TypedDicts for compatible PEP 584 updates#23806

Merged
charliermarsh merged 5 commits intomainfrom
charlie/or-dict
Mar 11, 2026
Merged

[ty] Preserve TypedDicts for compatible PEP 584 updates#23806
charliermarsh merged 5 commits intomainfrom
charlie/or-dict

Conversation

@charliermarsh
Copy link
Member

Summary

Preserve TypedDict types for compatible PEP 584 updates instead of widening to dict:

class Person(TypedDict):
    name: str
    age: int | None

bob: Person = {"name": "Bob", "age": 25}

reveal_type(bob | {"age": 26})      # Person
reveal_type({"age": 26} | bob)      # Person

Closes astral-sh/ty#2798.

@charliermarsh charliermarsh added the ty Multi-file analysis & type inference label Mar 8, 2026
@astral-sh-bot
Copy link

astral-sh-bot bot commented Mar 8, 2026

Typing conformance results

No changes detected ✅

Current numbers
The percentage of diagnostics emitted that were expected errors held steady at 85.05%. The percentage of expected errors that received a diagnostic held steady at 78.05%. The number of fully passing files held steady at 63/132.

@astral-sh-bot
Copy link

astral-sh-bot bot commented Mar 8, 2026

Memory usage report

Memory usage unchanged ✅

@astral-sh-bot
Copy link

astral-sh-bot bot commented Mar 8, 2026

mypy_primer results

Changes were detected when running on open source projects
scikit-build-core (https://github.com/scikit-build/scikit-build-core)
+ src/scikit_build_core/build/wheel.py:99:20: error[no-matching-overload] No overload of bound method `__init__` matches arguments
- Found 59 diagnostics
+ Found 60 diagnostics

core (https://github.com/home-assistant/core)
- homeassistant/helpers/discovery_flow.py:59:19: error[invalid-assignment] Object of type `dict[str, object]` is not assignable to `ConfigFlowContext`
- Found 12092 diagnostics
+ Found 12091 diagnostics

@charliermarsh charliermarsh force-pushed the charlie/or-dict branch 3 times, most recently from e297e20 to 8ac2cf6 Compare March 8, 2026 16:21
@charliermarsh charliermarsh marked this pull request as ready for review March 8, 2026 16:25
Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, what motivated the change in approach here from #23408? I liked the overall approach in that PR of synthesizing __(r)or__ in types/class.rs, I just wasn't convinced that the signatures of the synthesized methods you had were correct

@charliermarsh
Copy link
Member Author

Oh sorry, my impression was the prior approach was way off. Let me revisit.

@charliermarsh charliermarsh marked this pull request as draft March 8, 2026 16:43
@charliermarsh
Copy link
Member Author

Okay, I looked at #23408 again (and updated the relevant branch). The problem is, if we just add the synthesized method, we still see this behavior:

from typing import TypedDict

class MyDict(TypedDict):
    foo: int
    bar: str

val = MyDict(foo=1, bar="hello")
result = val | {"foo": 2}
reveal_type(result)  # dict[str, object], but should be MyDict

Because {"foo": 2} is inferred as dict[str, int], which hits the dict[str, Any] -> dict[str, object] fallback. So we still need some changes to these codepaths to add some amount of bidirectional inference, IIUC.

The resulting diff ends up being similar, except we have a synthesized method instead of:

(Type::TypedDict(left_typed_dict), rhs, ast::Operator::BitOr)
    if rhs.is_assignable_to(db, Type::TypedDict(left_typed_dict)) =>
{
    Some(Type::TypedDict(left_typed_dict))
}

(lhs, Type::TypedDict(right_typed_dict), ast::Operator::BitOr)
    if lhs.is_assignable_to(db, Type::TypedDict(right_typed_dict)) =>
{
    Some(Type::TypedDict(right_typed_dict))
}

To me, this approach seems a little better given the above.

@charliermarsh charliermarsh marked this pull request as ready for review March 8, 2026 18:53
@AlexWaygood
Copy link
Member

AlexWaygood commented Mar 8, 2026

The advantage of the synthesized-method approach is that these tests, which fail on your branch, would (I think) naturally pass:

from typing import TypedDict, Protocol, Self

class Foo(TypedDict):
    x: str
    y: int

class Bar(TypedDict, closed=True):
    x: str

def _(f: Foo, g: Bar):
    f |= g  # errors on this branch, but should pass

class Proto(Protocol):
    def __or__(self, other: Bar) -> Self: ...

def takes_proto(x: Proto): ...

def test(x: Foo):
    takes_proto(x)  # errors on this branch, but should pass

And I suspect we will have to add bidirectional inference for __(r)or__ calls anyway at some point, much like we already have for other dunders, such as __setitem__?

Comment on lines +129 to +140
/// Returns a synthesized `TypedDict` with the same keys and value types, but with all keys
/// marked `NotRequired`. The result is always a `Synthesized` variant (the original class
/// identity, if any, is not preserved).
pub(crate) fn to_partial(self, db: &'db dyn Db) -> Self {
let items: TypedDictSchema<'db> = self
.items(db)
.iter()
.map(|(name, field)| (name.clone(), field.clone().with_required(false)))
.collect();

Self::Synthesized(SynthesizedTypedDictType::new(db, items))
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sharkdp and I had a conversation over DMs after your original PR and concluded that the synthesized TypedDict should probably also be closed=True in order for the approach to be sound. We don't support PEP 728 yet, so we can't implement that right now, but we should probably add a TODO comment.

@charliermarsh
Copy link
Member Author

Okay, let me update this branch to use the synthesized approach and add those tests.

@charliermarsh charliermarsh marked this pull request as draft March 8, 2026 19:39
@charliermarsh
Copy link
Member Author

BTW, in the updated implementation, I actually needed three overloads...

def __or__(self: Self, value: Self, /) -> Self
def __or__(self: Self, value: Partial[Self], /) -> Self
def __or__(self: Self, value: dict[str, Any], /) -> dict[str, object]

Where Partial[Self] is Self with every field made NotRequired.

If you only have the first and third overload, then partial dicts on the RHS aren't accepted. If you only have the second and third, then if Self has a required field, the TypedDict itself isn't accepted on the RHS.

@AlexWaygood
Copy link
Member

Ah, hmm... can those first two overloads be combined?

def __or__(self: Self, value: Self | Partial[Self], /) -> Self
def __or__(self: Self, value: dict[str, Any], /) -> dict[str, object]

@charliermarsh charliermarsh force-pushed the charlie/or-dict branch 2 times, most recently from 238283e to a4f6eca Compare March 9, 2026 16:47
@charliermarsh
Copy link
Member Author

Yeah this one is getting quite challenging to get right everywhere.

@charliermarsh charliermarsh marked this pull request as ready for review March 10, 2026 00:38
@carljm carljm removed their request for review March 10, 2026 01:03
Copy link
Member

@dcreager dcreager left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of naming/comment nits, and a merge conflict to fix, otherwise looks good!

CallableTypeKind::FunctionLike,
)))
}
(CodeGeneratorKind::TypedDict, name @ ("__or__" | "__ror__" | "__ior__")) => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a comment here showing the Python syntax of the signatures you're creating?

/// Returns a synthesized `TypedDict` used to model PEP 584 update operands. This accepts
/// dictionary literals that update any subset of known keys, and also accepts other
/// `TypedDict`s as long as any overlapping keys are compatible.
pub(crate) fn to_update_operand(self, db: &'db dyn Db) -> Self {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I think I prefer the "partial" naming you used in the PR comments. This name talks about (and makes assumptions about) how the result will be used, but something like make_partial talks about the structure of the result more generally.

Copy link
Member

@ibraheemdev ibraheemdev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. I think a lot of this special casing can be removed with more general bidirectional inference for binary operator dunder calls, but it seems fine for now.

@charliermarsh charliermarsh enabled auto-merge (squash) March 11, 2026 01:27
@charliermarsh charliermarsh merged commit f510d54 into main Mar 11, 2026
50 checks passed
@charliermarsh charliermarsh deleted the charlie/or-dict branch March 11, 2026 01:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PEP-584 union operator incorrectly widens TypedDict values

4 participants