[ty] Validate TypedDict fields when subclassing#24338
Conversation
Typing conformance results improved 🎉The percentage of diagnostics emitted that were expected errors increased from 86.79% to 86.89%. The percentage of expected errors that received a diagnostic increased from 81.72% to 82.47%. The number of fully passing files improved from 70/132 to 72/132. SummaryHow are test cases classified?Each test case represents one expected error annotation or a group of annotations sharing a tag. Counts are per test case, not per diagnostic — multiple diagnostics on the same line count as one. Required annotations (
Test file breakdown2 files altered
True positives added (8)8 diagnostics
|
Memory usage reportSummary
Significant changesClick to expand detailed breakdowntrio
prefect
sphinx
|
|
| Lint rule | Added | Removed | Changed |
|---|---|---|---|
invalid-typed-dict-field |
53 | 0 | 0 |
unused-type-ignore-comment |
0 | 1 | 0 |
| Total | 53 | 1 | 0 |
Raw diff (54 changes)
bokeh (https://github.com/bokeh/bokeh)
+ src/bokeh/models/axes.pyi:115:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `ticker`: Inherited mutable field type `Ticker` is incompatible with `BasicTicker`
+ src/bokeh/models/axes.pyi:116:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `formatter`: Inherited mutable field type `TickFormatter` is incompatible with `BasicTickFormatter`
+ src/bokeh/models/axes.pyi:125:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `ticker`: Inherited mutable field type `Ticker` is incompatible with `LogTicker`
+ src/bokeh/models/axes.pyi:126:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `formatter`: Inherited mutable field type `TickFormatter` is incompatible with `LogTickFormatter`
+ src/bokeh/models/axes.pyi:135:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `ticker`: Inherited mutable field type `Ticker` is incompatible with `CategoricalTicker`
+ src/bokeh/models/axes.pyi:136:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `formatter`: Inherited mutable field type `TickFormatter` is incompatible with `CategoricalTickFormatter`
+ src/bokeh/models/axes.pyi:149:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `ticker`: Inherited mutable field type `BasicTicker` is incompatible with `DatetimeTicker`
+ src/bokeh/models/axes.pyi:150:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `formatter`: Inherited mutable field type `BasicTickFormatter` is incompatible with `DatetimeTickFormatter`
+ src/bokeh/models/axes.pyi:159:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `ticker`: Inherited mutable field type `BasicTicker` is incompatible with `MercatorTicker`
+ src/bokeh/models/axes.pyi:160:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `formatter`: Inherited mutable field type `BasicTickFormatter` is incompatible with `MercatorTickFormatter`
+ src/bokeh/models/axes.pyi:169:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `ticker`: Inherited mutable field type `BasicTicker` is incompatible with `TimedeltaTicker`
+ src/bokeh/models/axes.pyi:170:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `formatter`: Inherited mutable field type `BasicTickFormatter` is incompatible with `TimedeltaTickFormatter`
+ src/bokeh/plotting/_figure.pyi:107:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `x_range`: Inherited mutable field type `Range` is incompatible with `Range | tuple[int | float, int | float] | tuple[str | date, str | date] | ... omitted 3 union elements`
+ src/bokeh/plotting/_figure.pyi:108:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `y_range`: Inherited mutable field type `Range` is incompatible with `Range | tuple[int | float, int | float] | tuple[str | date, str | date] | ... omitted 3 union elements`
discord.py (https://github.com/Rapptz/discord.py)
+ discord/types/components.py:54:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[1]`
+ discord/types/components.py:59:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[2]`
+ discord/types/components.py:87:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[3]`
+ discord/types/components.py:92:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[5]`
+ discord/types/components.py:97:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[6]`
+ discord/types/components.py:102:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[7]`
+ discord/types/components.py:107:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[8]`
+ discord/types/components.py:113:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[4]`
+ discord/types/components.py:125:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[3, 5, 6, 7, 8]`
+ discord/types/components.py:133:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[9]`
+ discord/types/components.py:139:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[10]`
+ discord/types/components.py:156:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[11]`
+ discord/types/components.py:169:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[12]`
+ discord/types/components.py:174:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[13]`
+ discord/types/components.py:182:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[14]`
+ discord/types/components.py:188:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[17]`
+ discord/types/components.py:195:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[18]`
+ discord/types/components.py:202:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[19]`
+ discord/types/components.py:210:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[21]`
+ discord/types/components.py:220:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[22]`
+ discord/types/components.py:232:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[23]`
+ discord/types/interactions.py:214:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[4]`
+ discord/types/interactions.py:220:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[3, 5, 6, 7, 8]`
+ discord/types/interactions.py:226:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[19]`
+ discord/types/interactions.py:232:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[21]`
+ discord/types/interactions.py:234:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `id`: Mutable inherited `NotRequired` fields cannot be redeclared as required
+ discord/types/interactions.py:239:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[22]`
+ discord/types/interactions.py:241:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `id`: Mutable inherited `NotRequired` fields cannot be redeclared as required
+ discord/types/interactions.py:246:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[23]`
+ discord/types/interactions.py:248:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `id`: Mutable inherited `NotRequired` fields cannot be redeclared as required
+ discord/types/interactions.py:268:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[10]`
+ discord/types/interactions.py:273:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[18]`
+ discord/types/scheduled_event.py:84:7 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `user_count` while merging base classes: Required inherited fields cannot be redeclared as `NotRequired`
+ discord/types/scheduled_event.py:87:7 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `user_count` while merging base classes: Required inherited fields cannot be redeclared as `NotRequired`
+ discord/types/scheduled_event.py:90:7 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `user_count` while merging base classes: Required inherited fields cannot be redeclared as `NotRequired`
+ discord/ext/commands/hybrid.py:61:9 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `description`: Inherited mutable field type `str` is incompatible with `str | locale_str`
+ discord/ext/commands/hybrid.py:69:9 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `description`: Inherited mutable field type `str | locale_str` is incompatible with `str`
+ discord/ext/commands/hybrid.py:73:9 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `description`: Inherited mutable field type `str` is incompatible with `str | locale_str`
hydra-zen (https://github.com/mit-ll-responsible-ai/hydra-zen)
- src/hydra_zen/typing/_implementations.py:441:30 warning[unused-type-ignore-comment] Unused blanket `type: ignore` directive
meson (https://github.com/mesonbuild/meson)
+ mesonbuild/modules/python.py:55:9 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `install_dir`: Inherited mutable field type `list[str | bool]` is incompatible with `str | bool | None`831ef5c to
49ff753
Compare
|
Codex believes that the ecosystem diagnostics are all true positives. |
8918d87 to
efdfe4e
Compare
| diagnostic.info("Only annotated declarations (`<name>: <type>`) are allowed."); | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
(This was moved to validate_typed_dict_class_body.)
| validate_typed_dict_field_overrides(context, class, class_node, direct_bases); | ||
| } | ||
|
|
||
| fn validate_typed_dict_class_body(context: &InferContext<'_, '_>, class_node: &ast::StmtClassDef) { |
There was a problem hiding this comment.
This method is just logic moved over from crates/ty_python_semantic/src/types/infer/deferred/static_class.rs.)
efdfe4e to
1dfe35a
Compare
AlexWaygood
left a comment
There was a problem hiding this comment.
(not a full review, just a couple of things from skimming)
There was a problem hiding this comment.
Not a complete review yet.
For a simple example like:
from typing import Collection, TypedDict
from typing_extensions import NotRequired, ReadOnly, Required
class Movie(TypedDict):
name: str
class BadMovie(Movie):
name: intI get the following diagnostic:
which feels pretty overwhelming:
- I don't really understand where the primary (red) annotation points to?
- Marking the whole TypedDict in a secondary annotation does not feel super useful?
- For the child-class field, we take the full
name: intrange, but for the base-class field, we only highlight the name?
Maybe this also explains why we show a new diagnostic on hydra-zen here, even though there's a # type: ignore on the field declaration?
|
Will do a pass to clean those up and add some diagnostic coverage! |
8245bac to
f5a0402
Compare
f5a0402 to
74ef839
Compare
sharkdp
left a comment
There was a problem hiding this comment.
Thank you for the update. Very nice and clear implementation!
I extended the test suite a bit and added some explanations — mostly for my own understanding, but hopefully doesn't hurt.
| impl<'db> TypedDictFieldOverrideReason<'db> { | ||
| fn from_fields( |
|
(I will do the rebase) |
|
Thank you! |
f49ef2b to
45b613d
Compare
* main: Document adding fixes in CONTRIBUTING.md (#24393) Sort formatter diagnostics in snapshots (#24375) [`pyupgrade`] Fix panic caused by handling of octals in `UP012` (#24390) Upgrade to nix v0.31.2 (#24385) Strip form feeds from indent passed to `dedent_to` (#24381) add recent move of the `deferred` submodule to `.git-blame-ignore-revs` (#24379) [ty] Fix extra_items TypedDict tests (#24367) [ty] Use `infer_type_expression` for validating PEP-613 type aliases (#24370) [`flake8-simplify`] Make the fix for `collapsible-if` (`SIM102`) safe in `preview` (#24371) [ty] Validate TypedDict fields when subclassing (#24338) [ty] pass type context to sequence literals in binary operations (#24197) Add release environment to notify-dependents job (#24372) Bump 0.15.9 (#24369) [ty] Move the `deferred` submodule inside `infer/builder` (#24368) [ty] Infer the `extra_items` keyword argument to class-based TypedDicts as an annotation expression (#24362) [ty] Validate type qualifiers in functional TypedDict fields and the `extra_items` keyword to functional TypedDicts (#24360)

Summary
When a TypedDict inherits from another class, for each field, the child has to preserve the same value and the same
Required/NotRequiredclassification. We now enforce these requirements.For example, this isn't allowed: