Skip to content

[ty] Validate TypedDict fields when subclassing#24338

Merged
sharkdp merged 9 commits intomainfrom
charlie/typed-dict-validation
Apr 2, 2026
Merged

[ty] Validate TypedDict fields when subclassing#24338
sharkdp merged 9 commits intomainfrom
charlie/typed-dict-validation

Conversation

@charliermarsh
Copy link
Copy Markdown
Member

@charliermarsh charliermarsh commented Mar 31, 2026

Summary

When a TypedDict inherits from another class, for each field, the child has to preserve the same value and the same Required / NotRequired classification. We now enforce these requirements.

For example, this isn't allowed:

from typing import Literal, TypedDict

class Base(TypedDict):
    type: int

class Child(Base):
    type: Literal[1]  # This is an error.

def mutate(x: Base) -> None:
    x["type"] = 2

c: Child = {"type": 1}
mutate(c)  # `c` no longer satisfies `Child`.

@astral-sh-bot astral-sh-bot bot added the ty Multi-file analysis & type inference label Mar 31, 2026
@astral-sh-bot
Copy link
Copy Markdown

astral-sh-bot bot commented Mar 31, 2026

Typing conformance results improved 🎉

The percentage of diagnostics emitted that were expected errors increased from 86.79% to 86.89%. The percentage of expected errors that received a diagnostic increased from 81.72% to 82.47%. The number of fully passing files improved from 70/132 to 72/132.

Summary

How are test cases classified?

Each test case represents one expected error annotation or a group of annotations sharing a tag. Counts are per test case, not per diagnostic — multiple diagnostics on the same line count as one. Required annotations (E) are true positives when ty flags the expected location and false negatives when it does not. Optional annotations (E?) are true positives when flagged but true negatives (not false negatives) when not. Tagged annotations (E[tag]) require ty to flag exactly one of the tagged lines; tagged multi-annotations (E[tag+]) allow any number up to the tag count. Flagging unexpected locations counts as a false positive.

Metric Old New Diff Outcome
True Positives 867 875 +8 ⏫ (✅)
False Positives 132 132 +0
False Negatives 194 186 -8 ⏬ (✅)
Total Diagnostics 1052 1060 +8
Precision 86.79% 86.89% +0.10% ⏫ (✅)
Recall 81.72% 82.47% +0.75% ⏫ (✅)
Passing Files 70/132 72/132 +2 ⏫ (✅)

Test file breakdown

2 files altered
File True Positives False Positives False Negatives Status
typeddicts_readonly_inheritance.py 11 (+6) ✅ 0 0 (-6) ✅ ✅ Newly Passing 🎉
typeddicts_inheritance.py 3 (+2) ✅ 0 0 (-2) ✅ ✅ Newly Passing 🎉
Total (all files) 875 (+8) ✅ 132 186 (-8) ✅ 72/132

True positives added (8)

8 diagnostics
Test case Diff

typeddicts_inheritance.py:65

+error[invalid-typed-dict-field] Cannot overwrite TypedDict field `x` while merging base classes: Inherited mutable field type `str` is incompatible with `int`

typeddicts_inheritance.py:55

+error[invalid-typed-dict-field] Cannot overwrite TypedDict field `x`: Inherited mutable field type `str` is incompatible with `int`

typeddicts_readonly_inheritance.py:106

+error[invalid-typed-dict-field] Cannot overwrite TypedDict field `c`: Required inherited fields cannot be redeclared as `NotRequired`

typeddicts_readonly_inheritance.py:119

+error[invalid-typed-dict-field] Cannot overwrite TypedDict field `x` while merging base classes: Inherited mutable field type `int | float` is incompatible with `int`

typeddicts_readonly_inheritance.py:132

+error[invalid-typed-dict-field] Cannot overwrite TypedDict field `x` while merging base classes: Required inherited fields cannot be redeclared as `NotRequired`

typeddicts_readonly_inheritance.py:50

+error[invalid-typed-dict-field] Cannot overwrite TypedDict field `alt`: Inherited read-only field type `list[str | int]` is not assignable from `list[str]`

typeddicts_readonly_inheritance.py:94

+error[invalid-typed-dict-field] Cannot overwrite TypedDict field `a`: Mutable inherited fields cannot be redeclared as read-only

typeddicts_readonly_inheritance.py:98

+error[invalid-typed-dict-field] Cannot overwrite TypedDict field `a`: Required inherited fields cannot be redeclared as `NotRequired`

@astral-sh-bot
Copy link
Copy Markdown

astral-sh-bot bot commented Mar 31, 2026

Memory usage report

Summary

Project Old New Diff Outcome
trio 117.73MB 117.75MB +0.02% (18.89kB)
prefect 716.82MB 716.83MB +0.00% (14.12kB)
sphinx 264.62MB 264.63MB +0.00% (7.55kB)
flake8 47.99MB 47.99MB -

Significant changes

Click to expand detailed breakdown

trio

Name Old New Diff Outcome
StaticClassLiteral<'db>::fields_ 0.00B 14.58kB +14.58kB (new)
class_based_items 0.00B 3.62kB +3.62kB (new)
StaticClassLiteral<'db>::fields_::interned_arguments 0.00B 400.00B +400.00B (new)
infer_scope_types_impl 4.79MB 4.79MB +0.01% (312.00B)

prefect

Name Old New Diff Outcome
StaticClassLiteral<'db>::fields_ 118.11kB 127.18kB +7.68% (9.07kB)
class_based_items 7.03kB 9.94kB +41.33% (2.91kB)
infer_scope_types_impl 54.27MB 54.28MB +0.00% (1.76kB)
StaticClassLiteral<'db>::fields_::interned_arguments 7.66kB 8.05kB +5.10% (400.00B)

sphinx

Name Old New Diff Outcome
StaticClassLiteral<'db>::fields_ 22.86kB 27.93kB +22.14% (5.06kB)
class_based_items 200.00B 1.87kB +856.00% (1.67kB)
StaticClassLiteral<'db>::fields_::interned_arguments 2.19kB 2.66kB +21.43% (480.00B)
infer_scope_types_impl 15.51MB 15.51MB +0.00% (360.00B)

@astral-sh-bot
Copy link
Copy Markdown

astral-sh-bot bot commented Mar 31, 2026

ecosystem-analyzer results

Lint rule Added Removed Changed
invalid-typed-dict-field 53 0 0
unused-type-ignore-comment 0 1 0
Total 53 1 0
Raw diff (54 changes)
bokeh (https://github.com/bokeh/bokeh)
+ src/bokeh/models/axes.pyi:115:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `ticker`: Inherited mutable field type `Ticker` is incompatible with `BasicTicker`
+ src/bokeh/models/axes.pyi:116:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `formatter`: Inherited mutable field type `TickFormatter` is incompatible with `BasicTickFormatter`
+ src/bokeh/models/axes.pyi:125:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `ticker`: Inherited mutable field type `Ticker` is incompatible with `LogTicker`
+ src/bokeh/models/axes.pyi:126:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `formatter`: Inherited mutable field type `TickFormatter` is incompatible with `LogTickFormatter`
+ src/bokeh/models/axes.pyi:135:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `ticker`: Inherited mutable field type `Ticker` is incompatible with `CategoricalTicker`
+ src/bokeh/models/axes.pyi:136:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `formatter`: Inherited mutable field type `TickFormatter` is incompatible with `CategoricalTickFormatter`
+ src/bokeh/models/axes.pyi:149:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `ticker`: Inherited mutable field type `BasicTicker` is incompatible with `DatetimeTicker`
+ src/bokeh/models/axes.pyi:150:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `formatter`: Inherited mutable field type `BasicTickFormatter` is incompatible with `DatetimeTickFormatter`
+ src/bokeh/models/axes.pyi:159:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `ticker`: Inherited mutable field type `BasicTicker` is incompatible with `MercatorTicker`
+ src/bokeh/models/axes.pyi:160:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `formatter`: Inherited mutable field type `BasicTickFormatter` is incompatible with `MercatorTickFormatter`
+ src/bokeh/models/axes.pyi:169:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `ticker`: Inherited mutable field type `BasicTicker` is incompatible with `TimedeltaTicker`
+ src/bokeh/models/axes.pyi:170:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `formatter`: Inherited mutable field type `BasicTickFormatter` is incompatible with `TimedeltaTickFormatter`
+ src/bokeh/plotting/_figure.pyi:107:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `x_range`: Inherited mutable field type `Range` is incompatible with `Range | tuple[int | float, int | float] | tuple[str | date, str | date] | ... omitted 3 union elements`
+ src/bokeh/plotting/_figure.pyi:108:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `y_range`: Inherited mutable field type `Range` is incompatible with `Range | tuple[int | float, int | float] | tuple[str | date, str | date] | ... omitted 3 union elements`

discord.py (https://github.com/Rapptz/discord.py)
+ discord/types/components.py:54:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[1]`
+ discord/types/components.py:59:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[2]`
+ discord/types/components.py:87:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[3]`
+ discord/types/components.py:92:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[5]`
+ discord/types/components.py:97:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[6]`
+ discord/types/components.py:102:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[7]`
+ discord/types/components.py:107:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[8]`
+ discord/types/components.py:113:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[4]`
+ discord/types/components.py:125:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[3, 5, 6, 7, 8]`
+ discord/types/components.py:133:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[9]`
+ discord/types/components.py:139:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[10]`
+ discord/types/components.py:156:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[11]`
+ discord/types/components.py:169:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[12]`
+ discord/types/components.py:174:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[13]`
+ discord/types/components.py:182:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[14]`
+ discord/types/components.py:188:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[17]`
+ discord/types/components.py:195:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[18]`
+ discord/types/components.py:202:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[19]`
+ discord/types/components.py:210:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[21]`
+ discord/types/components.py:220:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[22]`
+ discord/types/components.py:232:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[23]`
+ discord/types/interactions.py:214:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[4]`
+ discord/types/interactions.py:220:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[3, 5, 6, 7, 8]`
+ discord/types/interactions.py:226:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[19]`
+ discord/types/interactions.py:232:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[21]`
+ discord/types/interactions.py:234:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `id`: Mutable inherited `NotRequired` fields cannot be redeclared as required
+ discord/types/interactions.py:239:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[22]`
+ discord/types/interactions.py:241:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `id`: Mutable inherited `NotRequired` fields cannot be redeclared as required
+ discord/types/interactions.py:246:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[23]`
+ discord/types/interactions.py:248:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `id`: Mutable inherited `NotRequired` fields cannot be redeclared as required
+ discord/types/interactions.py:268:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[10]`
+ discord/types/interactions.py:273:5 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `type`: Inherited mutable field type `int` is incompatible with `Literal[18]`
+ discord/types/scheduled_event.py:84:7 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `user_count` while merging base classes: Required inherited fields cannot be redeclared as `NotRequired`
+ discord/types/scheduled_event.py:87:7 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `user_count` while merging base classes: Required inherited fields cannot be redeclared as `NotRequired`
+ discord/types/scheduled_event.py:90:7 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `user_count` while merging base classes: Required inherited fields cannot be redeclared as `NotRequired`
+ discord/ext/commands/hybrid.py:61:9 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `description`: Inherited mutable field type `str` is incompatible with `str | locale_str`
+ discord/ext/commands/hybrid.py:69:9 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `description`: Inherited mutable field type `str | locale_str` is incompatible with `str`
+ discord/ext/commands/hybrid.py:73:9 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `description`: Inherited mutable field type `str` is incompatible with `str | locale_str`

hydra-zen (https://github.com/mit-ll-responsible-ai/hydra-zen)
- src/hydra_zen/typing/_implementations.py:441:30 warning[unused-type-ignore-comment] Unused blanket `type: ignore` directive

meson (https://github.com/mesonbuild/meson)
+ mesonbuild/modules/python.py:55:9 error[invalid-typed-dict-field] Cannot overwrite TypedDict field `install_dir`: Inherited mutable field type `list[str | bool]` is incompatible with `str | bool | None`

Full report with detailed diff (timing results)

@charliermarsh charliermarsh force-pushed the charlie/typed-dict-validation branch from 831ef5c to 49ff753 Compare March 31, 2026 21:32
@charliermarsh
Copy link
Copy Markdown
Member Author

Codex believes that the ecosystem diagnostics are all true positives.

@charliermarsh charliermarsh force-pushed the charlie/typed-dict-validation branch 2 times, most recently from 8918d87 to efdfe4e Compare March 31, 2026 21:45
diagnostic.info("Only annotated declarations (`<name>: <type>`) are allowed.");
}
}
}
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(This was moved to validate_typed_dict_class_body.)

validate_typed_dict_field_overrides(context, class, class_node, direct_bases);
}

fn validate_typed_dict_class_body(context: &InferContext<'_, '_>, class_node: &ast::StmtClassDef) {
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method is just logic moved over from crates/ty_python_semantic/src/types/infer/deferred/static_class.rs.)

@charliermarsh charliermarsh force-pushed the charlie/typed-dict-validation branch from efdfe4e to 1dfe35a Compare March 31, 2026 21:52
@charliermarsh charliermarsh marked this pull request as ready for review March 31, 2026 21:52
Copy link
Copy Markdown
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(not a full review, just a couple of things from skimming)

@carljm carljm removed their request for review March 31, 2026 22:49
Copy link
Copy Markdown
Contributor

@sharkdp sharkdp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a complete review yet.

For a simple example like:

from typing import Collection, TypedDict
from typing_extensions import NotRequired, ReadOnly, Required

class Movie(TypedDict):
    name: str

class BadMovie(Movie):
    name: int

I get the following diagnostic:

image

which feels pretty overwhelming:

  • I don't really understand where the primary (red) annotation points to?
  • Marking the whole TypedDict in a secondary annotation does not feel super useful?
  • For the child-class field, we take the full name: int range, but for the base-class field, we only highlight the name?

Maybe this also explains why we show a new diagnostic on hydra-zen here, even though there's a # type: ignore on the field declaration?

@charliermarsh charliermarsh marked this pull request as draft April 1, 2026 14:37
@charliermarsh
Copy link
Copy Markdown
Member Author

Will do a pass to clean those up and add some diagnostic coverage!

@charliermarsh charliermarsh force-pushed the charlie/typed-dict-validation branch from 8245bac to f5a0402 Compare April 1, 2026 16:18
@charliermarsh
Copy link
Copy Markdown
Member Author

Okay, I think this is better:

Screenshot 2026-04-01 at 12 19 58 PM

@charliermarsh charliermarsh marked this pull request as ready for review April 1, 2026 16:21
@astral-sh-bot astral-sh-bot bot requested a review from sharkdp April 1, 2026 16:21
@charliermarsh charliermarsh force-pushed the charlie/typed-dict-validation branch from f5a0402 to 74ef839 Compare April 1, 2026 16:25
Copy link
Copy Markdown
Contributor

@sharkdp sharkdp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the update. Very nice and clear implementation!

I extended the test suite a bit and added some explanations — mostly for my own understanding, but hopefully doesn't hurt.

Comment on lines +196 to +197
impl<'db> TypedDictFieldOverrideReason<'db> {
fn from_fields(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this :-)

@sharkdp
Copy link
Copy Markdown
Contributor

sharkdp commented Apr 2, 2026

(I will do the rebase)

@charliermarsh
Copy link
Copy Markdown
Member Author

Thank you!

@sharkdp sharkdp force-pushed the charlie/typed-dict-validation branch from f49ef2b to 45b613d Compare April 2, 2026 20:00
@sharkdp sharkdp merged commit de6d6be into main Apr 2, 2026
51 checks passed
@sharkdp sharkdp deleted the charlie/typed-dict-validation branch April 2, 2026 20:10
carljm added a commit that referenced this pull request Apr 3, 2026
* main:
  Document adding fixes in CONTRIBUTING.md (#24393)
  Sort formatter diagnostics in snapshots (#24375)
  [`pyupgrade`] Fix panic caused by handling of octals in `UP012` (#24390)
  Upgrade to nix v0.31.2 (#24385)
  Strip form feeds from indent passed to `dedent_to` (#24381)
  add recent move of the `deferred` submodule to `.git-blame-ignore-revs` (#24379)
  [ty] Fix extra_items TypedDict tests (#24367)
  [ty] Use `infer_type_expression` for validating PEP-613 type aliases (#24370)
  [`flake8-simplify`] Make the fix for `collapsible-if` (`SIM102`) safe in `preview` (#24371)
  [ty] Validate TypedDict fields when subclassing (#24338)
  [ty] pass type context to sequence literals in binary operations (#24197)
  Add release environment to notify-dependents job (#24372)
  Bump 0.15.9 (#24369)
  [ty] Move the `deferred` submodule inside `infer/builder` (#24368)
  [ty] Infer the `extra_items` keyword argument to class-based TypedDicts as an annotation expression (#24362)
  [ty] Validate type qualifiers in functional TypedDict fields and the `extra_items` keyword to functional TypedDicts (#24360)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants