Skip to content

[ty] fix too-many-cycle panics when inferring literal type loop variables#23875

Merged
mtshiba merged 3 commits intoastral-sh:mainfrom
mtshiba:fix-3011
Mar 13, 2026
Merged

[ty] fix too-many-cycle panics when inferring literal type loop variables#23875
mtshiba merged 3 commits intoastral-sh:mainfrom
mtshiba:fix-3011

Conversation

@mtshiba
Copy link
Collaborator

@mtshiba mtshiba commented Mar 10, 2026

Summary

Fixes astral-sh/ty#3011

See the comment in #3011 for an analysis of this bug.

We can simply add a recursively_defined field to the literal value type to prevent this bug. But there's one problem. Simply adding a field will increase the size of LiteralValueType, which will in turn increase the size of Type.
To address this, I decided to use a technique already used by LiteralValueTypeInner: embedding the flags as an enum discriminant. It's a bit of an ugly solution, but I couldn't find better way.

Test Plan

new corpus test

@mtshiba mtshiba added the ty Multi-file analysis & type inference label Mar 10, 2026
@astral-sh-bot
Copy link

astral-sh-bot bot commented Mar 10, 2026

Typing conformance results

No changes detected ✅

Current numbers
The percentage of diagnostics emitted that were expected errors held steady at 85.29%. The percentage of expected errors that received a diagnostic held steady at 78.13%. The number of fully passing files held steady at 64/132.

@astral-sh-bot
Copy link

astral-sh-bot bot commented Mar 10, 2026

mypy_primer results

No ecosystem changes detected ✅

@astral-sh-bot
Copy link

astral-sh-bot bot commented Mar 10, 2026

Memory usage report

Summary

Project Old New Diff Outcome
prefect 704.98MB 705.14MB +0.02% (171.34kB)
sphinx 265.15MB 265.23MB +0.03% (83.26kB)
trio 117.74MB 117.75MB +0.00% (3.37kB)
flake8 47.89MB 47.89MB +0.00% (560.00B)

Significant changes

Click to expand detailed breakdown

prefect

Name Old New Diff Outcome
infer_expression_types_impl 60.74MB 60.78MB +0.06% (34.23kB)
infer_expression_type_impl 14.31MB 14.35MB +0.23% (34.04kB)
infer_definition_types 88.85MB 88.88MB +0.03% (25.68kB)
all_narrowing_constraints_for_expression 6.98MB 7.00MB +0.21% (15.35kB)
Type<'db>::try_call_dunder_get_ 10.50MB 10.51MB +0.12% (13.39kB)
all_negative_narrowing_constraints_for_expression 2.59MB 2.60MB +0.44% (11.53kB)
Type<'db>::class_member_with_policy_ 17.31MB 17.32MB +0.04% (6.54kB)
loop_header_reachability 427.58kB 432.70kB +1.20% (5.12kB)
is_redundant_with_impl::interned_arguments 5.35MB 5.36MB +0.09% (5.07kB)
is_redundant_with_impl 5.57MB 5.57MB +0.06% (3.59kB)
Type<'db>::class_member_with_policy_::interned_arguments 9.36MB 9.37MB +0.03% (3.25kB)
Type<'db>::member_lookup_with_policy_ 15.40MB 15.41MB +0.02% (3.19kB)
Type<'db>::try_call_dunder_get_::interned_arguments 3.01MB 3.01MB +0.09% (2.74kB)
Type<'db>::member_lookup_with_policy_::interned_arguments 5.53MB 5.53MB +0.04% (2.44kB)
BoundMethodType 1.40MB 1.40MB +0.15% (2.11kB)
... 6 more

sphinx

Name Old New Diff Outcome
infer_definition_types 24.01MB 24.03MB +0.06% (14.29kB)
infer_expression_types_impl 21.50MB 21.51MB +0.04% (9.60kB)
infer_expression_type_impl 3.20MB 3.21MB +0.28% (9.19kB)
Type<'db>::try_call_dunder_get_ 4.94MB 4.95MB +0.18% (8.95kB)
Type<'db>::class_member_with_policy_ 7.55MB 7.55MB +0.10% (7.84kB)
all_narrowing_constraints_for_expression 2.33MB 2.34MB +0.29% (6.82kB)
is_redundant_with_impl::interned_arguments 2.07MB 2.07MB +0.21% (4.38kB)
all_negative_narrowing_constraints_for_expression 1.01MB 1.01MB +0.33% (3.41kB)
Type<'db>::class_member_with_policy_::interned_arguments 3.99MB 3.99MB +0.08% (3.35kB)
Type<'db>::member_lookup_with_policy_ 6.09MB 6.10MB +0.05% (3.03kB)
is_redundant_with_impl 1.81MB 1.81MB +0.16% (2.93kB)
Type<'db>::member_lookup_with_policy_::interned_arguments 2.54MB 2.54MB +0.10% (2.54kB)
IntersectionType 894.12kB 896.59kB +0.28% (2.46kB)
loop_header_reachability 378.56kB 380.67kB +0.56% (2.11kB)
IntersectionType<'db>::from_two_elements_ 203.44kB 201.53kB -0.94% (1.91kB)
... 10 more

trio

Name Old New Diff Outcome
infer_definition_types 7.57MB 7.57MB +0.01% (912.00B)
is_redundant_with_impl 479.27kB 479.86kB +0.12% (600.00B)
is_redundant_with_impl::interned_arguments 538.83kB 539.26kB +0.08% (440.00B)
infer_expression_types_impl 7.06MB 7.06MB +0.00% (360.00B)
infer_expression_type_impl 1.43MB 1.43MB +0.02% (336.00B)
Type<'db>::class_member_with_policy_ 1.97MB 1.97MB +0.01% (280.00B)
loop_header_reachability 133.50kB 133.74kB +0.18% (252.00B)
UnionType 303.23kB 303.09kB -0.05% (144.00B)
Type<'db>::member_lookup_with_policy_ 1.66MB 1.66MB +0.01% (112.00B)
all_narrowing_constraints_for_expression 600.50kB 600.60kB +0.02% (108.00B)
Type<'db>::class_member_with_policy_::interned_arguments 1.10MB 1.10MB +0.01% (104.00B)
Type<'db>::member_lookup_with_policy_::interned_arguments 868.77kB 868.87kB +0.01% (104.00B)
all_negative_narrowing_constraints_for_expression 184.75kB 184.73kB -0.01% (12.00B)

flake8

Name Old New Diff Outcome
is_redundant_with_impl::interned_arguments 140.16kB 140.34kB +0.12% (176.00B)
IntersectionType 72.63kB 72.80kB +0.23% (168.00B)
infer_definition_types 1.86MB 1.86MB +0.01% (120.00B)
is_redundant_with_impl 140.27kB 140.37kB +0.07% (96.00B)

@mtshiba mtshiba marked this pull request as ready for review March 10, 2026 16:00
Copy link
Member

@dcreager dcreager left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simply adding a field will increase the size of LiteralValueType, which will in turn increase the size of Type.
To address this, I decided to use a technique already used by LiteralValueTypeInner: embedding the flags as an enum discriminant. It's a bit of an ugly solution, but I couldn't find better way.

The max payload is for IntLiteral, which contains two u32s. My reading is that the u8 discriminant is padded to 32 bits, making the overall size of the type 12 bytes. That padding means that there should be room to squeeze an 8-bit bitflags or similar to store "promotable" and "recursively defined". (You'd have to add it to each variant of the enum, rather than putting it in an extra field of the struct wrapper, to make sure that it gets placed into the discriminant's padding.) i.e. in the playground this shows 12 bytes for both approaches.

(I'm fairly certain I got the types right in the playground. You could also add a static assertion like we have here to verify that the size of LiteralValueTypeInner stays consistent at 12 bytes.)

@carljm
Copy link
Contributor

carljm commented Mar 12, 2026

I took some time to understand why literal types are special here - why don't we need to carry this flag on all types? I guess it's because the recursively-defined flag in the end only has an effect on the maximum size of unions of literals. So there's no scenario where it would matter for another type to carry it.

@carljm carljm removed their request for review March 12, 2026 00:52
@mtshiba mtshiba merged commit 7b67825 into astral-sh:main Mar 13, 2026
51 checks passed
@mtshiba mtshiba deleted the fix-3011 branch March 13, 2026 05:55
carljm added a commit that referenced this pull request Mar 13, 2026
* main: (94 commits)
  Fix shell injection via `shell=True` in subprocess calls (#23894)
  [ty] Refactor `relation.rs` to store state on a struct rather than passing around 7 arguments every time we recurse (#23837)
  Don't return code actions for non-Python documents (#23905)
  [ty] Make the default database truly statically infallible (#23929)
  [ty] Add `Download` button to ty playground which creates a zip export (#23478)
  [ty] Respect `kw_only` overwrites in dataclasses (#23930)
  [ty] Clarify in diagnostics that `from __future__ import annotations` only stringifies type annotations (#23928)
  [ty]  Add a `Copy Markdown` button to playground (#23002)
  [ty] Fix folding range classification of lines starting with `#` (#23831)
  [ty] Fix folding ranges for notebooks (#23830)
  [ty] fix too-many-cycle panics when inferring literal type loop variables (#23875)
  Add `RegularCallableTypeOf` and `into_regular_callable` in `ty_extensions` (#23909)
  [ty] treat properties as full structural types (#23925)
  [ty] Avoid duplicated work during multi-inference (#23923)
  [ty]: make `possibly-missing-attribute` ignored by default
  [ty]: split out `possibly-missing-submodule` from `possibly-missing-attribute`
  Update astral-sh/setup-uv action to v7.5.0 (#23922)
  [ty] Show truthiness in ConstraintSet display and simplify falsy error message (#23913)
  Bump 0.15.6 (#23919)
  [ty] Narrow type context during collection literal inference (#23844)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

panic: "too many cycle iterations" when checking simple loop

4 participants