[dynamo] Fix struct.pack polyfill signature mismatch on Python 3.15#185403
[dynamo] Fix struct.pack polyfill signature mismatch on Python 3.15#185403zklaus wants to merge 1 commit into
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/185403
Note: Links to docs will display an error until the docs builds have been completed. ❌ 6 New Failures, 1 Unrelated Failure, 9 Unclassified FailuresAs of commit 729d443 with merge base e0ff4ad ( NEW FAILURES - The following jobs have failed:
UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:
UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
@claude explain those CI failures, check if they are related, and review the changes in this PR |
|
Claude finished @jansel's task in 2m 17s —— View job CI Failure Analysis & Code Review
CI Failures6 New Failures (aarch64 py3.15/3.15t test jobs): These are NOT caused by this PR. The failure is: The 6 Unclassified Failures (libtorch build/extract jobs): Also NOT caused by this PR. The libtorch Summary: None of the 12 failures are related to this PR's one-line rename. Code ReviewThe change ( -def pack(fmt: bytes | str, /, *v: Any) -> bytes:
- return struct.pack(fmt, *v)
+def pack(format: bytes | str, /, *v: Any) -> bytes:
+ return struct.pack(format, *v)Verdict: LGTM. This is a correct and minimal fix. Why it's correct: On Python 3.15, No concerns:
|
|
@pytorchbot rebase |
|
@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here |
Python 3.15 gives the builtin struct.pack an introspectable signature, (format, /, *values). substitute_in_graph validates that a polyfill's signature matches the original builtin's, comparing positional-only parameter names. The polyfill declared its positional-only parameter as fmt, which mismatched format, so substitute_in_graph raised TypeError at import time. Because the polyfills loader eagerly imports every polyfill module, this broke `import torch._dynamo` entirely on 3.15 (the binary smoke test imports it), failing all py3.15/3.15t binary test jobs. On Python <= 3.14 the builtin had no introspectable signature, so the check was skipped and the mismatch went unnoticed. Rename the parameter to format to match the builtin; the sibling unpack polyfill already uses format. The var-positional name is irrelevant here because substitute_in_graph ignores VAR_POSITIONAL parameter names. Test Plan: Local introspection confirms no regression on older Pythons and the canonical name on the versions that do expose a signature: ``` $ python --version Python 3.10.19 $ python -c "import inspect, struct; print(inspect.signature(struct.pack))" ValueError: no signature found for builtin <built-in function pack> $ python -c "import inspect, struct; print(inspect.signature(struct.unpack))" (format, buffer, /) ``` On <= 3.14 struct.pack has no signature, so the substitute_in_graph check is skipped and the rename is a no-op there. The 3.15 failure reproduces only in the py3.15 binary smoke test (manywheel-py3_15-*-test, which runs `import torch._dynamo`); validate with ciflow/binaries. Authored by Claude.
|
Successfully rebased |
7ec9c64 to
729d443
Compare
|
@claude explain those CI failures, are they related? |
|
Claude finished @jansel's task in 2m 27s —— View job CI Failure Analysis
None of the CI failures are related to this PR. All are pre-existing infrastructure issues. 6 New Failures (aarch64 py3.15 test jobs)Root cause from the job log: The 9 Unclassified Failures (libtorch build/extract jobs)The 1 Unstable (ROCm mi355)Marked as unstable/flaky by DrCI. Unrelated to this PR. |
|
@pytorchbot merge -i |
….15 (#185682) ## Summary Python 3.15 will expose an introspectable signature for `functools.reduce` using a PEP 661 sentinel default: `(function, iterable, /, initial=functools._initial_missing)`. The CPython change is [#149591](python/cpython#149591), which merged into CPython main on 2026-05-10, after 3.15.0b1 was tagged. So on 3.15.0b1 the default is still `<unrepresentable>` and `inspect.signature()` raises `ValueError`, which `substitute_in_graph` swallows and skips the check. From 3.15.0b2 the check will run. When it runs, it compares positional parameter names, keyword-only names, and default values. The polyfill declares its own local sentinel: ```python _initial_missing = object() @substitute_in_graph(functools.reduce) def reduce(function, iterable, initial=_initial_missing, /): ... ``` That local `object()` is a different instance from `functools._initial_missing`, so the defaults dict compares unequal and `substitute_in_graph` raises: ``` File ".../torch/_dynamo/polyfills/functools.py", line N, in <module> @substitute_in_graph(functools.reduce) TypeError: Signature mismatch between <built-in function reduce> and <function reduce at 0x...>: (function, iterable, /, initial=_initial_missing) != (function, iterable, initial=<object object at 0x...>, /) ``` The polyfills loader imports every polyfill module, so this will break `import torch._dynamo` entirely on Python 3.15.0b2+. Same failure mode as #185403's `struct.pack` fix, just one step ahead of the next beta. The fix imports `functools._initial_missing` instead of declaring a local one. The polyfill's internal `if initial is _initial_missing` identity check still works because both sides now reference the same object. No behavior change on any Python version. `functools._initial_missing` has existed in `functools` since well before 3.10 (it's used by the pure-Python `functools.reduce` fallback), so the import is safe on every supported version. Part of #184352 (Python 3.15 support). Similar to #185403. ## Test plan Verified on a locally-built Python 3.15-dev (CPython main at `heads/3.15:863c7e0`, which includes [#149591](python/cpython#149591)). The upstream signature is exposed natively: ``` $ python3.15 -c "import functools, inspect; print(inspect.signature(functools.reduce))" (function, iterable, /, initial=_initial_missing) $ python3.15 -c "import functools, inspect; sig = inspect.signature(functools.reduce); print(sig.parameters['initial'].default is functools._initial_missing)" True ``` Extracted the actual `substitute_in_graph` from `torch/_dynamo/decorators.py` and ran it against both polyfill versions with no monkey-patching. Unmodified polyfill (`_initial_missing = object()` declared locally): ``` TypeError: Signature mismatch between <built-in function reduce> and <function unmodified_reduce at 0x...>: (function, iterable, /, initial=_initial_missing) != (function, iterable, initial=<object object at 0x...>, /) ``` Fixed polyfill (`from functools import _initial_missing`): no `TypeError` at the signature check. Regression on Python 3.12.13 (oldest currently supported): ``` $ python3 -c "import inspect, functools; inspect.signature(functools.reduce)" ValueError: no signature found for builtin <built-in function reduce> ``` The signature check is silently skipped on Python <= 3.14, so the fix is a no-op there. `functools._initial_missing` is confirmed present on 3.12.13. Eager-vs-polyfill parity verified on every input shape: ``` reduce(lambda a, b: a+b, [1,2,3,4]) == 10 (matches functools.reduce) reduce(lambda a, b: a+b, [1,2,3], 100) == 106 (matches) reduce(lambda a, b: a+b, []) -> TypeError("reduce() of empty iterable with no initial value") (matches) reduce(lambda a, b: a+b, [], 99) == 99 (matches) ``` Pre-commit checks: `python3 -m py_compile torch/_dynamo/polyfills/functools.py` and `git diff --staged --check` both clean. Pull Request resolved: #185682 Approved by: https://github.com/ezyang Co-authored-by: Edward Z. Yang via mergedog <ezyang@meta.com>
…ytorch#185403) ## Summary Python 3.15 gives the builtin `struct.pack` an introspectable signature, `(format, /, *values)`. `substitute_in_graph` validates that a polyfill's signature matches the original builtin's, comparing positional-only parameter names. The `struct.pack` polyfill declared its positional-only parameter as `fmt`, mismatching `format`, so `substitute_in_graph` raised `TypeError` at import time: ``` File ".../torch/_dynamo/polyfills/struct.py", line 20, in <module> @substitute_in_graph(struct.pack, can_constant_fold_through=True) TypeError: Signature mismatch between <built-in function pack> and <function pack ...>: (format, /, *values) != (fmt: 'bytes | str', /, *v: 'Any') -> 'bytes' ``` The polyfills loader eagerly imports every polyfill module, so this broke `import torch._dynamo` entirely on 3.15, failing every py3.15/3.15t binary test job (the smoke test imports `torch._dynamo`). On Python <= 3.14 the builtin has no introspectable signature, so the check is skipped and the mismatch went unnoticed. The fix renames the parameter to `format` to match the builtin. The sibling `unpack` polyfill already uses `format`. The var-positional name (`*v`) is irrelevant here because `substitute_in_graph` ignores `VAR_POSITIONAL` parameter names. Part of pytorch#184352 (Python 3.15 support). ## Test plan Local introspection confirms there is no regression on older Pythons and that `format` is the canonical name where a signature is exposed: ``` $ python --version Python 3.10.19 $ python -c "import inspect, struct; print(inspect.signature(struct.pack))" ValueError: no signature found for builtin <built-in function pack> $ python -c "import inspect, struct; print(inspect.signature(struct.unpack))" (format, buffer, /) ``` On <= 3.14 `struct.pack` has no signature, so the `substitute_in_graph` check is skipped and the rename is a no-op there. The 3.15 failure reproduces only in the py3.15 binary smoke test (`manywheel-py3_15-*-test`); needs `ciflow/binaries` to validate. Authored by Claude. Pull Request resolved: pytorch#185403 Approved by: https://github.com/guilhermeleobas, https://github.com/rtimpe
…ytorch#185403) ## Summary Python 3.15 gives the builtin `struct.pack` an introspectable signature, `(format, /, *values)`. `substitute_in_graph` validates that a polyfill's signature matches the original builtin's, comparing positional-only parameter names. The `struct.pack` polyfill declared its positional-only parameter as `fmt`, mismatching `format`, so `substitute_in_graph` raised `TypeError` at import time: ``` File ".../torch/_dynamo/polyfills/struct.py", line 20, in <module> @substitute_in_graph(struct.pack, can_constant_fold_through=True) TypeError: Signature mismatch between <built-in function pack> and <function pack ...>: (format, /, *values) != (fmt: 'bytes | str', /, *v: 'Any') -> 'bytes' ``` The polyfills loader eagerly imports every polyfill module, so this broke `import torch._dynamo` entirely on 3.15, failing every py3.15/3.15t binary test job (the smoke test imports `torch._dynamo`). On Python <= 3.14 the builtin has no introspectable signature, so the check is skipped and the mismatch went unnoticed. The fix renames the parameter to `format` to match the builtin. The sibling `unpack` polyfill already uses `format`. The var-positional name (`*v`) is irrelevant here because `substitute_in_graph` ignores `VAR_POSITIONAL` parameter names. Part of pytorch#184352 (Python 3.15 support). ## Test plan Local introspection confirms there is no regression on older Pythons and that `format` is the canonical name where a signature is exposed: ``` $ python --version Python 3.10.19 $ python -c "import inspect, struct; print(inspect.signature(struct.pack))" ValueError: no signature found for builtin <built-in function pack> $ python -c "import inspect, struct; print(inspect.signature(struct.unpack))" (format, buffer, /) ``` On <= 3.14 `struct.pack` has no signature, so the `substitute_in_graph` check is skipped and the rename is a no-op there. The 3.15 failure reproduces only in the py3.15 binary smoke test (`manywheel-py3_15-*-test`); needs `ciflow/binaries` to validate. Authored by Claude. Pull Request resolved: pytorch#185403 Approved by: https://github.com/guilhermeleobas, https://github.com/rtimpe
….15 (pytorch#185682) ## Summary Python 3.15 will expose an introspectable signature for `functools.reduce` using a PEP 661 sentinel default: `(function, iterable, /, initial=functools._initial_missing)`. The CPython change is [pytorch#149591](python/cpython#149591), which merged into CPython main on 2026-05-10, after 3.15.0b1 was tagged. So on 3.15.0b1 the default is still `<unrepresentable>` and `inspect.signature()` raises `ValueError`, which `substitute_in_graph` swallows and skips the check. From 3.15.0b2 the check will run. When it runs, it compares positional parameter names, keyword-only names, and default values. The polyfill declares its own local sentinel: ```python _initial_missing = object() @substitute_in_graph(functools.reduce) def reduce(function, iterable, initial=_initial_missing, /): ... ``` That local `object()` is a different instance from `functools._initial_missing`, so the defaults dict compares unequal and `substitute_in_graph` raises: ``` File ".../torch/_dynamo/polyfills/functools.py", line N, in <module> @substitute_in_graph(functools.reduce) TypeError: Signature mismatch between <built-in function reduce> and <function reduce at 0x...>: (function, iterable, /, initial=_initial_missing) != (function, iterable, initial=<object object at 0x...>, /) ``` The polyfills loader imports every polyfill module, so this will break `import torch._dynamo` entirely on Python 3.15.0b2+. Same failure mode as pytorch#185403's `struct.pack` fix, just one step ahead of the next beta. The fix imports `functools._initial_missing` instead of declaring a local one. The polyfill's internal `if initial is _initial_missing` identity check still works because both sides now reference the same object. No behavior change on any Python version. `functools._initial_missing` has existed in `functools` since well before 3.10 (it's used by the pure-Python `functools.reduce` fallback), so the import is safe on every supported version. Part of pytorch#184352 (Python 3.15 support). Similar to pytorch#185403. ## Test plan Verified on a locally-built Python 3.15-dev (CPython main at `heads/3.15:863c7e0`, which includes [pytorch#149591](python/cpython#149591)). The upstream signature is exposed natively: ``` $ python3.15 -c "import functools, inspect; print(inspect.signature(functools.reduce))" (function, iterable, /, initial=_initial_missing) $ python3.15 -c "import functools, inspect; sig = inspect.signature(functools.reduce); print(sig.parameters['initial'].default is functools._initial_missing)" True ``` Extracted the actual `substitute_in_graph` from `torch/_dynamo/decorators.py` and ran it against both polyfill versions with no monkey-patching. Unmodified polyfill (`_initial_missing = object()` declared locally): ``` TypeError: Signature mismatch between <built-in function reduce> and <function unmodified_reduce at 0x...>: (function, iterable, /, initial=_initial_missing) != (function, iterable, initial=<object object at 0x...>, /) ``` Fixed polyfill (`from functools import _initial_missing`): no `TypeError` at the signature check. Regression on Python 3.12.13 (oldest currently supported): ``` $ python3 -c "import inspect, functools; inspect.signature(functools.reduce)" ValueError: no signature found for builtin <built-in function reduce> ``` The signature check is silently skipped on Python <= 3.14, so the fix is a no-op there. `functools._initial_missing` is confirmed present on 3.12.13. Eager-vs-polyfill parity verified on every input shape: ``` reduce(lambda a, b: a+b, [1,2,3,4]) == 10 (matches functools.reduce) reduce(lambda a, b: a+b, [1,2,3], 100) == 106 (matches) reduce(lambda a, b: a+b, []) -> TypeError("reduce() of empty iterable with no initial value") (matches) reduce(lambda a, b: a+b, [], 99) == 99 (matches) ``` Pre-commit checks: `python3 -m py_compile torch/_dynamo/polyfills/functools.py` and `git diff --staged --check` both clean. Pull Request resolved: pytorch#185682 Approved by: https://github.com/ezyang Co-authored-by: Edward Z. Yang via mergedog <ezyang@meta.com>
Summary
Python 3.15 gives the builtin
struct.packan introspectable signature,(format, /, *values).substitute_in_graphvalidates that a polyfill's signature matches the original builtin's, comparing positional-only parameter names. Thestruct.packpolyfill declared its positional-only parameter asfmt, mismatchingformat, sosubstitute_in_graphraisedTypeErrorat import time:The polyfills loader eagerly imports every polyfill module, so this broke
import torch._dynamoentirely on 3.15, failing every py3.15/3.15t binary test job (the smoke test importstorch._dynamo). On Python <= 3.14 the builtin has no introspectable signature, so the check is skipped and the mismatch went unnoticed.The fix renames the parameter to
formatto match the builtin. The siblingunpackpolyfill already usesformat. The var-positional name (*v) is irrelevant here becausesubstitute_in_graphignoresVAR_POSITIONALparameter names.Part of #184352 (Python 3.15 support).
Test plan
Local introspection confirms there is no regression on older Pythons and that
formatis the canonical name where a signature is exposed:On <= 3.14
struct.packhas no signature, so thesubstitute_in_graphcheck is skipped and the rename is a no-op there. The 3.15 failure reproduces only in the py3.15 binary smoke test (manywheel-py3_15-*-test); needsciflow/binariesto validate.Authored by Claude.
cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @kadeng @chauhang @amjames @jataylo @azahed98