Skip to content

[dynamo] Fix struct.pack polyfill signature mismatch on Python 3.15#185403

Closed
zklaus wants to merge 1 commit into
pytorch:mainfrom
zklaus:fix-struct-pack-polyfill-py315
Closed

[dynamo] Fix struct.pack polyfill signature mismatch on Python 3.15#185403
zklaus wants to merge 1 commit into
pytorch:mainfrom
zklaus:fix-struct-pack-polyfill-py315

Conversation

@zklaus

@zklaus zklaus commented May 27, 2026

Copy link
Copy Markdown
Collaborator

Summary

Python 3.15 gives the builtin struct.pack an introspectable signature, (format, /, *values). substitute_in_graph validates that a polyfill's signature matches the original builtin's, comparing positional-only parameter names. The struct.pack polyfill declared its positional-only parameter as fmt, mismatching format, so substitute_in_graph raised TypeError at import time:

File ".../torch/_dynamo/polyfills/struct.py", line 20, in <module>
  @substitute_in_graph(struct.pack, can_constant_fold_through=True)
TypeError: Signature mismatch between <built-in function pack> and <function pack ...>:
  (format, /, *values) != (fmt: 'bytes | str', /, *v: 'Any') -> 'bytes'

The polyfills loader eagerly imports every polyfill module, so this broke import torch._dynamo entirely on 3.15, failing every py3.15/3.15t binary test job (the smoke test imports torch._dynamo). On Python <= 3.14 the builtin has no introspectable signature, so the check is skipped and the mismatch went unnoticed.

The fix renames the parameter to format to match the builtin. The sibling unpack polyfill already uses format. The var-positional name (*v) is irrelevant here because substitute_in_graph ignores VAR_POSITIONAL parameter names.

Part of #184352 (Python 3.15 support).

Test plan

Local introspection confirms there is no regression on older Pythons and that format is the canonical name where a signature is exposed:

$ python --version
Python 3.10.19
$ python -c "import inspect, struct; print(inspect.signature(struct.pack))"
ValueError: no signature found for builtin <built-in function pack>
$ python -c "import inspect, struct; print(inspect.signature(struct.unpack))"
(format, buffer, /)

On <= 3.14 struct.pack has no signature, so the substitute_in_graph check is skipped and the rename is a no-op there. The 3.15 failure reproduces only in the py3.15 binary smoke test (manywheel-py3_15-*-test); needs ciflow/binaries to validate.

Authored by Claude.

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @kadeng @chauhang @amjames @jataylo @azahed98

@pytorch-bot

pytorch-bot Bot commented May 27, 2026

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/185403

Note: Links to docs will display an error until the docs builds have been completed.

❌ 6 New Failures, 1 Unrelated Failure, 9 Unclassified Failures

As of commit 729d443 with merge base e0ff4ad (image):

NEW FAILURES - The following jobs have failed:

UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:

UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorch-bot pytorch-bot Bot added ciflow/inductor ciflow/torchtitan Run TorchTitan integration tests module: dynamo labels May 27, 2026
@zklaus zklaus added ciflow/binaries Trigger all binary build and upload jobs on the PR ciflow/trunk Trigger trunk jobs on your pull request labels May 27, 2026
@jansel

jansel commented May 28, 2026

Copy link
Copy Markdown
Contributor

@claude explain those CI failures, check if they are related, and review the changes in this PR

@claude

claude Bot commented May 28, 2026

Copy link
Copy Markdown

Claude finished @jansel's task in 2m 17s —— View job


CI Failure Analysis & Code Review

  • Investigate the 6 new aarch64 py3.15 test failures
  • Investigate the 6 unclassified libtorch build failures
  • Determine if failures are related to this PR
  • Review the code changes

CI Failures

6 New Failures (aarch64 py3.15/3.15t test jobs): These are NOT caused by this PR. The failure is:

ERROR: Package 'triton' requires a different Python: 3.15.0 not in '<3.15,>=3.10'

The triton==3.7.0+git88b227e2 wheel being installed has python_requires='>=3.10,<3.15', which rejects Python 3.15. This is a dependency compatibility issue unrelated to the struct.pack polyfill fix — it would fail regardless of whether this PR exists. The triton package needs to update its version bounds to support Python 3.15.

6 Unclassified Failures (libtorch build/extract jobs): Also NOT caused by this PR. The libtorch manywheel-cuda-cu1{26,30,32}-build jobs completed the wheel build successfully (Successfully built torch-2.13.0.dev20260527+cu126-cp315-cp315-linux_x86_64.whl) but were then canceled (##[error]The operation was canceled.) — likely a timeout during the wheel repacking step or a workflow-level cancellation. The downstream -extract jobs then failed because the upload artifact wasn't available (No files were found with the provided path). This is infrastructure flakiness, not a code issue.

Summary: None of the 12 failures are related to this PR's one-line rename.


Code Review

The change (torch/_dynamo/polyfills/struct.py):

-def pack(fmt: bytes | str, /, *v: Any) -> bytes:
-    return struct.pack(fmt, *v)
+def pack(format: bytes | str, /, *v: Any) -> bytes:
+    return struct.pack(format, *v)

Verdict: LGTM. This is a correct and minimal fix.

Why it's correct: On Python 3.15, struct.pack gains an introspectable signature (format, /, *values). The substitute_in_graph decorator (at torch/_dynamo/decorators.py:958-1007) validates that positional-only parameter names match between the builtin and the polyfill. It explicitly skips VAR_POSITIONAL names (so *v vs *values doesn't matter), but it does compare positional-only parameter names — hence fmt != format triggered the TypeError. On Python <= 3.14, inspect.signature(struct.pack) raises ValueError (no signature found), so the check is skipped entirely via the except ValueError: pass at line 961.

No concerns:

  • The rename is purely cosmetic — format is not a reserved keyword in Python (unlike type or class), so shadowing the builtin format() function inside this tiny scope is fine and matches the canonical name.
  • The sibling unpack polyfill already uses format as its parameter name, so this makes the file consistent.
  • No behavioral change on Python <= 3.14.

@jansel

jansel commented May 28, 2026

Copy link
Copy Markdown
Contributor

@pytorchbot rebase

@pytorchmergebot

Copy link
Copy Markdown
Collaborator

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

Python 3.15 gives the builtin struct.pack an introspectable signature,
(format, /, *values). substitute_in_graph validates that a polyfill's
signature matches the original builtin's, comparing positional-only
parameter names. The polyfill declared its positional-only parameter as
fmt, which mismatched format, so substitute_in_graph raised TypeError at
import time. Because the polyfills loader eagerly imports every polyfill
module, this broke `import torch._dynamo` entirely on 3.15 (the binary
smoke test imports it), failing all py3.15/3.15t binary test jobs. On
Python <= 3.14 the builtin had no introspectable signature, so the check
was skipped and the mismatch went unnoticed.

Rename the parameter to format to match the builtin; the sibling unpack
polyfill already uses format. The var-positional name is irrelevant here
because substitute_in_graph ignores VAR_POSITIONAL parameter names.

Test Plan:
Local introspection confirms no regression on older Pythons and the
canonical name on the versions that do expose a signature:

```
$ python --version
Python 3.10.19
$ python -c "import inspect, struct; print(inspect.signature(struct.pack))"
ValueError: no signature found for builtin <built-in function pack>
$ python -c "import inspect, struct; print(inspect.signature(struct.unpack))"
(format, buffer, /)
```

On <= 3.14 struct.pack has no signature, so the substitute_in_graph
check is skipped and the rename is a no-op there. The 3.15 failure
reproduces only in the py3.15 binary smoke test (manywheel-py3_15-*-test,
which runs `import torch._dynamo`); validate with ciflow/binaries.

Authored by Claude.
@pytorchmergebot

Copy link
Copy Markdown
Collaborator

Successfully rebased fix-struct-pack-polyfill-py315 onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout fix-struct-pack-polyfill-py315 && git pull --rebase)

@pytorchmergebot pytorchmergebot force-pushed the fix-struct-pack-polyfill-py315 branch from 7ec9c64 to 729d443 Compare May 28, 2026 08:46
@zklaus zklaus added the topic: not user facing topic category label May 28, 2026
@jansel

jansel commented May 29, 2026

Copy link
Copy Markdown
Contributor

@claude explain those CI failures, are they related?

@claude

claude Bot commented May 29, 2026

Copy link
Copy Markdown

Claude finished @jansel's task in 2m 27s —— View job


CI Failure Analysis

  • Investigate the 6 new aarch64 py3.15 test failures
  • Investigate the unclassified libtorch build failures
  • Determine if any failures are related to this PR's change

None of the CI failures are related to this PR. All are pre-existing infrastructure issues.

6 New Failures (aarch64 py3.15 test jobs)

Root cause from the job log:

ERROR: Package 'triton' requires a different Python: 3.15.0 not in '<3.15,>=3.10'

The triton==3.7.0+git88b227e2 wheel declares python_requires='>=3.10,<3.15', which rejects Python 3.15. This pip install failure happens during the test environment setup, before any PyTorch code (including this PR's change) executes. This would fail on any PR that runs the py3.15 binary test jobs -- the triton package needs to update its version bounds.

9 Unclassified Failures (libtorch build/extract jobs)

The manywheel-cuda-cu1{26,30,32}-build jobs successfully built the wheel (confirmed: Successfully built torch-2.13.0.dev20260528+cu126-cp315-cp315-linux_x86_64.whl) but were canceled (##[error]The operation was canceled.) partway through the py3.15t build step -- likely a workflow timeout during the multi-Python wheel repacking phase. The downstream libtorch-*-extract jobs then failed because the upload artifact wasn't available (No files were found with the provided path). This is infrastructure flakiness unrelated to this one-line rename.

1 Unstable (ROCm mi355)

RuntimeError: inductor/test_max_autotune 1/4 failed!

Marked as unstable/flaky by DrCI. Unrelated to this PR.


@zklaus zklaus requested review from guilhermeleobas and rtimpe May 29, 2026 11:11
@zklaus zklaus mentioned this pull request May 29, 2026
16 tasks
@guilhermeleobas

Copy link
Copy Markdown
Collaborator

@pytorchbot merge -i

@pytorchmergebot

Copy link
Copy Markdown
Collaborator

pytorchmergebot pushed a commit that referenced this pull request Jun 1, 2026
….15 (#185682)

## Summary

Python 3.15 will expose an introspectable signature for `functools.reduce` using a PEP 661 sentinel default: `(function, iterable, /, initial=functools._initial_missing)`. The CPython change is [#149591](python/cpython#149591), which merged into CPython main on 2026-05-10, after 3.15.0b1 was tagged. So on 3.15.0b1 the default is still `<unrepresentable>` and `inspect.signature()` raises `ValueError`, which `substitute_in_graph` swallows and skips the check. From 3.15.0b2 the check will run.

When it runs, it compares positional parameter names, keyword-only names, and default values. The polyfill declares its own local sentinel:

```python
_initial_missing = object()

@substitute_in_graph(functools.reduce)
def reduce(function, iterable, initial=_initial_missing, /):
    ...
```

That local `object()` is a different instance from `functools._initial_missing`, so the defaults dict compares unequal and `substitute_in_graph` raises:

```
File ".../torch/_dynamo/polyfills/functools.py", line N, in <module>
  @substitute_in_graph(functools.reduce)
TypeError: Signature mismatch between <built-in function reduce> and <function reduce at 0x...>:
  (function, iterable, /, initial=_initial_missing)
  != (function, iterable, initial=<object object at 0x...>, /)
```

The polyfills loader imports every polyfill module, so this will break `import torch._dynamo` entirely on Python 3.15.0b2+. Same failure mode as #185403's `struct.pack` fix, just one step ahead of the next beta.

The fix imports `functools._initial_missing` instead of declaring a local one. The polyfill's internal `if initial is _initial_missing` identity check still works because both sides now reference the same object. No behavior change on any Python version. `functools._initial_missing` has existed in `functools` since well before 3.10 (it's used by the pure-Python `functools.reduce` fallback), so the import is safe on every supported version.

Part of #184352 (Python 3.15 support). Similar to #185403.

## Test plan

Verified on a locally-built Python 3.15-dev (CPython main at `heads/3.15:863c7e0`, which includes [#149591](python/cpython#149591)). The upstream signature is exposed natively:

```
$ python3.15 -c "import functools, inspect; print(inspect.signature(functools.reduce))"
(function, iterable, /, initial=_initial_missing)
$ python3.15 -c "import functools, inspect; sig = inspect.signature(functools.reduce); print(sig.parameters['initial'].default is functools._initial_missing)"
True
```

Extracted the actual `substitute_in_graph` from `torch/_dynamo/decorators.py` and ran it against both polyfill versions with no monkey-patching.

Unmodified polyfill (`_initial_missing = object()` declared locally):

```
TypeError: Signature mismatch between <built-in function reduce> and <function unmodified_reduce at 0x...>:
  (function, iterable, /, initial=_initial_missing)
  != (function, iterable, initial=<object object at 0x...>, /)
```

Fixed polyfill (`from functools import _initial_missing`): no `TypeError` at the signature check.

Regression on Python 3.12.13 (oldest currently supported):

```
$ python3 -c "import inspect, functools; inspect.signature(functools.reduce)"
ValueError: no signature found for builtin <built-in function reduce>
```

The signature check is silently skipped on Python <= 3.14, so the fix is a no-op there. `functools._initial_missing` is confirmed present on 3.12.13. Eager-vs-polyfill parity verified on every input shape:

```
reduce(lambda a, b: a+b, [1,2,3,4])       == 10     (matches functools.reduce)
reduce(lambda a, b: a+b, [1,2,3], 100)    == 106    (matches)
reduce(lambda a, b: a+b, [])              -> TypeError("reduce() of empty iterable with no initial value")  (matches)
reduce(lambda a, b: a+b, [], 99)          == 99     (matches)
```

Pre-commit checks: `python3 -m py_compile torch/_dynamo/polyfills/functools.py` and `git diff --staged --check` both clean.

Pull Request resolved: #185682
Approved by: https://github.com/ezyang

Co-authored-by: Edward Z. Yang via mergedog <ezyang@meta.com>
daisyden pushed a commit to daisyden/pytorch that referenced this pull request Jun 1, 2026
…ytorch#185403)

## Summary

Python 3.15 gives the builtin `struct.pack` an introspectable signature, `(format, /, *values)`. `substitute_in_graph` validates that a polyfill's signature matches the original builtin's, comparing positional-only parameter names. The `struct.pack` polyfill declared its positional-only parameter as `fmt`, mismatching `format`, so `substitute_in_graph` raised `TypeError` at import time:

```
File ".../torch/_dynamo/polyfills/struct.py", line 20, in <module>
  @substitute_in_graph(struct.pack, can_constant_fold_through=True)
TypeError: Signature mismatch between <built-in function pack> and <function pack ...>:
  (format, /, *values) != (fmt: 'bytes | str', /, *v: 'Any') -> 'bytes'
```

The polyfills loader eagerly imports every polyfill module, so this broke `import torch._dynamo` entirely on 3.15, failing every py3.15/3.15t binary test job (the smoke test imports `torch._dynamo`). On Python <= 3.14 the builtin has no introspectable signature, so the check is skipped and the mismatch went unnoticed.

The fix renames the parameter to `format` to match the builtin. The sibling `unpack` polyfill already uses `format`. The var-positional name (`*v`) is irrelevant here because `substitute_in_graph` ignores `VAR_POSITIONAL` parameter names.

Part of pytorch#184352 (Python 3.15 support).

## Test plan

Local introspection confirms there is no regression on older Pythons and that `format` is the canonical name where a signature is exposed:

```
$ python --version
Python 3.10.19
$ python -c "import inspect, struct; print(inspect.signature(struct.pack))"
ValueError: no signature found for builtin <built-in function pack>
$ python -c "import inspect, struct; print(inspect.signature(struct.unpack))"
(format, buffer, /)
```

On <= 3.14 `struct.pack` has no signature, so the `substitute_in_graph` check is skipped and the rename is a no-op there. The 3.15 failure reproduces only in the py3.15 binary smoke test (`manywheel-py3_15-*-test`); needs `ciflow/binaries` to validate.

Authored by Claude.

Pull Request resolved: pytorch#185403
Approved by: https://github.com/guilhermeleobas, https://github.com/rtimpe
khushi-411 pushed a commit to khushi-411/pytorch that referenced this pull request Jun 1, 2026
…ytorch#185403)

## Summary

Python 3.15 gives the builtin `struct.pack` an introspectable signature, `(format, /, *values)`. `substitute_in_graph` validates that a polyfill's signature matches the original builtin's, comparing positional-only parameter names. The `struct.pack` polyfill declared its positional-only parameter as `fmt`, mismatching `format`, so `substitute_in_graph` raised `TypeError` at import time:

```
File ".../torch/_dynamo/polyfills/struct.py", line 20, in <module>
  @substitute_in_graph(struct.pack, can_constant_fold_through=True)
TypeError: Signature mismatch between <built-in function pack> and <function pack ...>:
  (format, /, *values) != (fmt: 'bytes | str', /, *v: 'Any') -> 'bytes'
```

The polyfills loader eagerly imports every polyfill module, so this broke `import torch._dynamo` entirely on 3.15, failing every py3.15/3.15t binary test job (the smoke test imports `torch._dynamo`). On Python <= 3.14 the builtin has no introspectable signature, so the check is skipped and the mismatch went unnoticed.

The fix renames the parameter to `format` to match the builtin. The sibling `unpack` polyfill already uses `format`. The var-positional name (`*v`) is irrelevant here because `substitute_in_graph` ignores `VAR_POSITIONAL` parameter names.

Part of pytorch#184352 (Python 3.15 support).

## Test plan

Local introspection confirms there is no regression on older Pythons and that `format` is the canonical name where a signature is exposed:

```
$ python --version
Python 3.10.19
$ python -c "import inspect, struct; print(inspect.signature(struct.pack))"
ValueError: no signature found for builtin <built-in function pack>
$ python -c "import inspect, struct; print(inspect.signature(struct.unpack))"
(format, buffer, /)
```

On <= 3.14 `struct.pack` has no signature, so the `substitute_in_graph` check is skipped and the rename is a no-op there. The 3.15 failure reproduces only in the py3.15 binary smoke test (`manywheel-py3_15-*-test`); needs `ciflow/binaries` to validate.

Authored by Claude.

Pull Request resolved: pytorch#185403
Approved by: https://github.com/guilhermeleobas, https://github.com/rtimpe
khushi-411 pushed a commit to khushi-411/pytorch that referenced this pull request Jun 1, 2026
….15 (pytorch#185682)

## Summary

Python 3.15 will expose an introspectable signature for `functools.reduce` using a PEP 661 sentinel default: `(function, iterable, /, initial=functools._initial_missing)`. The CPython change is [pytorch#149591](python/cpython#149591), which merged into CPython main on 2026-05-10, after 3.15.0b1 was tagged. So on 3.15.0b1 the default is still `<unrepresentable>` and `inspect.signature()` raises `ValueError`, which `substitute_in_graph` swallows and skips the check. From 3.15.0b2 the check will run.

When it runs, it compares positional parameter names, keyword-only names, and default values. The polyfill declares its own local sentinel:

```python
_initial_missing = object()

@substitute_in_graph(functools.reduce)
def reduce(function, iterable, initial=_initial_missing, /):
    ...
```

That local `object()` is a different instance from `functools._initial_missing`, so the defaults dict compares unequal and `substitute_in_graph` raises:

```
File ".../torch/_dynamo/polyfills/functools.py", line N, in <module>
  @substitute_in_graph(functools.reduce)
TypeError: Signature mismatch between <built-in function reduce> and <function reduce at 0x...>:
  (function, iterable, /, initial=_initial_missing)
  != (function, iterable, initial=<object object at 0x...>, /)
```

The polyfills loader imports every polyfill module, so this will break `import torch._dynamo` entirely on Python 3.15.0b2+. Same failure mode as pytorch#185403's `struct.pack` fix, just one step ahead of the next beta.

The fix imports `functools._initial_missing` instead of declaring a local one. The polyfill's internal `if initial is _initial_missing` identity check still works because both sides now reference the same object. No behavior change on any Python version. `functools._initial_missing` has existed in `functools` since well before 3.10 (it's used by the pure-Python `functools.reduce` fallback), so the import is safe on every supported version.

Part of pytorch#184352 (Python 3.15 support). Similar to pytorch#185403.

## Test plan

Verified on a locally-built Python 3.15-dev (CPython main at `heads/3.15:863c7e0`, which includes [pytorch#149591](python/cpython#149591)). The upstream signature is exposed natively:

```
$ python3.15 -c "import functools, inspect; print(inspect.signature(functools.reduce))"
(function, iterable, /, initial=_initial_missing)
$ python3.15 -c "import functools, inspect; sig = inspect.signature(functools.reduce); print(sig.parameters['initial'].default is functools._initial_missing)"
True
```

Extracted the actual `substitute_in_graph` from `torch/_dynamo/decorators.py` and ran it against both polyfill versions with no monkey-patching.

Unmodified polyfill (`_initial_missing = object()` declared locally):

```
TypeError: Signature mismatch between <built-in function reduce> and <function unmodified_reduce at 0x...>:
  (function, iterable, /, initial=_initial_missing)
  != (function, iterable, initial=<object object at 0x...>, /)
```

Fixed polyfill (`from functools import _initial_missing`): no `TypeError` at the signature check.

Regression on Python 3.12.13 (oldest currently supported):

```
$ python3 -c "import inspect, functools; inspect.signature(functools.reduce)"
ValueError: no signature found for builtin <built-in function reduce>
```

The signature check is silently skipped on Python <= 3.14, so the fix is a no-op there. `functools._initial_missing` is confirmed present on 3.12.13. Eager-vs-polyfill parity verified on every input shape:

```
reduce(lambda a, b: a+b, [1,2,3,4])       == 10     (matches functools.reduce)
reduce(lambda a, b: a+b, [1,2,3], 100)    == 106    (matches)
reduce(lambda a, b: a+b, [])              -> TypeError("reduce() of empty iterable with no initial value")  (matches)
reduce(lambda a, b: a+b, [], 99)          == 99     (matches)
```

Pre-commit checks: `python3 -m py_compile torch/_dynamo/polyfills/functools.py` and `git diff --staged --check` both clean.

Pull Request resolved: pytorch#185682
Approved by: https://github.com/ezyang

Co-authored-by: Edward Z. Yang via mergedog <ezyang@meta.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ciflow/binaries Trigger all binary build and upload jobs on the PR ciflow/inductor ciflow/torchtitan Run TorchTitan integration tests ciflow/trunk Trigger trunk jobs on your pull request Merged module: dynamo open source topic: not user facing topic category

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants