Support Float8_e8m0fnu and Float4_e2m1fn_x2 in stable ABI by janeyx99 · Pull Request #173669 · pytorch/pytorch

janeyx99 · 2026-01-28T17:09:42Z

Missing tests! Will add after landing Nicolas' PR.

Ahh I still need to clean some stuff up: gotta remove my clang-tidy removal oop. And I'm now reconsidering whether I want float4 in the supported dtypes lol. cc @pearu if you have thoughts on this second point.

Stack from ghstack (oldest at bottom):

[ghstack-poisoned]

pytorch-bot · 2026-01-28T17:09:47Z

This PR needs a `release notes:` label

If your changes are user facing and intended to be a part of release notes, please use a label starting with release notes:.

If not, please add the topic: not user facing label.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "topic: not user facing"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

ghstack-source-id: 87c28c7 Pull Request resolved: #173669

pytorch-bot · 2026-01-28T17:09:50Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/173669

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (2 Unrelated Failures)

As of commit c12c194 with merge base 89f3102 ():

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

pull / linux-jammy-py3.10-gcc11 / test (distributed, 1, 2, linux.2xlarge) (gh) (similar failure)
test/distributed/tensor/test_dtensor_ops.py::TestUnbackedDTensorOpsCPU::test_unbacked_dtensor_op_db_scatter_add_cpu_float32
pull / linux-jammy-py3.14t-clang15 / test (dynamo_wrapped, 3, 3, linux.2xlarge) (gh) (matched linux rule in flaky-rules.json)
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions · 2026-01-28T17:13:29Z

Attention! PyTorch one of the C-stable API file was changed

You MUST NOT change existing function declarations in this, as this header defines a stable C ABI. If you need to change the signature for a function, introduce a new v2 version of the function and modify code generation to target the new version of the function.

Caused by:

torch/csrc/stable/c/shim.h

Missing tests! Will add after landing Nicolas' PR. [ghstack-poisoned]

ghstack-source-id: 0d77fed Pull Request resolved: #173669 mend

[ghstack-poisoned]

ghstack-source-id: e30d650 Pull Request resolved: #173669 mend

[ghstack-poisoned]

ghstack-source-id: 3fed2c0 Pull Request resolved: #173669 mend

mikaylagawarecki

Looks good other than .clang-tidy change

Could you list the dtypes that are unsupported by the stable ABI as of this PR in the PR description

mikaylagawarecki · 2026-02-06T17:48:02Z

 '
 HeaderFilterRegex: '^(aten/|c10/|torch/).*$'
 WarningsAsErrors: '*'
-LineFilter:


was this intentional

thanks for the catch, this was unintentional, but this LineFilter thing keeps tripping up my local clang extension. If this happens for u too i'll make a sep PR to remove this

andrewor14

Thanks, looks good to me. Tested this through pytorch/ao#3610 and it works well. I'll let @mikaylagawarecki do the final stamp

mikaylagawarecki

Stamp as long as clang-tidy is in a separate PR (I don't run into issues/am not sure what the change does)

[ghstack-poisoned]

pytorchmergebot · 2026-02-11T21:56:27Z

Starting merge as part of PR stack under #174809

pytorchmergebot · 2026-02-12T04:37:42Z

Starting merge as part of PR stack under #174809

pytorchmergebot · 2026-02-12T05:33:35Z

Starting merge as part of PR stack under #174809

pytorchmergebot · 2026-02-12T05:59:36Z

Starting merge as part of PR stack under #174809

pytorchmergebot · 2026-02-12T06:30:41Z

Starting merge as part of PR stack under #174735

Pull Request resolved: #174733 Approved by: https://github.com/zou3519 ghstack dependencies: #173669

This fixes the bug where SymInts, MemoryFormat, ScalarType, and Layout were all entering the IntType path instead of going on their respective branches. We do explicitly point SymInt to Int for now, because we do not support having a StableIValue from SymInt yet, but nothing has decreased in support due to this change. If you don't believe me, check out the test case I added in the stack just one above that tests that with or without this change, registering certain ops with SymInt schemas work and don't cause recompiles. Pull Request resolved: #174734 Approved by: https://github.com/albanD ghstack dependencies: #173669, #174733

We already had a shim for it, so we're merely adding a C++ convenience wrapper and tests. This is why we can stick it in libtorch_agn_2_9. I had wanted to just write a test that would take in a Tensor and return a Layout and then check that the layout matches in python, but I learned that our python binding code (pytorch/torch/csrc/jit/python/pybind_utils.cpp) doesn't properly convert Layout IValues to torch.layout (because layout, dtype and memory format are all treated like ints). So the test case I ended up writing takes in a Layout and runs the check from C++. Further testing I've done: I wanted to confirm that my_layout would still run with no problems on 2.9 even if it needs to be built with 2.11 for the layout API. So I: - built a wheel for libtorch_agn_2_9 with my PR's torch - went to a different conda env - installed 2.9.1 torch - installed the wheel i just built - ran the test `python test/cpp_extensions/test_libtorch_agnostic.py -v -k my_layout` and it passed! Pull Request resolved: #174735 Approved by: https://github.com/andrewor14 ghstack dependencies: #173669, #174733, #174734

ghstack-source-id: e5fafc4 Pull Request resolved: pytorch/pytorch#173669 mend

* fix: gate torch stable ABI on >=2.11 and allow env override torch 2.10's stable ABI is missing Float8_e8m0fnu (ScalarType 44), so passing a UE8M0 weight scale through torch.ops.humming.launch_kernel fails with "RuntimeError: Not yet supported ScalarType 44". Support for Float8_e8m0fnu and Float4_e2m1fn_x2 in the stable ABI was added in pytorch/pytorch#173669, gated by TORCH_FEATURE_VERSION >= TORCH_VERSION_2_11_0 (first available in torch 2.11.0, released 2026-03-23). Bump the version threshold to >= 2.11 so the stable ABI path is only taken when its dtype coverage matches humming's needs, and add the HUMMING_USE_TORCH_STABLE_API env var so users can force the choice. The non-stable ABI fallback already exists in csrc/launcher/torch_api.h and supports the full ScalarType enum via native at::Tensor. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * address review: include stable-ABI flag in cache key, drop redundant import - get_humming_launcher_build_dir now takes use_torch_stable_api and appends "stable" / "nostable" to the build dir, so toggling HUMMING_USE_TORCH_STABLE_API no longer risks loading a stale launcher binary compiled with the opposite flag value. - Drop the now-unused packaging.version.Version import in init_humming_launcher; the version comparison lives in _resolve_use_torch_stable_api. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Support Float8_e8m0fnu and Float4_e2m1fn_x2 in stable ABI

7f36bc7

[ghstack-poisoned]

janeyx99 added a commit that referenced this pull request Jan 28, 2026

Support Float8_e8m0fnu and Float4_e2m1fn_x2 in stable ABI

87a335d

ghstack-source-id: 87c28c7 Pull Request resolved: #173669

Update on "Support Float8_e8m0fnu and Float4_e2m1fn_x2 in stable ABI"

c4e8f78

Missing tests! Will add after landing Nicolas' PR. [ghstack-poisoned]

janeyx99 added a commit that referenced this pull request Feb 3, 2026

Support Float8_e8m0fnu and Float4_e2m1fn_x2 in stable ABI

890a445

ghstack-source-id: 0d77fed Pull Request resolved: #173669 mend

Update

b144a20

[ghstack-poisoned]

janeyx99 added a commit that referenced this pull request Feb 3, 2026

Support Float8_e8m0fnu and Float4_e2m1fn_x2 in stable ABI

74adb64

ghstack-source-id: e30d650 Pull Request resolved: #173669 mend

Update

d761ca6

[ghstack-poisoned]

janeyx99 added a commit that referenced this pull request Feb 6, 2026

Support Float8_e8m0fnu and Float4_e2m1fn_x2 in stable ABI

b64755d

ghstack-source-id: 3fed2c0 Pull Request resolved: #173669 mend

This was referenced Feb 5, 2026

Enable passing in compiler variables for TORCH_LIBRARY namespace #174424

Closed

Rescaffold libtorch agnostic tests: have new versions build older kernels #174425

Closed

janeyx99 requested a review from andrewor14 February 6, 2026 16:58

janeyx99 marked this pull request as ready for review February 6, 2026 16:58

janeyx99 requested a review from mikaylagawarecki as a code owner February 6, 2026 16:58

mikaylagawarecki reviewed Feb 6, 2026

View reviewed changes

andrewor14 mentioned this pull request Feb 6, 2026

[claude] Make MXFP8 cuda kernels ABI stable pytorch/ao#3610

Merged

andrewor14 reviewed Feb 6, 2026

View reviewed changes

mikaylagawarecki approved these changes Feb 6, 2026

View reviewed changes

Update

4872e95

[ghstack-poisoned]

This was referenced Feb 10, 2026

Add a test for SymInt handling in stable ABI #174733

Closed

Use real_type over type so that IValues go to intended path #174734

Closed

Add torch::stable::Tensor.layout() #174735

Closed

janeyx99 added 2 commits February 11, 2026 20:27

Update

af1e810

[ghstack-poisoned]

Update

c12c194

[ghstack-poisoned]

This was referenced Feb 11, 2026

[BE] Use newer utils to avoid deprecation error during build for libtorch agno tests #174809

Closed

Support qint dtypes in stable ABI #174812

Closed

janeyx99 added the release notes: cpp release notes category label Feb 11, 2026

janeyx99 added topic: new features topic category topic: improvements topic category and removed topic: new features topic category labels Feb 11, 2026

pytorchmergebot closed this in 0b6a87e Feb 12, 2026

pytorchmergebot pushed a commit that referenced this pull request Feb 12, 2026

Add a test for SymInt handling in stable ABI (#174733)

94e7085

Pull Request resolved: #174733 Approved by: https://github.com/zou3519 ghstack dependencies: #173669

pytorchmergebot added the Merged label Feb 12, 2026

sandy-gags pushed a commit to sandy-gags/pytorch that referenced this pull request Mar 12, 2026

Support Float8_e8m0fnu and Float4_e2m1fn_x2 in stable ABI

613166c

ghstack-source-id: e5fafc4 Pull Request resolved: pytorch/pytorch#173669 mend

github-actions Bot deleted the gh/janeyx99/339/head branch March 15, 2026 02:24

coderabbitai Bot mentioned this pull request May 4, 2026

[Examples] Add examples for operators in DeepSeek-V4 tile-ai/tilelang#2148

Merged

huangzhilin-hzl mentioned this pull request May 11, 2026

fix: gate torch stable ABI on >=2.11 and allow env override inclusionAI/humming#18

Merged

Conversation

janeyx99 commented Jan 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot Bot commented Jan 28, 2026

This PR needs a release notes: label

Uh oh!

pytorch-bot Bot commented Jan 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/173669

✅ You can merge normally! (2 Unrelated Failures)

Uh oh!

github-actions Bot commented Jan 28, 2026

Attention! PyTorch one of the C-stable API file was changed

Uh oh!

mikaylagawarecki left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mikaylagawarecki Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

janeyx99 Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

andrewor14 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mikaylagawarecki left a comment

Choose a reason for hiding this comment

Uh oh!

pytorchmergebot commented Feb 11, 2026

Uh oh!

pytorchmergebot commented Feb 12, 2026

Uh oh!

pytorchmergebot commented Feb 12, 2026

Uh oh!

pytorchmergebot commented Feb 12, 2026

Uh oh!

pytorchmergebot commented Feb 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

janeyx99 commented Jan 28, 2026 •

edited

Loading

This PR needs a `release notes:` label

pytorch-bot Bot commented Jan 28, 2026 •

edited

Loading

mikaylagawarecki left a comment •

edited

Loading

andrewor14 left a comment •

edited

Loading