[Reland][CPU] Add ops for float8 linear by Xia-Weiwen · Pull Request #3100 · pytorch/ao

Xia-Weiwen · 2025-09-30T01:33:27Z

Reland #3052 which is reverted by 5cbbd73

Revert reason: compiler is complaining about switch...case of bool values.

This PR fixes the issue by converting switch...case to if...else

pytorch-bot · 2025-09-30T01:33:31Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/3100

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit e1cfca5 with merge base c7b8e13 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

Xia-Weiwen · 2025-09-30T01:35:30Z

Hi @jcaip @jerryzh168 Could you please import this patch to check if the error is gone in your internal checks? Thanks.

Xia-Weiwen · 2025-09-30T02:25:59Z

Hi @jerryzh168 Thanks for approving. Do you still need to import it?

jcaip · 2025-09-30T13:04:52Z

Thanks for fixing this @Xia-Weiwen! This is still failing some tests, I'm seeing:

fbcode/pytorch/ao/torchao/csrc/cpu/aten_kernels/float8_linear.cpp:509:44: error: capturing a structured binding is not yet supported in OpenMP
  509 |     at::native::data_index_init(begin, mc, Mc_parallel, nc, Nc);
      |                                            ^
fbcode/pytorch/ao/torchao/csrc/cpu/aten_kernels/float8_linear.cpp:456:37: note: 'Mc_parallel' declared here
  456 |   auto [parallel_on_M, block_m, Mc, Mc_parallel] = get_m_blocking(M);
      |                                     ^
fbcode/pytorch/ao/torchao/csrc/cpu/aten_kernels/float8_linear.cpp:512:24: error: capturing a structured binding is not yet supported in OpenMP
  512 |       int64_t mc_end = parallel_on_M ? mc + 1 : Mc;
      |                        ^
fbcode/pytorch/ao/torchao/csrc/cpu/aten_kernels/float8_linear.cpp:456:9: note: 'parallel_on_M' declared here
  456 |   auto [parallel_on_M, block_m, Mc, Mc_parallel] = get_m_blocking(M);
      |         ^
fbcode/pytorch/ao/torchao/csrc/cpu/aten_kernels/float8_linear.cpp:512:49: error: capturing a structured binding is not yet supported in OpenMP
  512 |       int64_t mc_end = parallel_on_M ? mc + 1 : Mc;
      |                                                 ^
fbcode/pytorch/ao/torchao/csrc/cpu/aten_kernels/float8_linear.cpp:456:33: note: 'Mc' declared here
  456 |   auto [parallel_on_M, block_m, Mc, Mc_parallel] = get_m_blocking(M);
      |                                 ^
fbcode/pytorch/ao/torchao/csrc/cpu/aten_kernels/float8_linear.cpp:515:32: error: capturing a structured binding is not yet supported in OpenMP
  515 |         int64_t m_size = mci * block_m + block_m > M ? M - mci * block_m : block_m;
      |                                ^
fbcode/pytorch/ao/torchao/csrc/cpu/aten_kernels/float8_linear.cpp:456:24: note: 'block_m' declared here
  456 |   auto [parallel_on_M, block_m, Mc, Mc_parallel] = get_m_blocking(M);
      |                        ^
4 errors generated.

@jerryzh168 Long term I think we need to update our C++ linting to be as strict as whatever upstream dependencies we have, so we can find these issues earlier.

facebook-github-bot · 2025-09-30T13:07:20Z

@jcaip has imported this pull request. If you are a Meta employee, you can view this in D83564232.

Xia-Weiwen · 2025-10-02T08:49:10Z

Hi @jcaip Thanks for the feedback. I could not see the errors you saw, even not as warnings. I use the following environment to build Torchao:

Ubuntu 22.04.3 LTS
GCC 12.3.0
Python 3.10
CMD: USE_CPU_KERNELS=1 python setup.py develop

I can only see the following warnings:

warning: ignoring attributes on template argument '__m512' [-Wignored-attributes]
warning: ignoring attributes on template argument '__m256i' [-Wignored-attributes]
warning: "Py_LIMITED_API" redefined

And these warnings are not from the files this PR adds.
Can you please share the environment, configuration and command to build Torchao so that I can reproduce these errors and fix them in advance? Thanks.

jcaip · 2025-10-03T16:44:49Z

@Xia-Weiwen do you mind if i just push to your branch / give you a patch? I'll see if I can find out exactly why this is failing but in the meantime its prolly faster for me to just fix this myself.

jcaip · 2025-10-03T17:00:25Z

+  TORCH_CHECK(weight.size(3) == block_n, "Float8 linear: unexpected weight shape");
+  int64_t N = Nc * block_n;
+  TORCH_CHECK(K == Kc * block_k, "Float8 linear: weight and input shapes mismatch");
+  auto [parallel_on_M, block_m, Mc, Mc_parallel] = get_m_blocking(M);


FYI I think:

bool parallel_on_M; int64_t block_m, Mc, Mc_parallel; std::tie(parallel_on_M, block_m, Mc, Mc_parallel) = get_m_blocking(M);

instead of this line fixes the error -testing internally now @Xia-Weiwen

@jcaip Thanks. I have updated this part. If you find more issues, please feel free to update my branch directly. Thanks.

Xia-Weiwen · 2025-10-10T14:27:03Z

Hi @jcaip Could you please re-review this PR? Thanks.

jcaip · 2025-10-10T14:58:47Z

Hi @Xia-Weiwen Sorry, missed this earlier this week - will take a look today!

meta-codesync · 2025-10-10T15:08:16Z

@jcaip has imported this pull request. If you are a Meta employee, you can view this in D83564232.

jcaip

this passes tests, thanks for the fix @Xia-Weiwen!

Xia-Weiwen · 2025-10-11T01:13:32Z

@jcaip Thanks a lot

vkuzo · 2025-10-15T17:53:07Z

#3175, let's revert this

This reverts commit 29f6758.

Revert "[Reland][CPU] Add ops for float8 linear (#3100)" This reverts commit 29f6758.

* [Reland][CPU] Add ops for float8 linear * Update torchao/csrc/cpu/aten_kernels/float8_linear.cpp

Revert "[Reland][CPU] Add ops for float8 linear (pytorch#3100)" This reverts commit 29f6758.

pytorch-bot Bot added the ci-no-td label Sep 30, 2025

meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 30, 2025

Xia-Weiwen requested review from jcaip and jerryzh168 September 30, 2025 01:33

Xia-Weiwen added the module: not user facing Use this tag if you don't want this PR to show up in release notes label Sep 30, 2025

Xia-Weiwen mentioned this pull request Sep 30, 2025

[CPU] add Float8OpaqueTensor for dynamic float8 act float8 weight #3075

Merged

Xia-Weiwen marked this pull request as ready for review September 30, 2025 02:08

mingfeima approved these changes Sep 30, 2025

View reviewed changes

jerryzh168 approved these changes Sep 30, 2025

View reviewed changes

[Reland][CPU] Add ops for float8 linear

479111c

jcaip reviewed Oct 3, 2025

View reviewed changes

Xia-Weiwen requested a review from jcaip October 9, 2025 03:32

Xia-Weiwen added 2 commits October 9, 2025 09:12

Merge branch 'main' into reland_fp8_linear

3bacf14

Update torchao/csrc/cpu/aten_kernels/float8_linear.cpp

e1cfca5

jcaip approved these changes Oct 10, 2025

View reviewed changes

jcaip merged commit 29f6758 into pytorch:main Oct 10, 2025
20 of 21 checks passed

vkuzo added a commit that referenced this pull request Oct 15, 2025

Revert "[Reland][CPU] Add ops for float8 linear (#3100)"

3a42894

This reverts commit 29f6758.

vkuzo mentioned this pull request Oct 15, 2025

Revert "[Reland][CPU] Add ops for float8 linear" #3179

Merged

vkuzo added a commit that referenced this pull request Oct 15, 2025

Revert "[Reland][CPU] Add ops for float8 linear" (#3179)

30082cb

Revert "[Reland][CPU] Add ops for float8 linear (#3100)" This reverts commit 29f6758.

Xia-Weiwen mentioned this pull request Oct 29, 2025

Revert "Revert "[Reland][CPU] Add ops for float8 linear" (#3179)" #3254

Merged

namgyu-youn pushed a commit to namgyu-youn/ao that referenced this pull request Nov 21, 2025

[Reland][CPU] Add ops for float8 linear (pytorch#3100)

9b707eb

* [Reland][CPU] Add ops for float8 linear * Update torchao/csrc/cpu/aten_kernels/float8_linear.cpp

namgyu-youn pushed a commit to namgyu-youn/ao that referenced this pull request Nov 21, 2025

Revert "[Reland][CPU] Add ops for float8 linear" (pytorch#3179)

c7674d7

Revert "[Reland][CPU] Add ops for float8 linear (pytorch#3100)" This reverts commit 29f6758.

Conversation

Xia-Weiwen commented Sep 30, 2025

Uh oh!

pytorch-bot Bot commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/3100

✅ No Failures

Uh oh!

Xia-Weiwen commented Sep 30, 2025

Uh oh!

Xia-Weiwen commented Sep 30, 2025

Uh oh!

jcaip commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facebook-github-bot commented Sep 30, 2025

Uh oh!

Xia-Weiwen commented Oct 2, 2025

Uh oh!

jcaip commented Oct 3, 2025

Uh oh!

jcaip Oct 3, 2025

Choose a reason for hiding this comment

Uh oh!

Xia-Weiwen Oct 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Xia-Weiwen commented Oct 10, 2025

Uh oh!

jcaip commented Oct 10, 2025

Uh oh!

meta-codesync Bot commented Oct 10, 2025

Uh oh!

jcaip left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Xia-Weiwen commented Oct 11, 2025

Uh oh!

vkuzo commented Oct 15, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

pytorch-bot Bot commented Sep 30, 2025 •

edited

Loading

jcaip commented Sep 30, 2025 •

edited

Loading

Xia-Weiwen Oct 9, 2025 •

edited

Loading