Update warning of TF32 by yanbing-j · Pull Request #158209 · pytorch/pytorch

yanbing-j · 2025-07-14T02:42:06Z

Fixes #ISSUE_NUMBER

pytorch-bot · 2025-07-14T02:42:10Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/158209

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (2 Unrelated Failures)

As of commit 1291029 with merge base c8c221c ():

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

pull / linux-jammy-cuda12.8-py3.10-gcc11-sm89 / test (default, 1, 5, linux.g6.4xlarge.experimental.nvidia.gpu) (gh) (trunk failure)
inductor/test_inplace_padding.py::InplacePaddingTest::test_linear_and_cel

UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:

pull / cuda12.8-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu, unstable) (gh) (#153987)
MISSING REGRESSION TEST

This comment was automatically generated by Dr. CI and updates every 15 minutes.

jansel

If we are going to deprecate the old stuff, we should update the docs to not suggest using the old stuff. Things the docs tell you to do should not spam warnings at you.

A simpler fix would be to remove the warning.

jansel · 2025-07-14T03:06:48Z

aten/src/ATen/Context.cpp

+      "Suggest to use a new setting of API control of a more fine-grained TF32 behavior, e.g, "
+      "torch.backends.cudnn.conv.fp32_precision = 'tf32' or torch.backends.cuda.matmul.fp32_precision = 'ieee'. "
+      "Old setting, e.g, torch.backends.cuda.matmul.allow_tf32 = True, torch.backends.cudnn.allow_tf32 = True, "
+      "allowTF32CuDNN() and allowTF32CuBLAS() are still supported, and is going to be deprecated. Please see "


Perhaps:

Please use the new API settings to control TF32 behavior, such as ...

jansel · 2025-07-14T03:07:14Z

docs/source/notes/mkldnn.rst

 ---------------------------------------------------

-Starting in PyTorch 2.4, there is a set of APIs to control the internal computation precision
+Starting in PyTorch 2.9, there is a set of APIs to control the internal computation precision


Was 2.4 wrong?

No, #125888 was firsted drafted in PyTorch 2.4 window. Before that PR, we cannot control TF32 for mkldnn backend. I update this to 2.9, because PR is merged in PyTorch 2.9.

If the APIs existed in 2.4, this should say 2.4.

No, the APIs are finally upstreamed in 2.9, 2.4 didn't have that. #125888 was firsted drafted in PyTorch 2.4 window, but is upstreamed in 2.9. Mkldnn Docs needs to be updated to 2.9.

albanD · 2025-07-14T15:18:40Z

aten/src/ATen/Context.cpp

  C10_ALWAYS_INLINE void warn_deprecated_fp32_precision_api(){
    TORCH_WARN_ONCE(
-      "This API is going to be deprecated, please see "
+      "Please use the new API settings to control TF32 behavior, such as torch.backends.cudnn.conv.fp32_precision = tf32 "


I don't recall if we have a way, from here, to set the warning stacklevel argument.
Setting that value properly will make it point to the user code setting the value and will be more obvious on what they did wrong.

jansel · 2025-07-14T18:58:32Z

aten/src/ATen/Context.cpp

  C10_ALWAYS_INLINE void warn_deprecated_fp32_precision_api(){
    TORCH_WARN_ONCE(
-      "This API is going to be deprecated, please see "
+      "Please use the new API settings to control TF32 behavior, such as torch.backends.cudnn.conv.fp32_precision = tf32 "


Suggested change

"Please use the new API settings to control TF32 behavior, such as torch.backends.cudnn.conv.fp32_precision = tf32 "

"Please use the new API settings to control TF32 behavior, such as torch.backends.cudnn.conv.fp32_precision = 'tf32'"

jansel · 2025-07-14T18:58:45Z

aten/src/ATen/Context.cpp

    TORCH_WARN_ONCE(
-      "This API is going to be deprecated, please see "
+      "Please use the new API settings to control TF32 behavior, such as torch.backends.cudnn.conv.fp32_precision = tf32 "
+      "or torch.backends.cuda.matmul.fp32_precision = ieee. Old settings, e.g, torch.backends.cuda.matmul.allow_tf32 = True, "


Suggested change

"or torch.backends.cuda.matmul.fp32_precision = ieee. Old settings, e.g, torch.backends.cuda.matmul.allow_tf32 = True, "

"or torch.backends.cuda.matmul.fp32_precision = 'ieee'. Old settings, e.g, torch.backends.cuda.matmul.allow_tf32 = True, "

yanbing-j · 2025-07-16T01:21:22Z

@pytorchbot merge

pytorchmergebot · 2025-07-16T01:23:09Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

Summary: According to #158209, the API is deprecated and we should be using torch.backends.cuda.matmul.fp32_precision instead. Fixes #159440 Test Plan: CI [ghstack-poisoned]

Summary: According to #158209, the API is deprecated and we should be using torch.backends.cuda.matmul.fp32_precision instead. Fixes #159440 Test Plan: CI ghstack-source-id: a22b482 Pull Request resolved: #159480

…#159480) Summary: According to #158209, the API is deprecated and we should be using torch.backends.cuda.matmul.fp32_precision instead. Fixes #159440 Test Plan: CI Pull Request resolved: #159480 Approved by: https://github.com/xmfan, https://github.com/oulgen

pytorchbot added the open source label Jul 14, 2025

yanbing-j mentioned this pull request Jul 14, 2025

refine fp32 precision api #125888

Closed

jansel requested changes Jul 14, 2025

View reviewed changes

yanbing-j force-pushed the yanbing/update_tf32_warning branch from db58c80 to a2469b6 Compare July 14, 2025 06:48

yanbing-j added the ciflow/trunk Trigger trunk jobs on your pull request label Jul 14, 2025

albanD reviewed Jul 14, 2025

View reviewed changes

jansel reviewed Jul 14, 2025

View reviewed changes

yanbing-j mentioned this pull request Jul 15, 2025

Enable TF32 as fp32 internal precision for matmul/linear/conv #157520

Closed

yanbing-j force-pushed the yanbing/update_tf32_warning branch from 1480de7 to 1291029 Compare July 15, 2025 06:12

yanbing-j added 2 commits July 15, 2025 06:17

Update warning of TF32

5b0622d

Update based on comments

1291029

yanbing-j added the topic: not user facing topic category label Jul 15, 2025

jansel approved these changes Jul 15, 2025

View reviewed changes

yanbing-j marked this pull request as ready for review July 16, 2025 01:21

pytorchmergebot added the merging label Jul 16, 2025

pytorchmergebot added the Merged label Jul 16, 2025

pytorchmergebot closed this in 900fba4 Jul 16, 2025

pytorchmergebot removed the merging label Jul 16, 2025

xmfan mentioned this pull request Jul 30, 2025

torch.compile nightly: Warning spam from deprecated TF32 API #159440

Closed

masnesral mentioned this pull request Jul 30, 2025

Don't use torch.backends.cuda.matmul.allow_tf32 in inductor cache key #159480

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update warning of TF32#158209

Update warning of TF32#158209
yanbing-j wants to merge 2 commits intopytorch:mainfrom
yanbing-j:yanbing/update_tf32_warning

yanbing-j commented Jul 14, 2025

Uh oh!

pytorch-bot bot commented Jul 14, 2025 •

edited

Loading

Uh oh!

jansel left a comment •

edited

Loading

Uh oh!

jansel Jul 14, 2025

Uh oh!

jansel Jul 14, 2025

Uh oh!

yanbing-j Jul 14, 2025

Uh oh!

jansel Jul 14, 2025

Uh oh!

yanbing-j Jul 15, 2025

Uh oh!

albanD Jul 14, 2025

Uh oh!

jansel Jul 14, 2025

Uh oh!

jansel Jul 14, 2025

Uh oh!

yanbing-j commented Jul 16, 2025

Uh oh!

pytorchmergebot commented Jul 16, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

	"Please use the new API settings to control TF32 behavior, such as torch.backends.cudnn.conv.fp32_precision = tf32 "
	"Please use the new API settings to control TF32 behavior, such as torch.backends.cudnn.conv.fp32_precision = 'tf32'"

	"or torch.backends.cuda.matmul.fp32_precision = ieee. Old settings, e.g, torch.backends.cuda.matmul.allow_tf32 = True, "
	"or torch.backends.cuda.matmul.fp32_precision = 'ieee'. Old settings, e.g, torch.backends.cuda.matmul.allow_tf32 = True, "

Conversation

yanbing-j commented Jul 14, 2025

Uh oh!

pytorch-bot bot commented Jul 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/158209

✅ You can merge normally! (2 Unrelated Failures)

Uh oh!

jansel left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yanbing-j commented Jul 16, 2025

Uh oh!

pytorchmergebot commented Jul 16, 2025

Merge started

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

pytorch-bot bot commented Jul 14, 2025 •

edited

Loading

jansel left a comment •

edited

Loading