baby steps on patching inf/nan behavior & aten::amin support in nvfuser by jjsjann123 · Pull Request #75646 · pytorch/pytorch

jjsjann123 · 2022-04-12T02:56:45Z

Fixes #75622

Instead of getting max/min_value for reduction init value, we go with (-)infinity instead so we can properly preserve inf inputs;
Adding inf/(-)inf/nan for float value.
Adding aten::amin in nvfuser (@kevinstephano @rdspring1 for review)

facebook-github-bot · 2022-04-12T02:56:51Z

🔗 Helpful links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/75646
📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓Need help or want to give feedback on the CI? Visit our office hours

💊 CI failures summary and remediations

As of commit 80c232e (more details on the Dr. CI page):

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

jjsjann123 · 2022-04-12T02:57:17Z

cc'ing @kevinstephano @rdspring1

ngimel · 2022-04-12T16:10:48Z

test/test_jit_cuda_fuser.py

+                     "Requires fusion optimization pass to be effective")
+    @unittest.skipIf(is_pre_volta(), "reduction not supported in pre volta device")
+    def test_inf_quick_patch(self):
+        x = torch.tensor([-float('inf'), -float('inf'), 4.0], device="cuda")


This test is not enough to catch previously problematic values - you need input full of -inf (nothing else there) to catch that FLT_MIN is not the correct initializer, similarly for amin. You need yet another input for nan propagation (unless you have those tests in other places, in which case it's probably better to unify)

rdspring1

LGTM

kevinstephano

LGTM

jjsjann123 · 2022-04-13T15:49:54Z

@pytorchbot merge this

github-actions · 2022-04-13T15:51:58Z

Hey @jjsjann123.
You've committed this PR, but it does not have both a 'release notes: ...' and 'topics: ...' label. Please add one of each to the PR. The 'release notes: ...' label should represent the part of PyTorch that this PR changes (fx, autograd, distributed, etc) and the 'topics: ...' label should represent the kind of PR it is (not user facing, new feature, bug fix, perf improvement, etc). The list of valid labels can be found here for the 'release notes: ...' and here for the 'topics: ...'.
For changes that are 'topic: not user facing' there is no need for a release notes label.

…er (#75646) Summary: Fixes #75622 1. Instead of getting max/min_value for reduction init value, we go with (-)infinity instead so we can properly preserve inf inputs; 2. Adding inf/(-)inf/nan for float value. 3. Adding aten::amin in nvfuser (kevinstephano rdspring1 for review) Pull Request resolved: #75646 Approved by: https://github.com/rdspring1, https://github.com/kevinstephano, https://github.com/ngimel Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/692ebc8d8bbd10c21530254b29d458dc9b871386 Reviewed By: osalpekar Differential Revision: D35618790 Pulled By: mehtanirav fbshipit-source-id: 406965941919ad1777b74d36898709eb17580fa1

@kevinstephano

Fixes pytorch#75622 1. Instead of getting max/min_value for reduction init value, we go with (-)infinity instead so we can properly preserve inf inputs; 2. Adding inf/(-)inf/nan for float value. 3. Adding aten::amin in nvfuser (@kevinstephano @rdspring1 for review) Pull Request resolved: pytorch#75646 Approved by: https://github.com/rdspring1, https://github.com/kevinstephano, https://github.com/ngimel

@kevinstephano

…er (#1588) Fixes pytorch#75622 1. Instead of getting max/min_value for reduction init value, we go with (-)infinity instead so we can properly preserve inf inputs; 2. Adding inf/(-)inf/nan for float value. 3. Adding aten::amin in nvfuser (@kevinstephano @rdspring1 for review) Pull Request resolved: pytorch#75646 Approved by: https://github.com/rdspring1, https://github.com/kevinstephano, https://github.com/ngimel

@kevinstephano

Fixes pytorch#75622 1. Instead of getting max/min_value for reduction init value, we go with (-)infinity instead so we can properly preserve inf inputs; 2. Adding inf/(-)inf/nan for float value. 3. Adding aten::amin in nvfuser (@kevinstephano @rdspring1 for review) Pull Request resolved: pytorch#75646 Approved by: https://github.com/rdspring1, https://github.com/kevinstephano, https://github.com/ngimel

@kevinstephano

Fixes #75622 1. Instead of getting max/min_value for reduction init value, we go with (-)infinity instead so we can properly preserve inf inputs; 2. Adding inf/(-)inf/nan for float value. 3. Adding aten::amin in nvfuser (@kevinstephano @rdspring1 for review) Pull Request resolved: pytorch/pytorch#75646 Approved by: https://github.com/rdspring1, https://github.com/kevinstephano, https://github.com/ngimel

@kevinstephano

Fixes #75622 1. Instead of getting max/min_value for reduction init value, we go with (-)infinity instead so we can properly preserve inf inputs; 2. Adding inf/(-)inf/nan for float value. 3. Adding aten::amin in nvfuser (@kevinstephano @rdspring1 for review) Pull Request resolved: pytorch/pytorch#75646 Approved by: https://github.com/rdspring1, https://github.com/kevinstephano, https://github.com/ngimel

@kevinstephano

…er (#1588) Fixes #75622 1. Instead of getting max/min_value for reduction init value, we go with (-)infinity instead so we can properly preserve inf inputs; 2. Adding inf/(-)inf/nan for float value. 3. Adding aten::amin in nvfuser (@kevinstephano @rdspring1 for review) Pull Request resolved: pytorch/pytorch#75646 Approved by: https://github.com/rdspring1, https://github.com/kevinstephano, https://github.com/ngimel

baby steps on patching inf/nan behavior

76e8fc0

facebook-github-bot added the cla signed label Apr 12, 2022

facebook-github-bot added the oncall: jit Add this issue/PR to JIT oncall triage queue label Apr 12, 2022

jjsjann123 requested a review from ngimel April 12, 2022 02:57

Merge remote-tracking branch 'upstream/master' into HEAD

a0c49da

jjsjann123 mentioned this pull request Apr 12, 2022

NVFuser incorrectly computes max for extremal values #75622

Closed

pytorchbot added the open source label Apr 12, 2022

jjsjann123 added 2 commits April 11, 2022 21:10

flake8 clang-tidy

5bee616

adding aten::amin support in nvfuser

79567d3

jjsjann123 changed the title ~~baby steps on patching inf/nan behavior~~ baby steps on patching inf/nan behavior & aten::amin support in nvfuser Apr 12, 2022

disabling reduction test on pre-volta device

64e189b

ngimel reviewed Apr 12, 2022

View reviewed changes

rdspring1 approved these changes Apr 12, 2022

View reviewed changes

kevinstephano approved these changes Apr 12, 2022

View reviewed changes

updating tests

80c232e

jjsjann123 requested a review from ngimel April 12, 2022 21:02

ngimel approved these changes Apr 13, 2022

View reviewed changes

pytorchmergebot closed this in 692ebc8 Apr 13, 2022

jjsjann123 mentioned this pull request Apr 17, 2022

baby steps on patching inf/nan behavior & aten::amin support in nvfuser csarofeen/pytorch#1588

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

baby steps on patching inf/nan behavior & aten::amin support in nvfuser#75646

baby steps on patching inf/nan behavior & aten::amin support in nvfuser#75646
jjsjann123 wants to merge 6 commits intopytorch:masterfrom
jjsjann123:inf_nan_patch

jjsjann123 commented Apr 12, 2022 •

edited

Loading

Uh oh!

facebook-github-bot commented Apr 12, 2022 •

edited

Loading

Uh oh!

jjsjann123 commented Apr 12, 2022 •

edited

Loading

Uh oh!

ngimel Apr 12, 2022

Uh oh!

rdspring1 left a comment

Uh oh!

kevinstephano left a comment

Uh oh!

jjsjann123 commented Apr 13, 2022

Uh oh!

github-actions bot commented Apr 13, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

jjsjann123 commented Apr 12, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facebook-github-bot commented Apr 12, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful links

💊 CI failures summary and remediations

Uh oh!

jjsjann123 commented Apr 12, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ngimel Apr 12, 2022

Choose a reason for hiding this comment

Uh oh!

rdspring1 left a comment

Choose a reason for hiding this comment

Uh oh!

kevinstephano left a comment

Choose a reason for hiding this comment

Uh oh!

jjsjann123 commented Apr 13, 2022

Uh oh!

github-actions bot commented Apr 13, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

jjsjann123 commented Apr 12, 2022 •

edited

Loading

facebook-github-bot commented Apr 12, 2022 •

edited

Loading

jjsjann123 commented Apr 12, 2022 •

edited

Loading