Updated foreach pointwise ops to support complex and bloat16 on CUDA by izdeby · Pull Request #51170 · pytorch/pytorch

izdeby · 2021-01-27T01:21:04Z

Stack from ghstack:

wip #52450 wip
[wip] Replace optimizers in torch.optim with the ones from torch.optim._multi_tensor #49039 [wip] Replace optimizers in torch.optim with the ones from torch.optim._multi_tensor
[WIP] Add type promotion to a fast path for foreach APIs #52449 [WIP] Add type promotion to a fast path for foreach APIs
[WIP] Support broadcasting via slow path #52448 [WIP] Support broadcasting via slow path
[wip] Pushed unsupported scenarios to slow path #48224 [wip] Pushed unsupported scenarios to slow path
Updated foreach pointwise ops to support complex and bloat16 on CUDA #51170 Updated foreach pointwise ops to support complex and bloat16 on CUDA
Update foreach max/min #49714 Update foreach max/min
Refactor foreach binary ops tests with tensor lists to use OpInfo #51059 Refactor foreach binary ops tests with tensor lists to use OpInfo
Refactor foreach binary ops tests with scalars to use OpInfo #51058 Refactor foreach binary ops tests with scalars to use OpInfo

Updated foreach pointwise ops to support complex and bloat16 on CUDA

Differential Revision: D26103906

[ghstack-poisoned]

facebook-github-bot · 2021-01-27T01:21:16Z

💊 CI failures summary and remediations

As of commit bcff391 (more details on the Dr. CI page):

4/4 failures possibly* introduced in this PR
- 1/4 non-scanned failure(s)

🕵️ 2 new failures recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

pytorch_windows_vs2019_py36_cuda10.1_test1 (1/2)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

AssertionError: False is not true : Tensors failed to compare as equal!Real parts failed to compare as equal! With rtol=1.3e-06 and atol=1e-05, found 1 element(s) (out of 400) whose difference(s) exceeded the margin of error (including 0 nan comparisons). The greatest difference was 2.09808349609375e-05 (2.9861230850219727 vs. 2.9861440658569336), which occurred at index (18, 9).

  File "C:\Users\circleci\project\build\win_tmp\build\torch\testing\_internal\common_device_type.py", line 292, in instantiated_test
    result = test_fn(self, *args)
  File "C:\Users\circleci\project\build\win_tmp\build\torch\testing\_internal\common_device_type.py", line 266, in test_wrapper
    return test(*args, **kwargs)
  File "test_foreach.py", line 307, in test_pointwise_ops
    self.assertEqual(tensors, expected)
  File "C:\Users\circleci\project\build\win_tmp\build\torch\testing\_internal\common_utils.py", line 1309, in assertEqual
    exact_dtype=exact_dtype, exact_device=exact_device)
  File "C:\Users\circleci\project\build\win_tmp\build\torch\testing\_internal\common_utils.py", line 1275, in assertEqual
    super().assertTrue(result, msg=self._get_assert_msg(msg, debug_msg=debug_msg))
AssertionError: False is not true : Tensors failed to compare as equal!Real parts failed to compare as equal! With rtol=1.3e-06 and atol=1e-05, found 1 element(s) (out of 400) whose difference(s) exceeded the margin of error (including 0 nan comparisons). The greatest difference was 2.09808349609375e-05 (2.9861230850219727 vs. 2.9861440658569336), which occurred at index (18, 9).

----------------------------------------------------------------------
Ran 827 tests in 29.902s

FAILED (failures=1)

Generating XML reports...
Generated XML report: test-reports\dist-gloo\test_foreach\TEST-TestForeachCPU-20210401193543.xml
Generated XML report: test-reports\dist-gloo\test_foreach\TEST-TestForeachCUDA-20210401193543.xml
Traceback (most recent call last):

pytorch_linux_xenial_cuda10_2_cudnn7_py3_gcc7_test2 (2/2)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Apr 01 20:35:50 AssertionError: False is not true : Tensors failed to compare as equal!Real parts failed to compare as equal! With rtol=1.3e-06 and atol=1e-05, found 1 element(s) (out of 400) whose difference(s) exceeded the margin of error (including 0 nan comparisons). The greatest difference was 2.09808349609375e-05 (2.9861230850219727 vs. 2.9861440658569336), which occurred at index (18, 9).

Apr 01 20:35:50   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 292, in instantiated_test
Apr 01 20:35:50     result = test_fn(self, *args)
Apr 01 20:35:50   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 266, in test_wrapper
Apr 01 20:35:50     return test(*args, **kwargs)
Apr 01 20:35:50   File "test_foreach.py", line 307, in test_pointwise_ops
Apr 01 20:35:50     self.assertEqual(tensors, expected)
Apr 01 20:35:50   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_utils.py", line 1309, in assertEqual
Apr 01 20:35:50     exact_dtype=exact_dtype, exact_device=exact_device)
Apr 01 20:35:50   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_utils.py", line 1275, in assertEqual
Apr 01 20:35:50     super().assertTrue(result, msg=self._get_assert_msg(msg, debug_msg=debug_msg))
Apr 01 20:35:50 AssertionError: False is not true : Tensors failed to compare as equal!Real parts failed to compare as equal! With rtol=1.3e-06 and atol=1e-05, found 1 element(s) (out of 400) whose difference(s) exceeded the margin of error (including 0 nan comparisons). The greatest difference was 2.09808349609375e-05 (2.9861230850219727 vs. 2.9861440658569336), which occurred at index (18, 9).
Apr 01 20:35:50 
Apr 01 20:35:50 ----------------------------------------------------------------------
Apr 01 20:35:50 Ran 827 tests in 56.157s
Apr 01 20:35:50 
Apr 01 20:35:50 FAILED (failures=1)
Apr 01 20:35:50 
Apr 01 20:35:50 Generating XML reports...
Apr 01 20:35:50 Generated XML report: test-reports/dist-gloo/test_foreach/TEST-TestForeachCPU-20210401203454.xml
Apr 01 20:35:50 Generated XML report: test-reports/dist-gloo/test_foreach/TEST-TestForeachCUDA-20210401203454.xml
Apr 01 20:35:51 Traceback (most recent call last):

1 failure not recognized by patterns:

Job	Step	Action
^{pytorch_windows_vs2019_py36_cuda10.1_test2}	^{Install Cudnn}	🔁 rerun

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

…16 on CUDA" -------- Updated foreach pointwise ops to support complex and bloat16 on CUDA [ghstack-poisoned]

…16 on CUDA" -------- Updated foreach pointwise ops to support complex and bloat16 on CUDA Differential Revision: [D26103906](https://our.internmc.facebook.com/intern/diff/D26103906) [ghstack-poisoned]

ghstack-source-id: e5b80bf Pull Request resolved: #51170

…16 on CUDA" -------- Updated foreach pointwise ops to support complex and bloat16 on CUDA Differential Revision: [D26103906](https://our.internmc.facebook.com/intern/diff/D26103906) [ghstack-poisoned]

facebook-github-bot · 2021-04-09T09:44:03Z

Hi @izdeby!

Thank you for your pull request.

We require contributors to sign our Contributor License Agreement, and yours needs attention.

You currently have a record in our system, but the CLA is no longer valid, and will need to be resubmitted.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@fb.com. Thanks!

Updated foreach pointwise ops to support complex and bloat16 on CUDA

795a36d

[ghstack-poisoned]

facebook-github-bot added the cla signed label Jan 27, 2021

This was referenced Jan 27, 2021

Refactor foreach binary ops tests with scalars to use OpInfo #51058

Closed

Refactor foreach binary ops tests with tensor lists to use OpInfo #51059

Closed

Refactor foreach pointwise ops tests to use OpInfo #51060

Closed

This was referenced Jan 27, 2021

Update foreach max/min #49714

Closed

[wip] Pushed unsupported scenarios to slow path #48224

Closed

[wip] Replace optimizers in torch.optim with the ones from torch.optim._multi_tensor #49039

Closed

Update on "Updated foreach pointwise ops to support complex and bloat…

b737404

…16 on CUDA" -------- Updated foreach pointwise ops to support complex and bloat16 on CUDA [ghstack-poisoned]

izdeby requested review from gchanan, ngimel and zou3519 January 27, 2021 15:27

Iurii Zdebskyi added 5 commits January 27, 2021 09:23

Update on "Updated foreach pointwise ops to support complex and bloat…

a6cfaf0

…16 on CUDA" -------- Updated foreach pointwise ops to support complex and bloat16 on CUDA [ghstack-poisoned]

Update on "Updated foreach pointwise ops to support complex and bloat…

4d91ef5

…16 on CUDA" -------- Updated foreach pointwise ops to support complex and bloat16 on CUDA Differential Revision: [D26103906](https://our.internmc.facebook.com/intern/diff/D26103906) [ghstack-poisoned]

Update on "Updated foreach pointwise ops to support complex and bloat…

5b4eb0b

…16 on CUDA" -------- Updated foreach pointwise ops to support complex and bloat16 on CUDA Differential Revision: [D26103906](https://our.internmc.facebook.com/intern/diff/D26103906) [ghstack-poisoned]

Update on "Updated foreach pointwise ops to support complex and bloat…

e0c5e08

…16 on CUDA" -------- Updated foreach pointwise ops to support complex and bloat16 on CUDA Differential Revision: [D26103906](https://our.internmc.facebook.com/intern/diff/D26103906) [ghstack-poisoned]

Update on "Updated foreach pointwise ops to support complex and bloat…

e2aa212

…16 on CUDA" -------- Updated foreach pointwise ops to support complex and bloat16 on CUDA Differential Revision: [D26103906](https://our.internmc.facebook.com/intern/diff/D26103906) [ghstack-poisoned]

izdeby pushed a commit that referenced this pull request Feb 4, 2021

Updated foreach pointwise ops to support complex and bloat16 on CUDA

abdde18

ghstack-source-id: e5b80bf Pull Request resolved: #51170

Update on "Updated foreach pointwise ops to support complex and bloat…

e67a366

…16 on CUDA" -------- Updated foreach pointwise ops to support complex and bloat16 on CUDA Differential Revision: [D26103906](https://our.internmc.facebook.com/intern/diff/D26103906) [ghstack-poisoned]

This was referenced Feb 8, 2021

Update foreach APIs to use scalar lists #51893

Closed

Refactor ForeachUnaryOp.cu #51894

Closed

Update on "Updated foreach pointwise ops to support complex and bloat…

d08182e

…16 on CUDA" -------- Updated foreach pointwise ops to support complex and bloat16 on CUDA Differential Revision: [D26103906](https://our.internmc.facebook.com/intern/diff/D26103906) [ghstack-poisoned]

izdeby mentioned this pull request Feb 18, 2021

[WIP] Support broadcasting via slow path #52448

Closed

This was referenced Feb 18, 2021

[WIP] Add type promotion to a fast path for foreach APIs #52449

Closed

wip #52450

Closed

Iurii Zdebskyi and others added 17 commits February 18, 2021 12:16

Update on "Updated foreach pointwise ops to support complex and bloat…

a243f3a

…16 on CUDA" -------- Updated foreach pointwise ops to support complex and bloat16 on CUDA Differential Revision: [D26103906](https://our.internmc.facebook.com/intern/diff/D26103906) [ghstack-poisoned]

Update on "Updated foreach pointwise ops to support complex and bloat…

f55f682

…16 on CUDA" -------- Updated foreach pointwise ops to support complex and bloat16 on CUDA Differential Revision: [D26103906](https://our.internmc.facebook.com/intern/diff/D26103906) [ghstack-poisoned]

Update on "Updated foreach pointwise ops to support complex and bloat…

85db29a

…16 on CUDA" -------- Updated foreach pointwise ops to support complex and bloat16 on CUDA Differential Revision: [D26103906](https://our.internmc.facebook.com/intern/diff/D26103906) [ghstack-poisoned]

Update on "Updated foreach pointwise ops to support complex and bloat…

3cc789d

…16 on CUDA" -------- Updated foreach pointwise ops to support complex and bloat16 on CUDA Differential Revision: [D26103906](https://our.internmc.facebook.com/intern/diff/D26103906) [ghstack-poisoned]

Update on "Updated foreach pointwise ops to support complex and bloat…

2cd6b7c

…16 on CUDA" -------- Updated foreach pointwise ops to support complex and bloat16 on CUDA Differential Revision: [D26103906](https://our.internmc.facebook.com/intern/diff/D26103906) [ghstack-poisoned]

Update on "Updated foreach pointwise ops to support complex and bloat…

a2efeca

…16 on CUDA" -------- Updated foreach pointwise ops to support complex and bloat16 on CUDA Differential Revision: [D26103906](https://our.internmc.facebook.com/intern/diff/D26103906) [ghstack-poisoned]

Update on "Updated foreach pointwise ops to support complex and bloat…

53270f0

…16 on CUDA" -------- Updated foreach pointwise ops to support complex and bloat16 on CUDA Differential Revision: [D26103906](https://our.internmc.facebook.com/intern/diff/D26103906) [ghstack-poisoned]

Update on "Updated foreach pointwise ops to support complex and bloat…

5acd972

…16 on CUDA" -------- Updated foreach pointwise ops to support complex and bloat16 on CUDA Differential Revision: [D26103906](https://our.internmc.facebook.com/intern/diff/D26103906) [ghstack-poisoned]

Update on "Updated foreach pointwise ops to support complex and bloat…

19582b4

…16 on CUDA" -------- Updated foreach pointwise ops to support complex and bloat16 on CUDA Differential Revision: [D26103906](https://our.internmc.facebook.com/intern/diff/D26103906) [ghstack-poisoned]

Update on "Updated foreach pointwise ops to support complex and bloat…

272dc34

…16 on CUDA" -------- Updated foreach pointwise ops to support complex and bloat16 on CUDA Differential Revision: [D26103906](https://our.internmc.facebook.com/intern/diff/D26103906) [ghstack-poisoned]

Update on "Updated foreach pointwise ops to support complex and bloat…

99317f0

…16 on CUDA" -------- Updated foreach pointwise ops to support complex and bloat16 on CUDA Differential Revision: [D26103906](https://our.internmc.facebook.com/intern/diff/D26103906) [ghstack-poisoned]

Update on "Updated foreach pointwise ops to support complex and bloat…

dccd8b7

…16 on CUDA" -------- Updated foreach pointwise ops to support complex and bloat16 on CUDA Differential Revision: [D26103906](https://our.internmc.facebook.com/intern/diff/D26103906) [ghstack-poisoned]

Update on "Updated foreach pointwise ops to support complex and bloat…

dad940f

…16 on CUDA" -------- Updated foreach pointwise ops to support complex and bloat16 on CUDA Differential Revision: [D26103906](https://our.internmc.facebook.com/intern/diff/D26103906) [ghstack-poisoned]

Update on "Updated foreach pointwise ops to support complex and bloat…

de6c84e

…16 on CUDA" -------- Updated foreach pointwise ops to support complex and bloat16 on CUDA Differential Revision: [D26103906](https://our.internmc.facebook.com/intern/diff/D26103906) [ghstack-poisoned]

Update on "Updated foreach pointwise ops to support complex and bloat…

e8326c3

…16 on CUDA" -------- Updated foreach pointwise ops to support complex and bloat16 on CUDA Differential Revision: [D26103906](https://our.internmc.facebook.com/intern/diff/D26103906) [ghstack-poisoned]

Update on "Updated foreach pointwise ops to support complex and bloat…

90e4e39

…16 on CUDA" -------- Updated foreach pointwise ops to support complex and bloat16 on CUDA Differential Revision: [D26103906](https://our.internmc.facebook.com/intern/diff/D26103906) [ghstack-poisoned]

Update on "Updated foreach pointwise ops to support complex and bloat…

bcff391

…16 on CUDA" -------- Updated foreach pointwise ops to support complex and bloat16 on CUDA Differential Revision: [D26103906](https://our.internmc.facebook.com/intern/diff/D26103906) [ghstack-poisoned]

pytorchbot added the open source label Apr 9, 2021

ngimel removed their request for review May 30, 2021 23:39

zou3519 removed their request for review June 28, 2021 13:31

pytorchbot added the Stale label Apr 12, 2022

github-actions bot closed this May 12, 2022

facebook-github-bot deleted the gh/izdeby/79/head branch June 11, 2022 14:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updated foreach pointwise ops to support complex and bloat16 on CUDA#51170

Updated foreach pointwise ops to support complex and bloat16 on CUDA#51170
izdeby wants to merge 26 commits intogh/izdeby/79/basefrom
gh/izdeby/79/head

izdeby commented Jan 27, 2021 •

edited

Loading

Uh oh!

facebook-github-bot commented Jan 27, 2021 •

edited

Loading

Uh oh!

facebook-github-bot commented Apr 9, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

izdeby commented Jan 27, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facebook-github-bot commented Jan 27, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CI failures summary and remediations

🕵️ 2 new failures recognized by patterns

pytorch_windows_vs2019_py36_cuda10.1_test1 (1/2)

pytorch_linux_xenial_cuda10_2_cudnn7_py3_gcc7_test2 (2/2)

1 failure not recognized by patterns:

Uh oh!

facebook-github-bot commented Apr 9, 2021

Process

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

izdeby commented Jan 27, 2021 •

edited

Loading

facebook-github-bot commented Jan 27, 2021 •

edited

Loading