Update foreach binary ops with a single scalar and list of tensors by izdeby · Pull Request #49250 · pytorch/pytorch

izdeby · 2020-12-11T19:28:25Z

Stack from ghstack:

wip #52450 wip
[wip] Replace optimizers in torch.optim with the ones from torch.optim._multi_tensor #49039 [wip] Replace optimizers in torch.optim with the ones from torch.optim._multi_tensor
[WIP] Add type promotion to a fast path for foreach APIs #52449 Add type promotion to a fast path for foreach APIs
[WIP] Support broadcasting via slow path #52448 Support broadcasting via slow path
[wip] Pushed unsupported scenarios to slow path #48224 [wip] Pushed unsupported scenarios to slow path
Updated foreach pointwise ops to support complex and bloat16 on CUDA #51170 Updated foreach pointwise ops to support complex and bloat16 on CUDA
Update foreach max/min #49714 Update foreach max/min
Refactor foreach pointwise ops tests to use OpInfo #51060 Refactor foreach pointwise ops tests to use OpInfo
Refactor foreach binary ops tests with tensor lists to use OpInfo #51059 Refactor foreach binary ops tests with tensor lists to use OpInfo
Refactor foreach binary ops tests with scalars to use OpInfo #51058 Refactor foreach binary ops tests with scalars to use OpInfo
Refactor foreach unary ops tests to use OpInfo #49712 Refactor foreach unary ops tests to use OpInfo
Update foreach binary ops with a single scalar and list of tensors #49250 Update foreach binary ops with a single scalar and list of tensors
Update foreach binary ops with a scalar list #49249 Update foreach binary ops with a scalar list
Refactor ForeachUtils.h #51131 Refactor ForeachUtils.h

Differential Revision: D25502938

Update logic for 2 sets of the foreach APIs:

Binary ops with a scalar
Binary ops with a list of tensors

Update foreach_sub error checking
Update division logic to check for type promotion
Updated tests

[ghstack-poisoned]

Differential Revision: [D25502938](https://our.internmc.facebook.com/intern/diff/D25502938) [ghstack-poisoned]

Differential Revision: [D25502938](https://our.internmc.facebook.com/intern/diff/D25502938) ---- - Update ForeachBinaryOpScalar.cu to throw better errors in case of sub - Update division logic to check for type promotion [ghstack-poisoned]

Differential Revision: [D25502938](https://our.internmc.facebook.com/intern/diff/D25502938) ---- - Update foreach_sub error checking - Update division logic to check for type promotion - Updated tests [ghstack-poisoned]

ghstack-source-id: b2ea442 Pull Request resolved: #49250

Differential Revision: [D25502938](https://our.internmc.facebook.com/intern/diff/D25502938) ---- - Update foreach_sub error checking - Update division logic to check for type promotion - Updated tests [ghstack-poisoned]

… tensors" Differential Revision: [D25502938](https://our.internmc.facebook.com/intern/diff/D25502938) ---- Update logic for 2 set of foreach APIs: 1) Binary ops with a scalar 2) Binary ops with a list of tensors - Update foreach_sub error checking - Update division logic to check for type promotion - Updated tests [ghstack-poisoned]

… tensors" Differential Revision: [D25502938](https://our.internmc.facebook.com/intern/diff/D25502938) ---- Update logic for 2 sets of the foreach APIs: 1) Binary ops with a scalar 2) Binary ops with a list of tensors - Update foreach_sub error checking - Update division logic to check for type promotion - Updated tests [ghstack-poisoned]

zou3519

Had some minor comments. Splitting up the PRs really helped, this was easy to review

zou3519 · 2021-03-11T22:24:30Z

test/test_foreach.py

-                res = foreach_bin_op(tensors, scalar)

                if dtype == torch.bool:
+                    if foreach_bin_op == torch._foreach_sub:


nit: this test is called test_int_scalar. That's not very descriptive, it looks like what is being tested here is binary ops where the second argument is an int. Maybe something like test_tensorlist_scalar_op?

zou3519 · 2021-03-11T22:28:46Z

test/test_foreach.py

+                self.assertEqual(res, expected)
+
+                # In case of In-place op, we can't change the dtype
+                foreach_bin_op_(tensors, scalar)


NB: this modifies the input "tensors" in-place, which means that the values are different the next iteration of the outer for-loop. But that seems to be okay. Usually to avoid mutating we would clone the input before performing the in-place operation.

No action item required here. It may possibly be nicer if we cloned tensors before calling the in-place op but that's subjective. I also see this pattern in the other test that is being modified in this PR.

… tensors" Differential Revision: [D25502938](https://our.internmc.facebook.com/intern/diff/D25502938) ---- Update logic for 2 sets of the foreach APIs: 1) Binary ops with a scalar 2) Binary ops with a list of tensors - Update foreach_sub error checking - Update division logic to check for type promotion - Updated tests [ghstack-poisoned]

facebook-github-bot · 2021-03-15T19:17:59Z

@izdeby merged this pull request in b5cdb53.

Add division logic to a slow/fast path

d2a2a18

[ghstack-poisoned]

This was referenced Dec 11, 2020

Enabled Scalar lists #48222

Closed

Update foreach APIs to use scalar lists #48223

Closed

Refactor ForeachUnaryOp.cu #49248

Closed

facebook-github-bot added the cla signed label Dec 11, 2020

This was referenced Dec 11, 2020

Update foreach binary ops with a scalar list #49249

Closed

[wip] Pushed unsupported scenarios to slow path #48224

Closed

[wip] Replace optimizers in torch.optim with the ones from torch.optim._multi_tensor #49039

Closed

Iurii Zdebskyi added 2 commits December 11, 2020 12:31

Update on "Add division logic to a slow/fast path"

8cb8c17

Differential Revision: [D25502938](https://our.internmc.facebook.com/intern/diff/D25502938) [ghstack-poisoned]

Update on "Add division logic to a slow/fast path"

978ae5b

Differential Revision: [D25502938](https://our.internmc.facebook.com/intern/diff/D25502938) [ghstack-poisoned]

izdeby changed the title ~~Add division logic to a slow/fast path~~ [wip] Add division logic to a slow/fast path Dec 11, 2020

Iurii Zdebskyi added 2 commits December 15, 2020 10:53

Update on "[wip] Add division logic to a slow/fast path"

1325836

Differential Revision: [D25502938](https://our.internmc.facebook.com/intern/diff/D25502938) [ghstack-poisoned]

Update on "[wip] Add division logic to a slow/fast path"

e92abe1

Differential Revision: [D25502938](https://our.internmc.facebook.com/intern/diff/D25502938) [ghstack-poisoned]

izdeby changed the title ~~[wip] Add division logic to a slow/fast path~~ Update ForeachBinaryOpScalar.cu Dec 15, 2020

Iurii Zdebskyi added 2 commits December 15, 2020 15:08

Update on "Update ForeachBinaryOpScalar.cu"

74df416

Differential Revision: [D25502938](https://our.internmc.facebook.com/intern/diff/D25502938) [ghstack-poisoned]

Update on "Update ForeachBinaryOpScalar.cu"

4034302

Differential Revision: [D25502938](https://our.internmc.facebook.com/intern/diff/D25502938) [ghstack-poisoned]

izdeby requested review from ngimel and zou3519 and removed request for ngimel December 16, 2020 16:30

Iurii Zdebskyi added 6 commits December 16, 2020 08:33

Update on "Update ForeachBinaryOpScalar.cu"

2979a4c

Differential Revision: [D25502938](https://our.internmc.facebook.com/intern/diff/D25502938) ---- - Update ForeachBinaryOpScalar.cu to throw better errors in case of sub - Update division logic to check for type promotion [ghstack-poisoned]

Update on "Update ForeachBinaryOpScalar.cu"

4dad93f

Differential Revision: [D25502938](https://our.internmc.facebook.com/intern/diff/D25502938) ---- - Update ForeachBinaryOpScalar.cu to throw better errors in case of sub - Update division logic to check for type promotion [ghstack-poisoned]

Update on "Update ForeachBinaryOpScalar.cu"

e8e466f

Differential Revision: [D25502938](https://our.internmc.facebook.com/intern/diff/D25502938) ---- - Update ForeachBinaryOpScalar.cu to throw better errors in case of sub - Update division logic to check for type promotion [ghstack-poisoned]

Update on "Update ForeachBinaryOpScalar.cu"

5a725ec

Differential Revision: [D25502938](https://our.internmc.facebook.com/intern/diff/D25502938) ---- - Update ForeachBinaryOpScalar.cu to throw better errors in case of sub - Update division logic to check for type promotion [ghstack-poisoned]

Update on "Update ForeachBinaryOpScalar.cu"

9e56038

Differential Revision: [D25502938](https://our.internmc.facebook.com/intern/diff/D25502938) ---- - Update ForeachBinaryOpScalar.cu to throw better errors in case of sub - Update division logic to check for type promotion [ghstack-poisoned]

Update on "Update ForeachBinaryOpScalar.cu"

7a011ae

Differential Revision: [D25502938](https://our.internmc.facebook.com/intern/diff/D25502938) ---- - Update ForeachBinaryOpScalar.cu to throw better errors in case of sub - Update division logic to check for type promotion [ghstack-poisoned]

izdeby changed the title ~~Update ForeachBinaryOpScalar.cu~~ Update ForeachBinaryOpScalar.cu and ForeachBinaryOpList.cu Dec 21, 2020

This was referenced Dec 21, 2020

Refactor foreach unary ops tests to use OpInfo #49712

Closed

Update foreach max/min #49714

Closed

Iurii Zdebskyi added 3 commits January 29, 2021 09:28

Update on "Update foreach_div and foreach_sub logic"

28c3dae

Differential Revision: [D25502938](https://our.internmc.facebook.com/intern/diff/D25502938) ---- - Update foreach_sub error checking - Update division logic to check for type promotion - Updated tests [ghstack-poisoned]

Update on "Update foreach_div and foreach_sub logic"

65a5868

Differential Revision: [D25502938](https://our.internmc.facebook.com/intern/diff/D25502938) ---- - Update foreach_sub error checking - Update division logic to check for type promotion - Updated tests [ghstack-poisoned]

Update on "Update foreach_div and foreach_sub logic"

76143f0

Differential Revision: [D25502938](https://our.internmc.facebook.com/intern/diff/D25502938) ---- - Update foreach_sub error checking - Update division logic to check for type promotion - Updated tests [ghstack-poisoned]

izdeby pushed a commit that referenced this pull request Feb 4, 2021

Update foreach_div and foreach_sub logic

65f5bac

ghstack-source-id: b2ea442 Pull Request resolved: #49250

Update on "Update foreach_div and foreach_sub logic"

f1068eb

Differential Revision: [D25502938](https://our.internmc.facebook.com/intern/diff/D25502938) ---- - Update foreach_sub error checking - Update division logic to check for type promotion - Updated tests [ghstack-poisoned]

This was referenced Feb 8, 2021

Update foreach APIs to use scalar lists #51893

Closed

Refactor ForeachUnaryOp.cu #51894

Closed

Update on "Update foreach_div and foreach_sub logic"

7e416c7

Differential Revision: [D25502938](https://our.internmc.facebook.com/intern/diff/D25502938) ---- - Update foreach_sub error checking - Update division logic to check for type promotion - Updated tests [ghstack-poisoned]

This was referenced Feb 18, 2021

[WIP] Support broadcasting via slow path #52448

Closed

[WIP] Add type promotion to a fast path for foreach APIs #52449

Closed

wip #52450

Closed

izdeby changed the title ~~Update foreach_div and foreach_sub logic~~ Update foreach binary ops with a single scalar and list of tensors Feb 18, 2021

Iurii Zdebskyi and others added 9 commits February 18, 2021 12:15

zou3519 approved these changes Mar 11, 2021

View reviewed changes

izdeby added 3 commits March 12, 2021 10:56

facebook-github-bot closed this in b5cdb53 Mar 15, 2021

facebook-github-bot added the Merged label Mar 15, 2021

facebook-github-bot deleted the gh/izdeby/72/head branch March 19, 2021 14:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update foreach binary ops with a single scalar and list of tensors#49250

Update foreach binary ops with a single scalar and list of tensors#49250
izdeby wants to merge 40 commits intogh/izdeby/72/basefrom
gh/izdeby/72/head

izdeby commented Dec 11, 2020 •

edited

Loading

Uh oh!

zou3519 left a comment

Uh oh!

zou3519 Mar 11, 2021

Uh oh!

zou3519 Mar 11, 2021

Uh oh!

facebook-github-bot commented Mar 15, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

izdeby commented Dec 11, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zou3519 left a comment

Choose a reason for hiding this comment

Uh oh!

zou3519 Mar 11, 2021

Choose a reason for hiding this comment

Uh oh!

zou3519 Mar 11, 2021

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Mar 15, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

izdeby commented Dec 11, 2020 •

edited

Loading