[wip] Pushed unsupported scenarios to slow path#48224
Closed
izdeby wants to merge 56 commits intogh/izdeby/68/basefrom
Closed
[wip] Pushed unsupported scenarios to slow path#48224izdeby wants to merge 56 commits intogh/izdeby/68/basefrom
izdeby wants to merge 56 commits intogh/izdeby/68/basefrom
Conversation
[ghstack-poisoned]
This was referenced Nov 19, 2020
Closed
💊 CI failures summary and remediationsAs of commit 156288b (more details on the Dr. CI page):
🕵️ 7 new failures recognized by patternsThe following CI failures do not appear to be due to upstream breakages:
|
| Job | Step | Action |
|---|---|---|
| Ensure no trailing spaces | 🔁 rerun | |
| Fail if there were any warnings | 🔁 rerun |
ci.pytorch.org: 1 failed
This comment was automatically generated by Dr. CI (expand for details).
Follow this link to opt-out of these comments for your Pull Requests.Please report bugs/suggestions to the (internal) Dr. CI Users group.
Differential Revision: [D25074764](https://our.internmc.facebook.com/intern/diff/D25074764) [ghstack-poisoned]
Differential Revision: [D25074764](https://our.internmc.facebook.com/intern/diff/D25074764) [ghstack-poisoned]
Differential Revision: [D25074764](https://our.internmc.facebook.com/intern/diff/D25074764) Coming next for this stack: 1. Docs PR. Add more code documentation 2. Swap optimizers from torch.optim and torch.optim.multi_tensor -------------------------------- **Motivation** _foreach APIs should produce the exact same result as if they were implemented via for-loop implementation but have significant performance gains in particular scenarios. This PR achieves this. **What was done** - Stopped throwing an error in case when previous API restrictions were not met. Instead, execution will go via slow path if possible. - Updated code logic to acknowledge division op. This is a temp fix until MTA wont support type promotion. In case of integer input and division, we should go via slow path as MTA doesn't support return of a different dtype. - Significantly extended type support for binary/unary/pointwise ops. with this PR, _foreach APIs support exactly same list of types as their analogs in torch. - Completely rewrote test/test_foreach.py, see testing section **Testing** This is an arguable change. Im open to reconsidering. - Refactoring I've rewritten the whole test suite to have individual test per operator rather than having one master test with many else/if conditions. In my opinion, the tests became much more readable and easier to maintain. - Coverage and correctness Each _foreach API has its dedicated test where all possible dtypes/ combination of dtypes are being tested and compared to their counter parts in torch. [ghstack-poisoned]
Differential Revision: [D25074764](https://our.internmc.facebook.com/intern/diff/D25074764) Coming next for this stack: 1. Docs PR. Add more code documentation 2. Swap optimizers from torch.optim and torch.optim.multi_tensor -------------------------------- **Motivation** _foreach APIs should produce the exact same result as if they were implemented via for-loop implementation but have significant performance gains in particular scenarios. This PR achieves this. **What was done** - Stopped throwing an error in case when previous API restrictions were not met. Instead, execution will go via slow path if possible. - Updated code logic to acknowledge division op. This is a temp fix until MTA wont support type promotion. In case of integer input and division, we should go via slow path as MTA doesn't support return of a different dtype. - Significantly extended type support for binary/unary/pointwise ops. with this PR, _foreach APIs support exactly same list of types as their analogs in torch. - Completely rewrote test/test_foreach.py, see testing section **Testing** This is an arguable change. Im open to reconsidering. - Refactoring I've rewritten the whole test suite to have individual test per operator rather than having one master test with many else/if conditions. In my opinion, the tests became much more readable and easier to maintain. - Coverage and correctness Each _foreach API has its dedicated test where all possible dtypes/ combination of dtypes are being tested and compared to their counter parts in torch. [ghstack-poisoned]
added 4 commits
December 8, 2020 14:51
Differential Revision: [D25074764](https://our.internmc.facebook.com/intern/diff/D25074764) Coming next for this stack: 1. Docs PR. Add more code documentation 2. Swap optimizers from torch.optim and torch.optim.multi_tensor -------------------------------- **Motivation** _foreach APIs should produce the exact same result as if they were implemented via for-loop implementation but have significant performance gains in particular scenarios. This PR achieves this. **What was done** - Stopped throwing an error in case when previous API restrictions were not met. Instead, execution will go via slow path if possible. - Updated code logic to acknowledge division op. This is a temp fix until MTA wont support type promotion. In case of integer input and division, we should go via slow path as MTA doesn't support return of a different dtype. - Significantly extended type support for binary/unary/pointwise ops. with this PR, _foreach APIs support exactly same list of types as their analogs in torch. - Completely rewrote test/test_foreach.py, see testing section **Testing** This is an arguable change. Im open to reconsidering. - Refactoring I've rewritten the whole test suite to have individual test per operator rather than having one master test with many else/if conditions. In my opinion, the tests became much more readable and easier to maintain. - Coverage and correctness Each _foreach API has its dedicated test where all possible dtypes/ combination of dtypes are being tested and compared to their counter parts in torch. [ghstack-poisoned]
Differential Revision: [D25074764](https://our.internmc.facebook.com/intern/diff/D25074764) Coming next for this stack: 1. Docs PR. Add more code documentation 2. Swap optimizers from torch.optim and torch.optim.multi_tensor -------------------------------- **Motivation** _foreach APIs should produce the exact same result as if they were implemented via for-loop implementation but have significant performance gains in particular scenarios. This PR achieves this. **What was done** - Stopped throwing an error in case when previous API restrictions were not met. Instead, execution will go via slow path if possible. - Updated code logic to acknowledge division op. This is a temp fix until MTA wont support type promotion. In case of integer input and division, we should go via slow path as MTA doesn't support return of a different dtype. - Significantly extended type support for binary/unary/pointwise ops. with this PR, _foreach APIs support exactly same list of types as their analogs in torch. - Completely rewrote test/test_foreach.py, see testing section **Testing** This is an arguable change. Im open to reconsidering. - Refactoring I've rewritten the whole test suite to have individual test per operator rather than having one master test with many else/if conditions. In my opinion, the tests became much more readable and easier to maintain. - Coverage and correctness Each _foreach API has its dedicated test where all possible dtypes/ combination of dtypes are being tested and compared to their counter parts in torch. [ghstack-poisoned]
Differential Revision: [D25074764](https://our.internmc.facebook.com/intern/diff/D25074764) Coming next for this stack: 1. Docs PR. Add more code documentation 2. Swap optimizers from torch.optim and torch.optim.multi_tensor -------------------------------- **Motivation** _foreach APIs should produce the exact same result as if they were implemented via for-loop implementation but have significant performance gains in particular scenarios. This PR achieves this. **What was done** - Stopped throwing an error in case when previous API restrictions were not met. Instead, execution will go via slow path if possible. - Updated code logic to acknowledge division op. This is a temp fix until MTA wont support type promotion. In case of integer input and division, we should go via slow path as MTA doesn't support return of a different dtype. - Significantly extended type support for binary/unary/pointwise ops. with this PR, _foreach APIs support exactly same list of types as their analogs in torch. - Completely rewrote test/test_foreach.py, see testing section **Testing** This is an arguable change. Im open to reconsidering. - Refactoring I've rewritten the whole test suite to have individual test per operator rather than having one master test with many else/if conditions. In my opinion, the tests became much more readable and easier to maintain. - Coverage and correctness Each _foreach API has its dedicated test where all possible dtypes/ combination of dtypes are being tested and compared to their counter parts in torch. [ghstack-poisoned]
Differential Revision: [D25074764](https://our.internmc.facebook.com/intern/diff/D25074764) Coming next for this stack: 1. Docs PR. Add more code documentation 2. Swap optimizers from torch.optim and torch.optim.multi_tensor -------------------------------- **Motivation** _foreach APIs should produce the exact same result as if they were implemented via for-loop implementation but have significant performance gains in particular scenarios. This PR achieves this. **What was done** - Stopped throwing an error in case when previous API restrictions were not met. Instead, execution will go via slow path if possible. - Updated code logic to acknowledge division op. This is a temp fix until MTA wont support type promotion. In case of integer input and division, we should go via slow path as MTA doesn't support return of a different dtype. - Significantly extended type support for binary/unary/pointwise ops. with this PR, _foreach APIs support exactly same list of types as their analogs in torch. - Completely rewrote test/test_foreach.py, see testing section **Testing** This is an arguable change. Im open to reconsidering. - Refactoring I've rewritten the whole test suite to have individual test per operator rather than having one master test with many else/if conditions. In my opinion, the tests became much more readable and easier to maintain. - Coverage and correctness Each _foreach API has its dedicated test where all possible dtypes/ combination of dtypes are being tested and compared to their counter parts in torch. [ghstack-poisoned]
This was referenced Dec 11, 2020
Differential Revision: [D25074764](https://our.internmc.facebook.com/intern/diff/D25074764) Coming next for this stack: 1. Docs PR. Add more code documentation 2. Swap optimizers from torch.optim and torch.optim.multi_tensor -------------------------------- **Motivation** _foreach APIs should produce the exact same result as if they were implemented via for-loop implementation but have significant performance gains in particular scenarios. This PR achieves this. **What was done** - Stopped throwing an error in case when previous API restrictions were not met. Instead, execution will go via slow path if possible. - Updated code logic to acknowledge division op. This is a temp fix until MTA wont support type promotion. In case of integer input and division, we should go via slow path as MTA doesn't support return of a different dtype. - Significantly extended type support for binary/unary/pointwise ops. with this PR, _foreach APIs support exactly same list of types as their analogs in torch. - Completely rewrote test/test_foreach.py, see testing section **Testing** This is an arguable change. Im open to reconsidering. - Refactoring I've rewritten the whole test suite to have individual test per operator rather than having one master test with many else/if conditions. In my opinion, the tests became much more readable and easier to maintain. - Coverage and correctness Each _foreach API has its dedicated test where all possible dtypes/ combination of dtypes are being tested and compared to their counter parts in torch. [ghstack-poisoned]
Differential Revision: [D25074764](https://our.internmc.facebook.com/intern/diff/D25074764) Coming next for this stack: 1. Docs PR. Add more code documentation 2. Swap optimizers from torch.optim and torch.optim.multi_tensor -------------------------------- **Motivation** _foreach APIs should produce the exact same result as if they were implemented via for-loop implementation but have significant performance gains in particular scenarios. This PR achieves this. **What was done** - Stopped throwing an error in case when previous API restrictions were not met. Instead, execution will go via slow path if possible. - Updated code logic to acknowledge division op. This is a temp fix until MTA wont support type promotion. In case of integer input and division, we should go via slow path as MTA doesn't support return of a different dtype. - Significantly extended type support for binary/unary/pointwise ops. with this PR, _foreach APIs support exactly same list of types as their analogs in torch. - Completely rewrote test/test_foreach.py, see testing section **Testing** This is an arguable change. Im open to reconsidering. - Refactoring I've rewritten the whole test suite to have individual test per operator rather than having one master test with many else/if conditions. In my opinion, the tests became much more readable and easier to maintain. - Coverage and correctness Each _foreach API has its dedicated test where all possible dtypes/ combination of dtypes are being tested and compared to their counter parts in torch. [ghstack-poisoned]
Differential Revision: [D25074764](https://our.internmc.facebook.com/intern/diff/D25074764) Coming next for this stack: 1. Docs PR. Add more code documentation 2. Swap optimizers from torch.optim and torch.optim.multi_tensor -------------------------------- **Motivation** _foreach APIs should produce the exact same result as if they were implemented via for-loop implementation but have significant performance gains in particular scenarios. This PR achieves this. **What was done** - Stopped throwing an error in case when previous API restrictions were not met. Instead, execution will go via slow path if possible. - Updated code logic to acknowledge division op. This is a temp fix until MTA wont support type promotion. In case of integer input and division, we should go via slow path as MTA doesn't support return of a different dtype. - Significantly extended type support for binary/unary/pointwise ops. with this PR, _foreach APIs support exactly same list of types as their analogs in torch. - Completely rewrote test/test_foreach.py, see testing section **Testing** This is an arguable change. Im open to reconsidering. - Refactoring I've rewritten the whole test suite to have individual test per operator rather than having one master test with many else/if conditions. In my opinion, the tests became much more readable and easier to maintain. - Coverage and correctness Each _foreach API has its dedicated test where all possible dtypes/ combination of dtypes are being tested and compared to their counter parts in torch. [ghstack-poisoned]
Differential Revision: [D25074764](https://our.internmc.facebook.com/intern/diff/D25074764) Coming next for this stack: 1. Docs PR. Add more code documentation 2. Swap optimizers from torch.optim and torch.optim.multi_tensor -------------------------------- **Motivation** _foreach APIs should produce the exact same result as if they were implemented via for-loop implementation but have significant performance gains in particular scenarios. This PR achieves this. **What was done** - Stopped throwing an error in case when previous API restrictions were not met. Instead, execution will go via slow path if possible. - Updated code logic to acknowledge division op. This is a temp fix until MTA wont support type promotion. In case of integer input and division, we should go via slow path as MTA doesn't support return of a different dtype. - Significantly extended type support for binary/unary/pointwise ops. with this PR, _foreach APIs support exactly same list of types as their analogs in torch. - Completely rewrote test/test_foreach.py, see testing section **Testing** This is an arguable change. Im open to reconsidering. - Refactoring I've rewritten the whole test suite to have individual test per operator rather than having one master test with many else/if conditions. In my opinion, the tests became much more readable and easier to maintain. - Coverage and correctness Each _foreach API has its dedicated test where all possible dtypes/ combination of dtypes are being tested and compared to their counter parts in torch. [ghstack-poisoned]
Differential Revision: [D25074764](https://our.internmc.facebook.com/intern/diff/D25074764) Coming next for this stack: 1. Docs PR. Add more code documentation 2. Swap optimizers from torch.optim and torch.optim.multi_tensor -------------------------------- **Motivation** _foreach APIs should produce the exact same result as if they were implemented via for-loop implementation but have significant performance gains in particular scenarios. This PR achieves this. **What was done** - Stopped throwing an error in case when previous API restrictions were not met. Instead, execution will go via slow path if possible. - Updated code logic to acknowledge division op. This is a temp fix until MTA wont support type promotion. In case of integer input and division, we should go via slow path as MTA doesn't support return of a different dtype. - Significantly extended type support for binary/unary/pointwise ops. with this PR, _foreach APIs support exactly same list of types as their analogs in torch. - Completely rewrote test/test_foreach.py, see testing section **Testing** This is an arguable change. Im open to reconsidering. - Refactoring I've rewritten the whole test suite to have individual test per operator rather than having one master test with many else/if conditions. In my opinion, the tests became much more readable and easier to maintain. - Coverage and correctness Each _foreach API has its dedicated test where all possible dtypes/ combination of dtypes are being tested and compared to their counter parts in torch. [ghstack-poisoned]
Differential Revision: [D25074764](https://our.internmc.facebook.com/intern/diff/D25074764) Coming next for this stack: 1. Docs PR. Add more code documentation 2. Swap optimizers from torch.optim and torch.optim.multi_tensor -------------------------------- **Motivation** _foreach APIs should produce the exact same result as if they were implemented via for-loop implementation but have significant performance gains in particular scenarios. This PR achieves this. **What was done** - Stopped throwing an error in case when previous API restrictions were not met. Instead, execution will go via slow path if possible. - Updated code logic to acknowledge division op. This is a temp fix until MTA wont support type promotion. In case of integer input and division, we should go via slow path as MTA doesn't support return of a different dtype. - Significantly extended type support for binary/unary/pointwise ops. with this PR, _foreach APIs support exactly same list of types as their analogs in torch. - Completely rewrote test/test_foreach.py, see testing section **Testing** This is an arguable change. Im open to reconsidering. - Refactoring I've rewritten the whole test suite to have individual test per operator rather than having one master test with many else/if conditions. In my opinion, the tests became much more readable and easier to maintain. - Coverage and correctness Each _foreach API has its dedicated test where all possible dtypes/ combination of dtypes are being tested and compared to their counter parts in torch. [ghstack-poisoned]
Differential Revision: [D25074764](https://our.internmc.facebook.com/intern/diff/D25074764) Coming next for this stack: 1. Docs PR. Add more code documentation 2. Swap optimizers from torch.optim and torch.optim.multi_tensor -------------------------------- **Motivation** _foreach APIs should produce the exact same result as if they were implemented via for-loop implementation but have significant performance gains in particular scenarios. This PR achieves this. **What was done** - Stopped throwing an error in case when previous API restrictions were not met. Instead, execution will go via slow path if possible. - Updated code logic to acknowledge division op. This is a temp fix until MTA wont support type promotion. In case of integer input and division, we should go via slow path as MTA doesn't support return of a different dtype. - Significantly extended type support for binary/unary/pointwise ops. with this PR, _foreach APIs support exactly same list of types as their analogs in torch. - Completely rewrote test/test_foreach.py, see testing section **Testing** This is an arguable change. Im open to reconsidering. - Refactoring I've rewritten the whole test suite to have individual test per operator rather than having one master test with many else/if conditions. In my opinion, the tests became much more readable and easier to maintain. - Coverage and correctness Each _foreach API has its dedicated test where all possible dtypes/ combination of dtypes are being tested and compared to their counter parts in torch. [ghstack-poisoned]
Differential Revision: [D25074764](https://our.internmc.facebook.com/intern/diff/D25074764) Coming next for this stack: 1. Docs PR. Add more code documentation 2. Swap optimizers from torch.optim and torch.optim.multi_tensor -------------------------------- **Motivation** _foreach APIs should produce the exact same result as if they were implemented via for-loop implementation but have significant performance gains in particular scenarios. This PR achieves this. **What was done** - Stopped throwing an error in case when previous API restrictions were not met. Instead, execution will go via slow path if possible. - Updated code logic to acknowledge division op. This is a temp fix until MTA wont support type promotion. In case of integer input and division, we should go via slow path as MTA doesn't support return of a different dtype. - Significantly extended type support for binary/unary/pointwise ops. with this PR, _foreach APIs support exactly same list of types as their analogs in torch. - Completely rewrote test/test_foreach.py, see testing section **Testing** This is an arguable change. Im open to reconsidering. - Refactoring I've rewritten the whole test suite to have individual test per operator rather than having one master test with many else/if conditions. In my opinion, the tests became much more readable and easier to maintain. - Coverage and correctness Each _foreach API has its dedicated test where all possible dtypes/ combination of dtypes are being tested and compared to their counter parts in torch. [ghstack-poisoned]
Differential Revision: [D25074764](https://our.internmc.facebook.com/intern/diff/D25074764) Coming next for this stack: 1. Docs PR. Add more code documentation 2. Swap optimizers from torch.optim and torch.optim.multi_tensor -------------------------------- **Motivation** _foreach APIs should produce the exact same result as if they were implemented via for-loop implementation but have significant performance gains in particular scenarios. This PR achieves this. **What was done** - Stopped throwing an error in case when previous API restrictions were not met. Instead, execution will go via slow path if possible. - Updated code logic to acknowledge division op. This is a temp fix until MTA wont support type promotion. In case of integer input and division, we should go via slow path as MTA doesn't support return of a different dtype. - Significantly extended type support for binary/unary/pointwise ops. with this PR, _foreach APIs support exactly same list of types as their analogs in torch. - Completely rewrote test/test_foreach.py, see testing section **Testing** This is an arguable change. Im open to reconsidering. - Refactoring I've rewritten the whole test suite to have individual test per operator rather than having one master test with many else/if conditions. In my opinion, the tests became much more readable and easier to maintain. - Coverage and correctness Each _foreach API has its dedicated test where all possible dtypes/ combination of dtypes are being tested and compared to their counter parts in torch. [ghstack-poisoned]
Differential Revision: [D25074764](https://our.internmc.facebook.com/intern/diff/D25074764) Coming next for this stack: 1. Docs PR. Add more code documentation 2. Swap optimizers from torch.optim and torch.optim.multi_tensor -------------------------------- **Motivation** _foreach APIs should produce the exact same result as if they were implemented via for-loop implementation but have significant performance gains in particular scenarios. This PR achieves this. **What was done** - Stopped throwing an error in case when previous API restrictions were not met. Instead, execution will go via slow path if possible. - Updated code logic to acknowledge division op. This is a temp fix until MTA wont support type promotion. In case of integer input and division, we should go via slow path as MTA doesn't support return of a different dtype. - Significantly extended type support for binary/unary/pointwise ops. with this PR, _foreach APIs support exactly same list of types as their analogs in torch. - Completely rewrote test/test_foreach.py, see testing section **Testing** This is an arguable change. Im open to reconsidering. - Refactoring I've rewritten the whole test suite to have individual test per operator rather than having one master test with many else/if conditions. In my opinion, the tests became much more readable and easier to maintain. - Coverage and correctness Each _foreach API has its dedicated test where all possible dtypes/ combination of dtypes are being tested and compared to their counter parts in torch. [ghstack-poisoned]
Differential Revision: [D25074764](https://our.internmc.facebook.com/intern/diff/D25074764) Coming next for this stack: 1. Docs PR. Add more code documentation 2. Swap optimizers from torch.optim and torch.optim.multi_tensor -------------------------------- **Motivation** _foreach APIs should produce the exact same result as if they were implemented via for-loop implementation but have significant performance gains in particular scenarios. This PR achieves this. **What was done** - Stopped throwing an error in case when previous API restrictions were not met. Instead, execution will go via slow path if possible. - Updated code logic to acknowledge division op. This is a temp fix until MTA wont support type promotion. In case of integer input and division, we should go via slow path as MTA doesn't support return of a different dtype. - Significantly extended type support for binary/unary/pointwise ops. with this PR, _foreach APIs support exactly same list of types as their analogs in torch. - Completely rewrote test/test_foreach.py, see testing section **Testing** This is an arguable change. Im open to reconsidering. - Refactoring I've rewritten the whole test suite to have individual test per operator rather than having one master test with many else/if conditions. In my opinion, the tests became much more readable and easier to maintain. - Coverage and correctness Each _foreach API has its dedicated test where all possible dtypes/ combination of dtypes are being tested and compared to their counter parts in torch. [ghstack-poisoned]
Differential Revision: [D25074764](https://our.internmc.facebook.com/intern/diff/D25074764) Coming next for this stack: 1. Docs PR. Add more code documentation 2. Swap optimizers from torch.optim and torch.optim.multi_tensor -------------------------------- **Motivation** _foreach APIs should produce the exact same result as if they were implemented via for-loop implementation but have significant performance gains in particular scenarios. This PR achieves this. **What was done** - Stopped throwing an error in case when previous API restrictions were not met. Instead, execution will go via slow path if possible. - Updated code logic to acknowledge division op. This is a temp fix until MTA wont support type promotion. In case of integer input and division, we should go via slow path as MTA doesn't support return of a different dtype. - Significantly extended type support for binary/unary/pointwise ops. with this PR, _foreach APIs support exactly same list of types as their analogs in torch. - Completely rewrote test/test_foreach.py, see testing section **Testing** This is an arguable change. Im open to reconsidering. - Refactoring I've rewritten the whole test suite to have individual test per operator rather than having one master test with many else/if conditions. In my opinion, the tests became much more readable and easier to maintain. - Coverage and correctness Each _foreach API has its dedicated test where all possible dtypes/ combination of dtypes are being tested and compared to their counter parts in torch. [ghstack-poisoned]
Differential Revision: [D25074764](https://our.internmc.facebook.com/intern/diff/D25074764) Coming next for this stack: 1. Docs PR. Add more code documentation 2. Swap optimizers from torch.optim and torch.optim.multi_tensor -------------------------------- **Motivation** _foreach APIs should produce the exact same result as if they were implemented via for-loop implementation but have significant performance gains in particular scenarios. This PR achieves this. **What was done** - Stopped throwing an error in case when previous API restrictions were not met. Instead, execution will go via slow path if possible. - Updated code logic to acknowledge division op. This is a temp fix until MTA wont support type promotion. In case of integer input and division, we should go via slow path as MTA doesn't support return of a different dtype. - Significantly extended type support for binary/unary/pointwise ops. with this PR, _foreach APIs support exactly same list of types as their analogs in torch. - Completely rewrote test/test_foreach.py, see testing section **Testing** This is an arguable change. Im open to reconsidering. - Refactoring I've rewritten the whole test suite to have individual test per operator rather than having one master test with many else/if conditions. In my opinion, the tests became much more readable and easier to maintain. - Coverage and correctness Each _foreach API has its dedicated test where all possible dtypes/ combination of dtypes are being tested and compared to their counter parts in torch. [ghstack-poisoned]
Differential Revision: [D25074764](https://our.internmc.facebook.com/intern/diff/D25074764) Coming next for this stack: 1. Docs PR. Add more code documentation 2. Swap optimizers from torch.optim and torch.optim.multi_tensor -------------------------------- **Motivation** _foreach APIs should produce the exact same result as if they were implemented via for-loop implementation but have significant performance gains in particular scenarios. This PR achieves this. **What was done** - Stopped throwing an error in case when previous API restrictions were not met. Instead, execution will go via slow path if possible. - Updated code logic to acknowledge division op. This is a temp fix until MTA wont support type promotion. In case of integer input and division, we should go via slow path as MTA doesn't support return of a different dtype. - Significantly extended type support for binary/unary/pointwise ops. with this PR, _foreach APIs support exactly same list of types as their analogs in torch. - Completely rewrote test/test_foreach.py, see testing section **Testing** This is an arguable change. Im open to reconsidering. - Refactoring I've rewritten the whole test suite to have individual test per operator rather than having one master test with many else/if conditions. In my opinion, the tests became much more readable and easier to maintain. - Coverage and correctness Each _foreach API has its dedicated test where all possible dtypes/ combination of dtypes are being tested and compared to their counter parts in torch. [ghstack-poisoned]
Differential Revision: [D25074764](https://our.internmc.facebook.com/intern/diff/D25074764) Coming next for this stack: 1. Docs PR. Add more code documentation 2. Swap optimizers from torch.optim and torch.optim.multi_tensor -------------------------------- **Motivation** _foreach APIs should produce the exact same result as if they were implemented via for-loop implementation but have significant performance gains in particular scenarios. This PR achieves this. **What was done** - Stopped throwing an error in case when previous API restrictions were not met. Instead, execution will go via slow path if possible. - Updated code logic to acknowledge division op. This is a temp fix until MTA wont support type promotion. In case of integer input and division, we should go via slow path as MTA doesn't support return of a different dtype. - Significantly extended type support for binary/unary/pointwise ops. with this PR, _foreach APIs support exactly same list of types as their analogs in torch. - Completely rewrote test/test_foreach.py, see testing section **Testing** This is an arguable change. Im open to reconsidering. - Refactoring I've rewritten the whole test suite to have individual test per operator rather than having one master test with many else/if conditions. In my opinion, the tests became much more readable and easier to maintain. - Coverage and correctness Each _foreach API has its dedicated test where all possible dtypes/ combination of dtypes are being tested and compared to their counter parts in torch. [ghstack-poisoned]
Differential Revision: [D25074764](https://our.internmc.facebook.com/intern/diff/D25074764) Coming next for this stack: 1. Docs PR. Add more code documentation 2. Swap optimizers from torch.optim and torch.optim.multi_tensor -------------------------------- **Motivation** _foreach APIs should produce the exact same result as if they were implemented via for-loop implementation but have significant performance gains in particular scenarios. This PR achieves this. **What was done** - Stopped throwing an error in case when previous API restrictions were not met. Instead, execution will go via slow path if possible. - Updated code logic to acknowledge division op. This is a temp fix until MTA wont support type promotion. In case of integer input and division, we should go via slow path as MTA doesn't support return of a different dtype. - Significantly extended type support for binary/unary/pointwise ops. with this PR, _foreach APIs support exactly same list of types as their analogs in torch. - Completely rewrote test/test_foreach.py, see testing section **Testing** This is an arguable change. Im open to reconsidering. - Refactoring I've rewritten the whole test suite to have individual test per operator rather than having one master test with many else/if conditions. In my opinion, the tests became much more readable and easier to maintain. - Coverage and correctness Each _foreach API has its dedicated test where all possible dtypes/ combination of dtypes are being tested and compared to their counter parts in torch. [ghstack-poisoned]
Differential Revision: [D25074764](https://our.internmc.facebook.com/intern/diff/D25074764) Coming next for this stack: 1. Docs PR. Add more code documentation 2. Swap optimizers from torch.optim and torch.optim.multi_tensor -------------------------------- **Motivation** _foreach APIs should produce the exact same result as if they were implemented via for-loop implementation but have significant performance gains in particular scenarios. This PR achieves this. **What was done** - Stopped throwing an error in case when previous API restrictions were not met. Instead, execution will go via slow path if possible. - Updated code logic to acknowledge division op. This is a temp fix until MTA wont support type promotion. In case of integer input and division, we should go via slow path as MTA doesn't support return of a different dtype. - Significantly extended type support for binary/unary/pointwise ops. with this PR, _foreach APIs support exactly same list of types as their analogs in torch. - Completely rewrote test/test_foreach.py, see testing section **Testing** This is an arguable change. Im open to reconsidering. - Refactoring I've rewritten the whole test suite to have individual test per operator rather than having one master test with many else/if conditions. In my opinion, the tests became much more readable and easier to maintain. - Coverage and correctness Each _foreach API has its dedicated test where all possible dtypes/ combination of dtypes are being tested and compared to their counter parts in torch. [ghstack-poisoned]
Contributor
|
Hi @izdeby! Thank you for your pull request. We require contributors to sign our Contributor License Agreement, and yours needs attention. You currently have a record in our system, but the CLA is no longer valid, and will need to be resubmitted. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at cla@fb.com. Thanks! |
crcrpar
added a commit
to crcrpar/pytorch
that referenced
this pull request
Apr 22, 2021
crcrpar
added a commit
to crcrpar/pytorch
that referenced
this pull request
Apr 22, 2021
crcrpar
added a commit
to crcrpar/pytorch
that referenced
this pull request
Apr 27, 2021
crcrpar
added a commit
to crcrpar/pytorch
that referenced
this pull request
May 2, 2021
Ref: pytorch#48224 apply int/bool check to all tensors
facebook-github-bot
pushed a commit
that referenced
this pull request
May 25, 2021
Summary: This is based on #48224. To make `foreach` more flexible, this PR pushes unsupported cases to slow path. Also, this adds some tests to verify that - `foreach` functions work with tensors of different dtypes and/or memory layouts in 7bd4b2c - `foreach` functions work with tensors on different devices in a list, but are on the same device if the indices are the same: def4b9b Future plans: 1. Improve the coverage of unittests using `ops` decorator & updating `foreach_unary_op_db` and creating `foreach_(binary|pointwise|minmax)_db`. 2. Support broadcasting in slow path. Ref: #52448 3. Support type promotion in fast path. Ref #52449 CC: ngimel mcarilli ptrblck Pull Request resolved: #56993 Reviewed By: zou3519 Differential Revision: D28630580 Pulled By: ngimel fbshipit-source-id: e26ee74a39a591025e18c1ead48948cb7ec53c19
deniskokarev
pushed a commit
to deniskokarev/pytorch
that referenced
this pull request
Jun 9, 2021
Summary: This is based on pytorch#48224. To make `foreach` more flexible, this PR pushes unsupported cases to slow path. Also, this adds some tests to verify that - `foreach` functions work with tensors of different dtypes and/or memory layouts in pytorch@7bd4b2c - `foreach` functions work with tensors on different devices in a list, but are on the same device if the indices are the same: pytorch@def4b9b Future plans: 1. Improve the coverage of unittests using `ops` decorator & updating `foreach_unary_op_db` and creating `foreach_(binary|pointwise|minmax)_db`. 2. Support broadcasting in slow path. Ref: pytorch#52448 3. Support type promotion in fast path. Ref pytorch#52449 CC: ngimel mcarilli ptrblck Pull Request resolved: pytorch#56993 Reviewed By: zou3519 Differential Revision: D28630580 Pulled By: ngimel fbshipit-source-id: e26ee74a39a591025e18c1ead48948cb7ec53c19
Collaborator
|
Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack from ghstack:
Differential Revision: D25074764
Coming next for this stack:
Motivation
_foreach APIs should produce the exact same result as if they were implemented via for-loop implementation but have significant performance gains in particular scenarios. This PR achieves this.
What was done
Testing
This is an arguable change. Im open to reconsidering.
Refactoring
I've rewritten the whole test suite to have individual test per operator rather than having one master test with many else/if conditions. In my opinion, the tests became much more readable and easier to maintain.
Coverage and correctness
Each _foreach API has its dedicated test where all possible dtypes/ combination of dtypes are being tested and compared to their counter parts in torch.