Reduce memory requirement for test_argminmax_large_axis#40036
Closed
peterbell10 wants to merge 1 commit intopytorch:masterfrom
Closed
Reduce memory requirement for test_argminmax_large_axis#40036peterbell10 wants to merge 1 commit intopytorch:masterfrom
peterbell10 wants to merge 1 commit intopytorch:masterfrom
Conversation
ezyang
approved these changes
Jun 16, 2020
Contributor
facebook-github-bot
left a comment
There was a problem hiding this comment.
@ezyang is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
Contributor
xwang233
pushed a commit
to xwang233/pytorch
that referenced
this pull request
Jun 20, 2020
Summary: Closes pytorchgh-39060 The `TensorIterator` splitting is based on `can_use_32bit_indexing` which assumes 32-bit signed ints, so we can get away with just 2**31 as the axis length. Also tested on an old commit that I can reproduce the test failure on just a 1d tensor, overall quartering the memory requirement for the test. https://github.com/pytorch/pytorch/blob/4c7d81f8479bce320cc11d1eb3adaf8ab0b90099/aten/src/ATen/native/TensorIterator.cpp#L879 For reference, the test was first added in pytorchgh-33310. Pull Request resolved: pytorch#40036 Differential Revision: D22068690 Pulled By: ezyang fbshipit-source-id: 83199fd31647d1ef106b08f471c0e9517d3516e3
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes gh-39060
The
TensorIteratorsplitting is based oncan_use_32bit_indexingwhich assumes 32-bit signed ints, so we can get away with just 2**31 as the axis length. Also tested on an old commit that I can reproduce the test failure on just a 1d tensor, overall quartering the memory requirement for the test.pytorch/aten/src/ATen/native/TensorIterator.cpp
Line 879 in 4c7d81f
For reference, the test was first added in gh-33310.