Add memory format support to cpu and cuda operators#27223
Closed
VitalyFedyunin wants to merge 11 commits intogh/VitalyFedyunin/3/basefrom
Closed
Add memory format support to cpu and cuda operators#27223VitalyFedyunin wants to merge 11 commits intogh/VitalyFedyunin/3/basefrom
cpu and cuda operators#27223VitalyFedyunin wants to merge 11 commits intogh/VitalyFedyunin/3/basefrom
Conversation
[ghstack-poisoned]
This was referenced Oct 2, 2019
[ghstack-poisoned]
[ghstack-poisoned]
Adds memory_format keyword argument (positional for cpp). 'Preserve' behavior now follows next rules: 1) If tensor is non-overlapping and dense - output tensor will have the same strides as input tensor. 2) If not (1) and tensor is stored in the channels last format, output tensor going to have channels last format. 3) Output tensor is going to be contiguous in all other cases. --- Dense tensor is the tensor that store values in a contiguous block of memory. Non-overlapping tensor is the tensor in which elements occupy individual non-repetitive memory. [ghstack-poisoned]
Adds memory_format keyword argument (positional for cpp). 'Preserve' behavior now follows next rules: 1) If tensor is non-overlapping and dense - output tensor will have the same strides as input tensor. 2) If not (1) and tensor is stored in the channels last format, output tensor going to have channels last format. 3) Output tensor is going to be contiguous in all other cases. --- Dense tensor is the tensor that store values in a contiguous block of memory. Non-overlapping tensor is the tensor in which elements occupy individual non-repetitive memory. [ghstack-poisoned]
This was referenced Oct 8, 2019
Adds memory_format keyword argument (positional for cpp). 'Preserve' behavior now follows next rules: 1) If tensor is non-overlapping and dense - output tensor will have the same strides as input tensor. 2) If not (1) and tensor is stored in the channels last format, output tensor going to have channels last format. 3) Output tensor is going to be contiguous in all other cases. --- Dense tensor is the tensor that store values in a contiguous block of memory. Non-overlapping tensor is the tensor in which elements occupy individual non-repetitive memory. [ghstack-poisoned]
Closed
Adds memory_format keyword argument (positional for cpp). 'Preserve' behavior now follows next rules: 1) If tensor is non-overlapping and dense - output tensor will have the same strides as input tensor. 2) If not (1) and tensor is stored in the channels last format, output tensor going to have channels last format. 3) Output tensor is going to be contiguous in all other cases. --- Dense tensor is the tensor that store values in a contiguous block of memory. Non-overlapping tensor is the tensor in which elements occupy individual non-repetitive memory. [ghstack-poisoned]
This was referenced Oct 14, 2019
…operators" Adds memory_format keyword argument (positional for cpp). 'Preserve' behavior now follows next rules: 1) If tensor is non-overlapping and dense - output tensor will have the same strides as input tensor. 2) If not (1) and tensor is stored in the channels last format, output tensor going to have channels last format. 3) Output tensor is going to be contiguous in all other cases. --- Dense tensor is the tensor that store values in a contiguous block of memory. Non-overlapping tensor is the tensor in which elements occupy individual non-repetitive memory. [ghstack-poisoned]
…da` operators" Adds memory_format keyword argument (positional for cpp). 'Preserve' behavior now follows next rules: 1) If tensor is non-overlapping and dense - output tensor will have the same strides as input tensor. 2) If not (1) and tensor is stored in the channels last format, output tensor going to have channels last format. 3) Output tensor is going to be contiguous in all other cases. --- Dense tensor is the tensor that store values in a contiguous block of memory. Non-overlapping tensor is the tensor in which elements occupy individual non-repetitive memory. [ghstack-poisoned]
Contributor
Author
|
@ailzhang breaks XLA |
gchanan
approved these changes
Oct 15, 2019
Contributor
gchanan
left a comment
There was a problem hiding this comment.
lgtm.
It would be nice if there were JIT tests for all of this. This area of the code is janky enough that I could believe: 1) JIT doesn't support it, 2) JIT tries to support it, but it's not consistent with the eager implementation 3) JIT does support it, but the support is fragile and we could break it without intending to.
…uda` operators" Adds memory_format keyword argument (positional for cpp). 'Preserve' behavior now follows next rules: 1) If tensor is non-overlapping and dense - output tensor will have the same strides as input tensor. 2) If not (1) and tensor is stored in the channels last format, output tensor going to have channels last format. 3) Output tensor is going to be contiguous in all other cases. --- Dense tensor is the tensor that store values in a contiguous block of memory. Non-overlapping tensor is the tensor in which elements occupy individual non-repetitive memory. [ghstack-poisoned]
This was referenced Oct 16, 2019
Contributor
Author
|
Good point about JIT support:
|
Contributor
Author
…operators" Adds memory_format keyword argument (positional for cpp). 'Preserve' behavior now follows next rules: 1) If tensor is non-overlapping and dense - output tensor will have the same strides as input tensor. 2) If not (1) and tensor is stored in the channels last format, output tensor going to have channels last format. 3) Output tensor is going to be contiguous in all other cases. --- Dense tensor is the tensor that store values in a contiguous block of memory. Non-overlapping tensor is the tensor in which elements occupy individual non-repetitive memory. Differential Revision: [D17980128](https://our.internmc.facebook.com/intern/diff/D17980128) [ghstack-poisoned]
xxtEchjovs44
pushed a commit
to xxtEchjovs44/pytorch
that referenced
this pull request
Jan 29, 2020
ghstack-source-id: 6d93f33 Pull Request resolved: pytorch/pytorch#27223
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack from ghstack:
operator==of TensorOptions as confusing one #28076 Killoperator==of TensorOptions as confusing oneresize_as_operator #27979 Add memory format support toresize_as_operatorrandn_likeoperator #27890 Add memory format support torandn_likeoperatorrandint_likeoperator #27889 Add memory format support torandint_likeoperatorzeros_likeoperator #27562 Add memory format support tozeros_likeoperatorrand_likeoperator #27561 Add memory format support torand_likeoperatorones_likeoperator #27270 Add memory format support toones_likeoperatorfull_likeoperator #27262 Add memory format support tofull_likeoperatorempty_likeoperator #27244 Add memory format support toempty_likeoperatorbyte,char,double,bool,half,int,long,short,float,bfloat16#27228 Add memory format support to typecasting shortcutsbyte,char,double,bool,half,int,long,short,float,bfloat16cpuandcudaoperators #27223 Add memory format support tocpuandcudaoperatorsAdds memory_format keyword argument (positional for cpp).
'Preserve' behavior now follows next rules:
Dense tensor is the tensor that store values in a contiguous block of memory.
Non-overlapping tensor is the tensor in which elements occupy individual non-repetitive memory.
Differential Revision: D17980128