Skip to content

Remove methods that start with an underscore from at::Tensor#11152

Closed
goldsborough wants to merge 12 commits intopytorch:masterfrom
goldsborough:remove-tensor-underscore
Closed

Remove methods that start with an underscore from at::Tensor#11152
goldsborough wants to merge 12 commits intopytorch:masterfrom
goldsborough:remove-tensor-underscore

Conversation

@goldsborough
Copy link
Contributor

This PR cleans up the at::Tensor class by removing all methods that start with an underscore in favor of functions in the at:: namespace. This greatly cleans up the Tensor class and makes it clearer what is the public and non-public API.

For this I changed native_functions.yaml and Declarations.cwrap to make all underscore methods variant: function (or add such a statement to begin with), and then fixed all code locations using the underscore methods.

@ezyang @colesbury @gchanan

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

@goldsborough goldsborough force-pushed the remove-tensor-underscore branch from eb24083 to aa3ff2d Compare August 31, 2018 21:42
@goldsborough
Copy link
Contributor Author

Ok so I:

  1. Renamed {_values,_indices,_nnz} to {values,indices,nnz} (no underscore) in C++ and expose those in Python
  2. Created wrappers for {_values,_indices,_nnz} on torch/tensor.py to preserve BC in Python
  3. Rewrote C++ code to use .values() instead of .indices()

I think this should make everyone happy @zou3519 @ssnl @weiyangfb?

@goldsborough goldsborough force-pushed the remove-tensor-underscore branch from aa3ff2d to d1c49e4 Compare August 31, 2018 21:45
@ezyang
Copy link
Contributor

ezyang commented Sep 2, 2018

Renamed {_values,_indices,_nnz} to {values,indices,nnz} (no underscore) in C++ and expose those in Python

No, please don't do this. I'll quote the current sparse documentation:

Second, some operators will produce different values depending on whether or not they are coalesced or not (e.g., torch.sparse.FloatTensor._values() and torch.sparse.FloatTensor._indices(), as well as torch.Tensor._sparse_mask()). These operators are prefixed by an underscore to indicate that they reveal internal implementation details and should be used with care, since code that works with coalesced sparse tensors may not work with uncoalesced sparse tensors; generally speaking, it is safest to explicitly coalesce before working with these operators.

https://pytorch.org/docs/stable/sparse.html

@ezyang
Copy link
Contributor

ezyang commented Sep 2, 2018

@goldsborough Is your goal to unconditionally remove all underscore-prefixed methods from Tensor, or are you OK with exceptions? _indices and _values is something of a live wire in the current sparse tensor API, and can't really be adequately resolved without some careful documentation and addition of new methods. I actually do think that something like _indices or raw_indices should be in the steady state sparse API, but they should be clear that they have to be used with care.

@goldsborough
Copy link
Contributor Author

@ezyang I'm ok with exceptions, I just thought we might as well fix the API since public underscore fields seem wrong either way. But maybe that's a discussion for another PR. I'll leave them be in this one :)

@goldsborough goldsborough force-pushed the remove-tensor-underscore branch 3 times, most recently from af9b4e9 to f5ca400 Compare September 5, 2018 00:18

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

Copy link
Contributor

@ezyang ezyang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just realized this is going to intersect with #11247 ; both diffs de-method-ize some functions where methods don't make sense (but yours is more aggressive) (I didn't realize this because the latter diff is doing something else: changing interpretation of missing variants in native). Do you want to fold that patch in, or should I just rebase on top? Or should I land mine first?

@ezyang
Copy link
Contributor

ezyang commented Sep 5, 2018

Test failures are real...

@goldsborough goldsborough force-pushed the remove-tensor-underscore branch 2 times, most recently from e4f3ee0 to f62dad6 Compare September 5, 2018 23:00
@goldsborough goldsborough force-pushed the remove-tensor-underscore branch from f62dad6 to 570efe9 Compare September 6, 2018 00:48
Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

goldsborough is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

zdevito pushed a commit to zdevito/ATen that referenced this pull request Sep 7, 2018
Summary:
This PR cleans up the `at::Tensor` class by removing all methods that start with an underscore in favor of functions in the `at::` namespace. This greatly cleans up the `Tensor` class and makes it clearer what is the public and non-public API.

For this I changed `native_functions.yaml` and `Declarations.cwrap` to make all underscore methods `variant: function` (or add such a statement to begin with), and then fixed all code locations using the underscore methods.

ezyang colesbury gchanan
Pull Request resolved: pytorch/pytorch#11152

Differential Revision: D9683607

Pulled By: goldsborough

fbshipit-source-id: 97f869f788fa56639c05a439e2a33be49f10f543
PenghuiCheng pushed a commit to PenghuiCheng/pytorch that referenced this pull request Sep 11, 2018
…#11152)

Summary:
This PR cleans up the `at::Tensor` class by removing all methods that start with an underscore in favor of functions in the `at::` namespace. This greatly cleans up the `Tensor` class and makes it clearer what is the public and non-public API.

For this I changed `native_functions.yaml` and `Declarations.cwrap` to make all underscore methods `variant: function` (or add such a statement to begin with), and then fixed all code locations using the underscore methods.

ezyang colesbury gchanan
Pull Request resolved: pytorch#11152

Differential Revision: D9683607

Pulled By: goldsborough

fbshipit-source-id: 97f869f788fa56639c05a439e2a33be49f10f543
@ezyang ezyang added the merged label Jun 26, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants