Remove template parameter from Tensor by jerryzh168 · Pull Request #9125 · pytorch/pytorch

jerryzh168 · 2018-07-03T03:40:17Z

Summary:
Use inheritance for polymorphism, and remove template parameter
This is to change the templating in call sites, the core implementations will change later

We removed template parameter from Tensor as an effort to Pytorch and Caffe2 backend unification.

New caffe2::Tensor class

Before Caffe2 Tensor class was compile-time fixed to bind to a particular device/context. With this change, we're making it a runtime property (stored inside the tensor), but preserve the same semantics. For example, one has to specify device type in order to create a Tensor - there are no uninitialized tensors. More specifically the changes are:

We added an extra argument DeviceType to most of the constructors of the tensor, e.g. (Tensor(DeviceType type)),
Semantics of constructor Tensor(const Tensor& src, ContextForCopy* context); is changed, in this constructor, the second context is passed in to enable us to call the templated Copy function, it could be in a different context as source and target previously, now we'll enforce that the context should have same device type as src, if it is provided.
To preserve 'get-or-construct' semantics of Blob, we added specialized getter Blob::GetMutableTensor that verifies both that Blob contains a Tensor and that it's of a correct type
Specifically, Tensor type is not default-constructible any more (as we don't have unknown device tensors) and thus some of the code handling STL containers needs to change

For details about how the tensor class look like right now, please see https://github.com/pytorch/pytorch/blob/master/caffe2/core/tensor.h

We did a massive codemod of the fbsource and hopefully covered majority of usages. However, there are probably a few that slipped through the contbuild. You might need to refactor your custom code according the following guide:

# Tensor
## Type
using TensorCPU = Tensor<CPUContext>; --> using TensorCPU = Tensor;

## Construction
### As class member variable
TensorCPU x; --> Tensor x{CPU};
// TensorCPU is just Tensor<CPUContext>, if Tensor is templated on Context,
// we'll write:
Tensor<Context> x; --> Tensor x{Context::GetDeviceType()};
// In the following we'll just use TensorCPU as example
### As local variable
TensorCPU x; --> Tensor x(CPU);
### make_unique
make_unique<TensorCPU>() --> make_unique<Tensor>(CPU)
### Constructor by context
CUDAContext context;
// y is a TensorCPU
TensorCPU x(y, &context); // this constructor is removed

-->

Tensor x(y, &context, CPU);
## Containers
### Vector initialization
 vector<TensorCPU> tensors(numTensors);
 -->
 vector<TensorCPU> tensors;
 for (auto i = 0; i < numTensors; ++i) {
   tensors.emplace_back(CPU);
 }
### Vector access
No change
### Map construction
map<string, TensorCPU> tensor_map;
tensor_map.insert(std::pair<string, TensorCPU>(name, TensorCPU()));
-->
tensor_map.emplace(name, Tensor(CPU));
 
### Map access ([])
map<TensorCPU> map;
map[name]
--> 
map.emplace(name, Tensor(CPU));
 map.at(name);

# Blob
GetMutable<TensorCPU> --> GetMutableTensor(CPU)

# Note for Get function we just keep it as Get<TensorCPU>()(equivalent to Get<Tensor>()) at this moment
# Since it won't affect the current running code
# ideally it should be rewritten as following, which gives an extra verification
# on the type of tensor contained in the blob
Get<TensorCPU>() --> Get<Tensor>(CPU);

Context and StaticContext

We virtualized the functions in Context in order to achieve runtime polymorphism. The static functions has been split into a StaticContext class, which has one to one correspondence with DeviceType (CPU, CUDA etc.) and it has a global singleton object. We'll store a pointer to the singleton in Tensor to indicate the context of Tensor.
Original Context class contained routines to copy between different contexts, however, most of the time the use case is between three types of copies: CopyToCPU, CopyFromCPU and CopySameDevice, therefore, in the new Context, while we still keep the one general templated function that can copy between arbitrary contexts (CopyBytes), we rewrote the other uses of the Copy into these three special variants of copies that do not rely on template.

# Context
Copy<int, CPUContext, CUDAContext>(...) --> CopyFromCPU<int>(...)
Copy<int, CUDAContext, CPUContext>(...) --> CopyToCPU<int>(...)
Copy<int, CUDAContext, CUDAContext>(...) --> CopySameDevice<int>(...)
Similarily,
Copy<CPUContext, CUDAContext>(...) --> CopyFromCPU(...)
We also have calls using CopyItems/CopyBytes

As a side note, the distinction between three copies is that:
- CopyBytes is copy of data pointed by raw pointers (void*)
- Copy is templated on the data type that we want to copy (T*)
- CopyItems get the type info from explicitly passed TypeMeta

Differential Revision: D8121878

jerryzh168 · 2018-07-07T00:54:21Z

@pytorchbot retest this please

Summary: Pull Request resolved: pytorch/pytorch#9125 Closes pytorch/pytorch#9125 Use inheritance for polymorphism, and remove template parameter This is to change the templating in call sites, the core implementations will change later - Added a `static_context_` field, which stores a pointer to the per-device singleton `StaticContext`, e.g. `CPUStaticContext`, `CUDAStaticContext` etc. - Default constructor is deleted. - All constructor of `Tensor` are pass in an extra argument `DeviceType` that indicates the type we want the tensor to have. except for `Tensor(const vector<TIndex>& dims, const vector<T>& values, BaseContext* context)`, where we use `context` to specify the `DeviceType` of the `Tensor`. - constructor `Tensor(const Tensor& src, ContextForCopy* context)` is removed, since with the templated being removed, we have no way of knowing what the type we want the target Tensor to have now, it is changed to `Tensor(const Tensor& src, DeviceType device_type)`, where `device_type` is the device type we want the target tensor to have. - Changed semantics of CopyFrom a bit, previously, it is `CopyFrom(const Tensor<SrcContext>& src, ContextForCopy* context)` the second argument is used for calling the templated Copy functions. This is changed to `CopyFrom(const Tensor& src, BaseContext* context = nullptr)`, when the second argument is not provided, we'll use the context from source tensor, when it is provided, we'll need to make sure it has the same type as source tensor. Also, in case that we are copying from a cpu tensor to a non-cpu tensor, we'll need to create a non-cpu context to perform the copy since CPU context don't know how to do the copy from CPU to non-CPU tensor. Note: Some changes are postponed just to keep this diff a bit smaller. Please see `TODO`s. Reviewed By: dzhulgakov Differential Revision: D8121878 fbshipit-source-id: 0964bde5b27a12f2e0c52c7040f1214f15cf3231

Summary: Pull Request resolved: https://github.com/facebookresearch/weakly-supervised-action-detection/pull/13 Pull Request resolved: pytorch#166 Pull Request resolved: pytorch/pytorch#9125 Closes pytorch/pytorch#9125 Use inheritance for polymorphism, and remove template parameter This is to change the templating in call sites, the core implementations will change later Before Caffe2 Tensor class was compile-time fixed to bind to a particular device/context. With this change, we're making it a runtime property (stored inside the tensor), but preserve the same semantics. For example, one has to specify device type in order to create a Tensor - there are no uninitialized tensors. More specifically the changes are: 1. We added an extra argument *DeviceType* to most of the constructors of the tensor, e.g. (Tensor(DeviceType type)), 2. Semantics of constructor Tensor(const Tensor<SrcContext>& src, ContextForCopy* context); is changed, in this constructor, the second context is passed in to enable us to call the templated Copy function, it could be in a different context as source and target previously, now we'll enforce that the context should have same device type as src, if it is provided. 3. To preserve 'get-or-construct' semantics of Blob, we added specialized getter Blob::GetMutableTensor that verifies both that Blob contains a Tensor and that it's of a correct type 4. Specifically, Tensor type is not default-constructible any more (as we don't have unknown device tensors) and thus some of the code handling STL containers needs to change Note: Some changes are postponed just to keep this diff a bit smaller. Please see `TODO`s. Reviewed By: dzhulgakov Differential Revision: D8121878 fbshipit-source-id: a0ee4bf360c49bf75e85796de3a4eb21a4f6fc5f

Summary: Pull Request resolved: https://github.com/facebookresearch/weakly-supervised-action-detection/pull/13 Pull Request resolved: pytorch#166 Pull Request resolved: pytorch/pytorch#9125 Closes pytorch/pytorch#9125 Use inheritance for polymorphism, and remove template parameter This is to change the templating in call sites, the core implementations will change later Before Caffe2 Tensor class was compile-time fixed to bind to a particular device/context. With this change, we're making it a runtime property (stored inside the tensor), but preserve the same semantics. For example, one has to specify device type in order to create a Tensor - there are no uninitialized tensors. More specifically the changes are: 1. We added an extra argument *DeviceType* to most of the constructors of the tensor, e.g. (Tensor(DeviceType type)), 2. Semantics of constructor Tensor(const Tensor<SrcContext>& src, ContextForCopy* context); is changed, in this constructor, the second context is passed in to enable us to call the templated Copy function, it could be in a different context as source and target previously, now we'll enforce that the context should have same device type as src, if it is provided. 3. To preserve 'get-or-construct' semantics of Blob, we added specialized getter Blob::GetMutableTensor that verifies both that Blob contains a Tensor and that it's of a correct type 4. Specifically, Tensor type is not default-constructible any more (as we don't have unknown device tensors) and thus some of the code handling STL containers needs to change Note: Some changes are postponed just to keep this diff a bit smaller. Please see `TODO`s. Reviewed By: dzhulgakov Differential Revision: D8121878 fbshipit-source-id: 88f93b92b8f3716fc43e01252fc64a1d0f8b1097

ezyang · 2018-07-24T23:56:37Z

@pytorchbot retest this please

jerryzh168 · 2018-07-25T22:25:55Z

@pytorchbot retest this please

Summary: Pull Request resolved: https://github.com/facebookresearch/weakly-supervised-action-detection/pull/13 Pull Request resolved: pytorch/translate#166 Pull Request resolved: pytorch#9125 Closes pytorch#9125 Use inheritance for polymorphism, and remove template parameter This is to change the templating in call sites, the core implementations will change later Before Caffe2 Tensor class was compile-time fixed to bind to a particular device/context. With this change, we're making it a runtime property (stored inside the tensor), but preserve the same semantics. For example, one has to specify device type in order to create a Tensor - there are no uninitialized tensors. More specifically the changes are: 1. We added an extra argument *DeviceType* to most of the constructors of the tensor, e.g. (Tensor(DeviceType type)), 2. Semantics of constructor Tensor(const Tensor<SrcContext>& src, ContextForCopy* context); is changed, in this constructor, the second context is passed in to enable us to call the templated Copy function, it could be in a different context as source and target previously, now we'll enforce that the context should have same device type as src, if it is provided. 3. To preserve 'get-or-construct' semantics of Blob, we added specialized getter Blob::GetMutableTensor that verifies both that Blob contains a Tensor and that it's of a correct type 4. Specifically, Tensor type is not default-constructible any more (as we don't have unknown device tensors) and thus some of the code handling STL containers needs to change Note: Some changes are postponed just to keep this diff a bit smaller. Please see `TODO`s. Reviewed By: xw285cornell Differential Revision: D8121878 fbshipit-source-id: 5597b009f6a546bed6e474d3507e672f8ea3ffe5

jerryzh168 force-pushed the export-D8121878 branch from 3de1c30 to fd8cad5 Compare July 3, 2018 05:00

jerryzh168 force-pushed the export-D8121878 branch from fd8cad5 to abb67ea Compare July 3, 2018 15:55

jerryzh168 force-pushed the export-D8121878 branch from abb67ea to 0cf6385 Compare July 3, 2018 16:43

jerryzh168 force-pushed the export-D8121878 branch from 0cf6385 to 2b15f8d Compare July 3, 2018 16:58

jerryzh168 force-pushed the export-D8121878 branch from 2b15f8d to e0cae40 Compare July 3, 2018 19:30

jerryzh168 force-pushed the export-D8121878 branch from e0cae40 to dde3ef8 Compare July 3, 2018 19:52

jerryzh168 force-pushed the export-D8121878 branch from dde3ef8 to 4bb7cc7 Compare July 3, 2018 21:17

jerryzh168 force-pushed the export-D8121878 branch from 4bb7cc7 to 4a9eab9 Compare July 3, 2018 21:26

jerryzh168 force-pushed the export-D8121878 branch from 4a9eab9 to 79d12c8 Compare July 3, 2018 21:58

jerryzh168 force-pushed the export-D8121878 branch from 79d12c8 to 0d923ff Compare July 4, 2018 00:35

jerryzh168 force-pushed the export-D8121878 branch from 0d923ff to c591917 Compare July 5, 2018 17:11

jerryzh168 force-pushed the export-D8121878 branch from c591917 to 4c8056a Compare July 5, 2018 17:44

jerryzh168 force-pushed the export-D8121878 branch from 4c8056a to 5d842f8 Compare July 5, 2018 18:55

jerryzh168 force-pushed the export-D8121878 branch from 5d842f8 to 26a9c6e Compare July 5, 2018 19:18

jerryzh168 force-pushed the export-D8121878 branch from 26a9c6e to 8c320fb Compare July 5, 2018 19:46

jerryzh168 force-pushed the export-D8121878 branch from 1b3070c to 84bcc2b Compare July 6, 2018 22:51

weiyangfb added the caffe2 label Jul 10, 2018

jerryzh168 force-pushed the export-D8121878 branch from 84bcc2b to c997c28 Compare July 19, 2018 18:32

jerryzh168 force-pushed the export-D8121878 branch from c997c28 to 44e2026 Compare July 20, 2018 21:52

jerryzh168 force-pushed the export-D8121878 branch from 44e2026 to 1409f26 Compare July 21, 2018 17:13

jerryzh168 force-pushed the export-D8121878 branch from 1409f26 to 99383ed Compare July 23, 2018 04:18

jerryzh168 force-pushed the export-D8121878 branch from 99383ed to 7f0b736 Compare July 23, 2018 06:11

jerryzh168 force-pushed the export-D8121878 branch from 7f0b736 to 5290cdf Compare July 23, 2018 17:02

jerryzh168 mentioned this pull request Jul 23, 2018

Remove template parameter from Tensor (#9125) pytorch/translate#166

Closed

jerryzh168 force-pushed the export-D8121878 branch from 5290cdf to 395256e Compare July 24, 2018 21:51

jerryzh168 force-pushed the export-D8121878 branch from 395256e to 9c5b4c4 Compare July 24, 2018 22:14

This was referenced Jul 27, 2018

Remove template parameter from Tensor (#13) pytorch/translate#175

Closed

Remove template parameter from Tensor (#13) #9939

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove template parameter from Tensor#9125

Remove template parameter from Tensor#9125
jerryzh168 wants to merge 1 commit intopytorch:masterfrom
jerryzh168:export-D8121878

jerryzh168 commented Jul 3, 2018 •

edited

Loading

Uh oh!

jerryzh168 commented Jul 7, 2018

Uh oh!

ezyang commented Jul 24, 2018

Uh oh!

jerryzh168 commented Jul 25, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

jerryzh168 commented Jul 3, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

New caffe2::Tensor class

Context and StaticContext

Uh oh!

jerryzh168 commented Jul 7, 2018

Uh oh!

ezyang commented Jul 24, 2018

Uh oh!

jerryzh168 commented Jul 25, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jerryzh168 commented Jul 3, 2018 •

edited

Loading