Skip to content

[pt1][tensor] caffe2::empty for Resize+mutable_data refactor#12407

Closed
jerryzh168 wants to merge 70 commits intomasterfrom
export-D10207890
Closed

[pt1][tensor] caffe2::empty for Resize+mutable_data refactor#12407
jerryzh168 wants to merge 70 commits intomasterfrom
export-D10207890

Conversation

@jerryzh168
Copy link
Contributor

@jerryzh168 jerryzh168 commented Oct 5, 2018 •
edited
Loading

Stack:
    :black_circle:  #12407 [pt1][tensor] caffe2::empty for Resize+mutable_data refactor  💚

Motivation

We want to use tensor factory to refactor the caffe2's old way of initialize Tensor by Resize and mutable_data
in order to eliminate uninitialized Tensor.

Previously when we want to create a Tensor in caffe2, we'll do the following

Tensor x(CPU); // device type provided
x.Resize({1, 2, 3}); // size provided
x.mutable_data<float>(); // data type provided and memory allocated

This leaves Tensor in not fully initialized state during the process, to eliminate this, we
want to provide all the needed information in the begining. ATen already has its TensorFactories: https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/TensorFactories.cpp, and there is a TensorOption, we want to adopt the same interface to ease future refactoring.

In the callsite, we used to have Output(i) that returns a Blob that contains an uninitialized Tensor and we'll call Resize and mutable_data afterwards to provide dimension and data type,

// uninitialized tensor
auto* Y = Output(0);
// set dimensions
Y->Resize({1, 2, 3});
// actually allocate the data
auto* data = Y->mutable_data<float>();
// After this step, Tensor is fully initialized.

We want to change it to the following:

// provide dimensions and TensorOptions which include device type and data type.
// This will set all the information of Tensor properly and also allocate memory.
auto* Y = Output(0, {1, 2, 3}, at::device({context_.device_type()}).template dtype<T>());
// Tensor is fully initialized after this step

// following `mutable_data` call won't allocate memory.
auto* data = Y->mutable_data<float>();

Perf result

microbenchmarks

============================================================================
caffe2/caffe2/fb/benchmarks/core_overhead_benchmark.ccrelative  time/iter  iters/s
============================================================================
OperatorNewOutputTensorAPI                                   3.27us  306.05K
OperatorOldOutputTensorAPI                                   3.55us  281.54K
============================================================================

Differential Revision: D10207890

@jerryzh168
V8: rebase
1a182fc 
Differential Revision: D9779821
Differential Version: 57806436
@jerryzh168
V16: .
ae56e3e 
Differential Revision: D9779821
Differential Version: 58572220
@jerryzh168
V18: .
2596553 
Differential Revision: D9779821
Differential Version: 58709105
@jerryzh168
V19: rebase
20b11ae 
Differential Revision: D9779821
Differential Version: 58713403
@jerryzh168
V20: Commit rFBS8b239ea50af7a0556fde2382f58f79194f0e3dc1
d546673 
Differential Revision: D9779821
Differential Version: 58748651
@jerryzh168
V11: context_test
b27ccdc 
Differential Revision: D9889990
Differential Version: 58748485
@jerryzh168
V13: change function name to move_context
8de7bec 
Differential Revision: D9889990
Differential Version: 58751934
@jerryzh168
V16: .
2f5f6d9 
Differential Revision: D9889990
Differential Version: 58799279
@jerryzh168
V1: Initial commit
364d610 
Differential Revision: D10033396
Differential Version: 58815881
@jerryzh168
V17: .
72fcbd2 
Differential Revision: D9889990
Differential Version: 58825947
@jerryzh168
V2: .
e5cd9bf 
Differential Revision: D10033396
Differential Version: 58835275
@jerryzh168
V18: build
267aef3 
Differential Revision: D9889990
Differential Version: 58856304
@jerryzh168
V3: restore Registry macro
14cae72 
Differential Revision: D10033396
Differential Version: 58856963
@jerryzh168
V19: .
8f4b964 
Differential Revision: D9889990
Differential Version: 58858304
@jerryzh168
V20: miopen_wrapper
6c9b900 
Differential Revision: D9889990
Differential Version: 58875107
@jerryzh168
V21: .
a823989 
Differential Revision: D9889990
Differential Version: 58881242
@jerryzh168
V5: .
b671424 
Differential Revision: D10001033
Differential Version: 58937972
@jerryzh168
V2: Allocator -> AllocatorGetter
0f1d7cf 
Differential Revision: D10022853
Differential Version: 58947676
@jerryzh168
V3: .
902657a 
Differential Revision: D10022853
Differential Version: 58951172
@jerryzh168
V5: rebase
cba9952 
Differential Revision: D10033396
Differential Version: 59085834
@jerryzh168
V6: .
732473e 
Differential Revision: D10033396
Differential Version: 59088508
@jerryzh168
V7: .
8dabf8b 
Differential Revision: D10033396
Differential Version: 59091126
@jerryzh168
V22: addressing comments
a2d121f 
Differential Revision: D9889990
Differential Version: 59110247
@jerryzh168
V12: rebase
a65359c 
Differential Revision: D10033396
Differential Version: 59227528
@jerryzh168
V28: .
5bb592a 
Differential Revision: D9889990
Differential Version: 59236356
@jerryzh168
V39: .
ce7d13f 
Differential Revision: D9889990
Differential Version: 59596351
@jerryzh168
V40: lints and comments
ed51f1d 
Differential Revision: D9889990
Differential Version: 59619253
@jerryzh168
V41: inline data_ptr
78c23c7 
Differential Revision: D9889990
Differential Version: 59621759
@jerryzh168
V8: OpenCL context
701c879 
Differential Revision: D10001033
Differential Version: 59624116
@jerryzh168
V42: error in HIP
4acb24c 
Differential Revision: D9889990
Differential Version: 59625764
@jerryzh168 jerryzh168 mentioned this pull request Dec 12, 2018
Closed
@jerryzh168 jerryzh168 mentioned this pull request Dec 12, 2018
Closed
@jerryzh168 jerryzh168 mentioned this pull request Dec 12, 2018
Closed
@jerryzh168 jerryzh168 mentioned this pull request Dec 12, 2018
Closed
This was referenced Dec 28, 2018
Closed
Closed
This was referenced Jan 9, 2019
Closed
Closed
Closed
Closed
Closed
Closed
Closed
This was referenced Jan 25, 2019
Closed
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ezyang ezyang ezyang approved these changes

@dzhulgakov dzhulgakov dzhulgakov left review comments

+1 more reviewer

@Yangqing Yangqing Yangqing left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@jerryzh168 @ezyang @Yangqing @dzhulgakov