Add __torch_function__ benchmarks. by hameerabbasi · Pull Request #35530 · pytorch/pytorch

hameerabbasi · 2020-03-27T11:47:39Z

Since the last one was apparently reverted.

dr-ci · 2020-03-27T11:48:47Z

💊 CircleCI build failures summary and remediations

As of commit 369c0ce (more details on the Dr. CI page):

4/4 failures introduced in this PR

🕵️ 4 new failures recognized by patterns

The following build failures do not appear to be due to upstream breakages:

pytorch_linux_backward_compatibility_check_test (1/4)

Step: "Test" (full log | pattern match details) <confirmed not flaky by 2 failures>

Mar 30 17:13:56 The PR is introducing backward incompatible changes to the operator library. Please contact PyTorch team to confirm whether this change is wanted or not.

Mar 30 17:13:56 processing existing schema:  aten::sparse_coo_tensor.size(int[] size, *, int dtype, int layout, Device device, bool pin_memory=False) -> (Tensor) 
Mar 30 17:13:56 processing existing schema:  aten::sparse_coo_tensor.indices(Tensor indices, Tensor values, *, int? dtype=None, int? layout=None, Device? device=None, bool? pin_memory=None) -> (Tensor) 
Mar 30 17:13:56 processing existing schema:  aten::sparse_coo_tensor.indices_size(Tensor indices, Tensor values, int[] size, *, int? dtype=None, int? layout=None, Device? device=None, bool? pin_memory=None) -> (Tensor) 
Mar 30 17:13:56 processing existing schema:  aten::split_with_sizes(Tensor self, int[] split_sizes, int dim=0) -> (Tensor[]) 
Mar 30 17:13:56 processing existing schema:  aten::squeeze(Tensor(a) self) -> (Tensor(a)) 
Mar 30 17:13:56 processing existing schema:  aten::squeeze.dim(Tensor(a) self, int dim) -> (Tensor(a)) 
Mar 30 17:13:56 processing existing schema:  aten::stft(Tensor self, int n_fft, int? hop_length=None, int? win_length=None, Tensor? window=None, bool normalized=False, bool onesided=True) -> (Tensor) 
Mar 30 17:13:56 skipping schema:  aten::sub_.Tensor(Tensor(a!) self, Tensor other, *, Scalar alpha=1) -> (Tensor(a!)) 
Mar 30 17:13:56 skipping schema:  aten::sub_.Scalar(Tensor(a!) self, Scalar other, Scalar alpha=1) -> (Tensor(a!)) 
Mar 30 17:13:56 processing existing schema:  aten::t(Tensor(a) self) -> (Tensor(a)) 
Mar 30 17:13:56 The PR is introducing backward incompatible changes to the operator library. Please contact PyTorch team to confirm whether this change is wanted or not.  
Mar 30 17:13:56  
Mar 30 17:13:56 Broken ops: [ 
Mar 30 17:13:56 	aten::owner(RRef(t) self) -> (__torch__.torch.classes.dist_rpc.WorkerInfo) 
Mar 30 17:13:56 	prepacked::conv2d_clamp_run(Tensor X, __torch__.torch.classes.xnnpack.Conv2dOpContext W_prepack) -> (Tensor Y) 
Mar 30 17:13:56 	prepacked::conv2d_clamp_prepack(Tensor W, Tensor? B, int[2] stride, int[2] padding, int[2] dilation, int groups, float? output_min=None, float? output_max=None) -> (__torch__.torch.classes.xnnpack.Conv2dOpContext) 
Mar 30 17:13:56 	prepacked::linear_clamp_run(Tensor X, __torch__.torch.classes.xnnpack.LinearOpContext W_prepack) -> (Tensor Y) 
Mar 30 17:13:56 	prepacked::linear_clamp_prepack(Tensor W, Tensor? B=None, float? output_min=None, float? output_max=None) -> (__torch__.torch.classes.xnnpack.LinearOpContext) 
Mar 30 17:13:56 ] 
Mar 30 17:13:56 + cleanup 
Mar 30 17:13:56 + retcode=1

pytorch_linux_xenial_py3_6_gcc5_4_test (2/4)

Step: "Test" (full log | pattern match details) <confirmed not flaky by 2 failures>

Mar 30 18:35:33 [E request_callback_impl.cpp:94] Received error while processing request type 2: PickleError: ScriptModules cannot be deepcopied using copy.deepcopy or saved using torch.save. Mixed serialization of script and non-script modules is not supported. For purely script modules use my_script_module.save() instead.

Mar 30 18:35:33   /opt/conda/lib/python3.6/site-packages/torch/distributed/rpc/internal.py(86): serialize 
Mar 30 18:35:33   /opt/conda/lib/python3.6/site-packages/torch/distributed/rpc/internal.py(135): serialize 
Mar 30 18:35:33  
Mar 30 18:35:33 [E request_callback_impl.cpp:94] Received error while processing request type 2: PickleError: ScriptModules cannot be deepcopied using copy.deepcopy or saved using torch.save. Mixed serialization of script and non-script modules is not supported. For purely script modules use my_script_module.save(<filename>) instead. 
Mar 30 18:35:33  
Mar 30 18:35:33 At: 
Mar 30 18:35:33   /opt/conda/lib/python3.6/site-packages/torch/jit/__init__.py(1773): __getstate__ 
Mar 30 18:35:33   /opt/conda/lib/python3.6/site-packages/torch/distributed/rpc/internal.py(86): serialize 
Mar 30 18:35:33   /opt/conda/lib/python3.6/site-packages/torch/distributed/rpc/internal.py(135): serialize 
Mar 30 18:35:33  
Mar 30 18:35:33 [E request_callback_impl.cpp:94] Received error while processing request type 2: PickleError: ScriptModules cannot be deepcopied using copy.deepcopy or saved using torch.save. Mixed serialization of script and non-script modules is not supported. For purely script modules use my_script_module.save(<filename>) instead. 
Mar 30 18:35:33  
Mar 30 18:35:33 At: 
Mar 30 18:35:33   /opt/conda/lib/python3.6/site-packages/torch/jit/__init__.py(1773): __getstate__ 
Mar 30 18:35:33   /opt/conda/lib/python3.6/site-packages/torch/distributed/rpc/internal.py(86): serialize 
Mar 30 18:35:33   /opt/conda/lib/python3.6/site-packages/torch/distributed/rpc/internal.py(135): serialize 
Mar 30 18:35:33  
Mar 30 18:35:33 ok (1.121s) 
Mar 30 18:35:34   test_unexepected_kwarg_is_specified (__main__.JitRpcTestWithSpawn) ... ok (1.117s) 
Mar 30 18:35:35   test_user_rrefs_confirmed (__main__.JitRpcTestWithSpawn) ... ok (1.118s) 
Mar 30 18:35:36   test_user_rrefs_confirmed_remote (__main__.JitRpcTestWithSpawn) ... ok (1.117s)

pytorch_linux_xenial_cuda10_2_cudnn7_py3_gcc7_test (3/4)

Step: "Test" (full log | pattern match details) <confirmed not flaky by 2 failures>

Mar 30 18:46:37 RuntimeError: test_autograd failed!

Mar 30 18:46:37 Generated XML report: test-reports/python-unittest/TEST-TestAutograd-20200330183849.xml 
Mar 30 18:46:37 Generated XML report: test-reports/python-unittest/TEST-TestAutogradDeviceTypeCPU-20200330183849.xml 
Mar 30 18:46:37 Generated XML report: test-reports/python-unittest/TEST-TestAutogradDeviceTypeCUDA-20200330183849.xml 
Mar 30 18:46:37 Generated XML report: test-reports/python-unittest/TEST-TestAutogradFunctional-20200330183849.xml 
Mar 30 18:46:37 Generated XML report: test-reports/python-unittest/TEST-TestMultithreadAutograd-20200330183849.xml 
Mar 30 18:46:37 Traceback (most recent call last): 
Mar 30 18:46:37   File "test/run_test.py", line 682, in <module> 
Mar 30 18:46:37     main() 
Mar 30 18:46:37   File "test/run_test.py", line 675, in main 
Mar 30 18:46:37     raise RuntimeError(message) 
Mar 30 18:46:37 RuntimeError: test_autograd failed! 
Mar 30 18:46:38 + cleanup 
Mar 30 18:46:38 + retcode=1 
Mar 30 18:46:38 + set +x 
Mar 30 18:46:38 =================== sccache compilation log =================== 
Mar 30 18:46:38 =========== If your build fails, please take a look at the log above for possible reasons =========== 
Mar 30 18:46:38 Compile requests                 0 
Mar 30 18:46:38 Compile requests executed        0 
Mar 30 18:46:38 Cache hits                       0 
Mar 30 18:46:38 Cache misses                     0 
Mar 30 18:46:38 Cache timeouts                   0

pytorch_linux_xenial_py3_clang5_asan_test (4/4)

Step: "Test" (full log | pattern match details) <confirmed not flaky by 2 failures>

Mar 30 17:22:42 caused by: Connection refused (os error 111)

Mar 30 17:22:42 +++ eval 'extract_trap_cmd ' 
Mar 30 17:22:42 ++++ extract_trap_cmd 
Mar 30 17:22:42 ++++ printf '%s\n' '' 
Mar 30 17:22:42 +++ printf '%s\n' cleanup 
Mar 30 17:22:42 ++ trap -- ' 
Mar 30 17:22:42 cleanup' EXIT 
Mar 30 17:22:42 ++ which sccache 
Mar 30 17:22:42 ++ sccache --stop-server 
Mar 30 17:22:42 Stopping sccache server... 
Mar 30 17:22:42 error: couldn't connect to server 
Mar 30 17:22:42 caused by: Connection refused (os error 111) 
Mar 30 17:22:42 ++ true 
Mar 30 17:22:42 ++ rm /var/lib/jenkins/sccache_error.log 
Mar 30 17:22:42 ++ SCCACHE_ERROR_LOG=/var/lib/jenkins/sccache_error.log 
Mar 30 17:22:42 ++ SCCACHE_IDLE_TIMEOUT=1200 
Mar 30 17:22:42 ++ RUST_LOG=sccache::server=error 
Mar 30 17:22:42 ++ sccache --start-server 
Mar 30 17:22:42 Starting sccache server... 
Mar 30 17:22:42 ++ sccache --zero-stats 
Mar 30 17:22:42 Compile requests                 0 
Mar 30 17:22:42 Compile requests executed        0

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker.

See how this bot performed.

This comment has been revised 27 times.

ezyang · 2020-03-30T14:12:37Z

Last time #34645

ezyang · 2020-03-30T14:13:48Z

You haven't fixed the issue that caused CI to fail?

Mar 26 20:13:22 + python bench.py -n 1 -m 1 Mar 26 20:13:22 ~/workspace/benchmarks/overrides_benchmark ~/workspace Mar 26 20:13:22 Traceback (most recent call last): Mar 26 20:13:22 File "bench.py", line 67, in <module> Mar 26 20:13:22 main() Mar 26 20:13:22 File "bench.py", line 61, in main Mar 26 20:13:22 t.__name__, (10 ** 6) * bench_min, (10 ** 6) * bench_std, Mar 26 20:13:22 UnicodeEncodeError: 'ascii' codec can't encode character '\u03bc' in position 54: ordinal not in range(128)

You may have to push to pytorch/pytorch on a branch named ci-all/your-branch-name to trigger the relevant ci job

Ah, I assumed it was hitting that error when parsing the file, not when printing the char.

You may have to push to pytorch/pytorch on a branch named ci-all/your-branch-name to trigger the relevant ci job

Pushed to ci-all/torch-function-benchmarks

hameerabbasi · 2020-03-31T06:27:37Z

This time, the failures really are unrelated.

facebook-github-bot

@ezyang is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

facebook-github-bot · 2020-04-01T14:24:34Z

@ezyang merged this pull request in 8c534bb.

suo · 2020-04-01T17:12:05Z

I think this broke master:

Apr 01 15:52:49 + python python pyspybench.py Tensor -n 1
Apr 01 16:33:10 python: can't open file 'python': [Errno 2] No such file or directory

example build: https://app.circleci.com/pipelines/github/pytorch/pytorch/149594/workflows/e8dd000c-8a01-4938-9fa8-ab9598b46e65/jobs/5020601

hameerabbasi · 2020-04-01T17:15:06Z

I apologize, I wonder why the CI didn't catch that.

Summary: Re-land of #35530 and #34645 Pull Request resolved: #36138 Differential Revision: D20893770 Pulled By: ezyang fbshipit-source-id: 75ab688a086f5fb87412a853df5246c0c39704ca

Summary: Re-land of pytorch#35530 and pytorch#34645 Pull Request resolved: pytorch#36138 Differential Revision: D20893770 Pulled By: ezyang fbshipit-source-id: 75ab688a086f5fb87412a853df5246c0c39704ca

Summary: Since the last one was apparently reverted. Pull Request resolved: pytorch#35530 Differential Revision: D20777341 Pulled By: ezyang fbshipit-source-id: 6aaaf2a0755359074ae3d0efe32018d78dafe976

Summary: Re-land of pytorch#35530 and pytorch#34645 Pull Request resolved: pytorch#36138 Differential Revision: D20893770 Pulled By: ezyang fbshipit-source-id: 75ab688a086f5fb87412a853df5246c0c39704ca

hameerabbasi requested a review from ezyang March 27, 2020 11:47

hameerabbasi changed the title ~~Torch function benchmark~~ Add __torch_function__ benchmarks. Mar 27, 2020

hameerabbasi force-pushed the torch-function-benchmark branch from 05049cb to 09ae99a Compare March 27, 2020 11:53

pytorchbot added the open source label Mar 27, 2020

hameerabbasi force-pushed the torch-function-benchmark branch 2 times, most recently from 24b004e to 7a84697 Compare March 30, 2020 08:34

hameerabbasi added 3 commits March 30, 2020 11:34

Add __torch_function__ benchmarks.

fb1f4aa

Address review by @ngoldblaum.

10bfd7e

Add types argument.

96e6d69

hameerabbasi force-pushed the torch-function-benchmark branch from 7a84697 to 4b34fb7 Compare March 30, 2020 09:34

ezyang reviewed Mar 30, 2020

View reviewed changes

hameerabbasi added 2 commits March 30, 2020 18:12

Add benchmarks to CI and add options for number of repetitions.

c28665c

Fix unicode printing issue.

369c0ce

hameerabbasi force-pushed the torch-function-benchmark branch from fa968a5 to 369c0ce Compare March 30, 2020 16:13

vincentqb added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Mar 30, 2020

facebook-github-bot reviewed Mar 31, 2020

View reviewed changes

facebook-github-bot closed this in 8c534bb Apr 1, 2020

facebook-github-bot added the merged label Apr 1, 2020

hameerabbasi mentioned this pull request Apr 7, 2020

[RELAND] Add __torch_function__ benchmarks #36138

Closed

mruberry added the Merged label Oct 28, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add __torch_function__ benchmarks.#35530

Add __torch_function__ benchmarks.#35530
hameerabbasi wants to merge 5 commits intopytorch:masterfrom
Quansight:torch-function-benchmark

hameerabbasi commented Mar 27, 2020

Uh oh!

dr-ci Bot commented Mar 27, 2020 •

edited

Loading

Uh oh!

ezyang commented Mar 30, 2020

Uh oh!

ezyang Mar 30, 2020

Uh oh!

hameerabbasi Mar 30, 2020

Uh oh!

hameerabbasi Mar 30, 2020 •

edited

Loading

Uh oh!

hameerabbasi commented Mar 31, 2020

Uh oh!

facebook-github-bot left a comment

Uh oh!

facebook-github-bot commented Apr 1, 2020

Uh oh!

suo commented Apr 1, 2020 •

edited

Loading

Uh oh!

hameerabbasi commented Apr 1, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Conversation

hameerabbasi commented Mar 27, 2020

Uh oh!

dr-ci Bot commented Mar 27, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CircleCI build failures summary and remediations

🕵️ 4 new failures recognized by patterns

pytorch_linux_backward_compatibility_check_test (1/4)

pytorch_linux_xenial_py3_6_gcc5_4_test (2/4)

pytorch_linux_xenial_cuda10_2_cudnn7_py3_gcc7_test (3/4)

pytorch_linux_xenial_py3_clang5_asan_test (4/4)

Uh oh!

ezyang commented Mar 30, 2020

Uh oh!

ezyang Mar 30, 2020

Choose a reason for hiding this comment

Uh oh!

hameerabbasi Mar 30, 2020

Choose a reason for hiding this comment

Uh oh!

hameerabbasi Mar 30, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hameerabbasi commented Mar 31, 2020

Uh oh!

facebook-github-bot left a comment

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Apr 1, 2020

Uh oh!

suo commented Apr 1, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hameerabbasi commented Apr 1, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

dr-ci Bot commented Mar 27, 2020 •

edited

Loading

hameerabbasi Mar 30, 2020 •

edited

Loading

suo commented Apr 1, 2020 •

edited

Loading