Vectorize copysign on CPU by peterbell10 · Pull Request #51792 · pytorch/pytorch

peterbell10 · 2021-02-05T18:08:34Z

Stack from ghstack:

Support JIT shape analysis for div with rounding_mode argument #50438 Support JIT shape analysis for div with rounding_mode argument
grad_fn getter for optional strings #55225 grad_fn getter for optional strings
Sparse support for division rounding_mode argument #51989 Sparse support for division rounding_mode argument
Vectorize floating point floor_divide #55380 Vectorize floating point floor_divide
Vectorize copysign on CPU #51792 Vectorize copysign on CPU

Differential Revision: D27769007

[ghstack-poisoned]

ghstack-source-id: 4089993 Pull Request resolved: pytorch#51792

facebook-github-bot · 2021-02-05T21:14:34Z

💊 CI failures summary and remediations

As of commit cc2a5eb (more details on the Dr. CI page):

1/2 failures introduced in this PR
1/2 tentatively recognized as flaky ❄️
- Click here to rerun these jobs

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

pytorch_macos_10_13_py3_test (1/1)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

Apr 13 17:52:49 [E request_callback_no_python.cpp:656] Received error while processing request type 256: The following operation failed in the TorchScript interpreter.

Apr 13 17:52:49 
Apr 13 17:52:49 [E request_callback_no_python.cpp:656] Received error while processing request type 256: The following operation failed in the TorchScript interpreter.
Apr 13 17:52:49 Traceback of TorchScript (most recent call last):
Apr 13 17:52:49   File "/Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/testing/_internal/distributed/rpc/rpc_test.py", line 338, in raise_func_script
Apr 13 17:52:49 @torch.jit.script
Apr 13 17:52:49 def raise_func_script(expected_err: str) -> torch.Tensor:
Apr 13 17:52:49     raise ValueError(expected_err)
Apr 13 17:52:49     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
Apr 13 17:52:49 RuntimeError: Expected error
Apr 13 17:52:49 
Apr 13 17:52:49 [E request_callback_no_python.cpp:656] Received error while processing request type 256: The following operation failed in the TorchScript interpreter.
Apr 13 17:52:49 Traceback of TorchScript (most recent call last):
Apr 13 17:52:49   File "/Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/testing/_internal/distributed/rpc/rpc_test.py", line 338, in raise_func_script
Apr 13 17:52:49 @torch.jit.script
Apr 13 17:52:49 def raise_func_script(expected_err: str) -> torch.Tensor:
Apr 13 17:52:49     raise ValueError(expected_err)
Apr 13 17:52:49     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
Apr 13 17:52:49 RuntimeError: Expected error
Apr 13 17:52:49 
Apr 13 17:52:50 ok (2.708s)
Apr 13 17:52:52   test_wait_all_multiple_call (__main__.TensorPipeRpcTestWithSpawn) ... [W tensorpipe_agent.cpp:110] Failed to look up the IP address for the hostname (EAI_NONAME: unknown node or service (this error originated at tensorpipe/transport/uv/utility.cc:97)), defaulting to 127.0.0.1

❄️ 1 failure tentatively classified as flaky

but reruns have not yet been triggered to confirm:

pytorch_linux_bionic_py3_8_gcc9_coverage_test2 (1/1)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun) ❄️

Apr 13 16:37:09 RuntimeError: Process 0 terminated or timed out after 100.07356643676758 seconds

Apr 13 16:37:09 ======================================================================
Apr 13 16:37:09 ERROR [100.127s]: test_py_tensors_multi_async_call (__main__.TensorPipeRpcTestWithSpawn)
Apr 13 16:37:09 ----------------------------------------------------------------------
Apr 13 16:37:09 Traceback (most recent call last):
Apr 13 16:37:09   File "/opt/conda/lib/python3.8/site-packages/torch/testing/_internal/common_distributed.py", line 322, in wrapper
Apr 13 16:37:09     self._join_processes(fn)
Apr 13 16:37:09   File "/opt/conda/lib/python3.8/site-packages/torch/testing/_internal/common_distributed.py", line 515, in _join_processes
Apr 13 16:37:09     self._check_return_codes(elapsed_time)
Apr 13 16:37:09   File "/opt/conda/lib/python3.8/site-packages/torch/testing/_internal/common_distributed.py", line 563, in _check_return_codes
Apr 13 16:37:09     raise RuntimeError('Process {} terminated or timed out after {} seconds'.format(i, elapsed_time))
Apr 13 16:37:09 RuntimeError: Process 0 terminated or timed out after 100.07356643676758 seconds
Apr 13 16:37:09 
Apr 13 16:37:09 ----------------------------------------------------------------------
Apr 13 16:37:09 Ran 356 tests in 1269.955s
Apr 13 16:37:09 
Apr 13 16:37:09 FAILED (errors=1, skipped=6)
Apr 13 16:37:09 
Apr 13 16:37:09 Generating XML reports...
Apr 13 16:37:09 Generated XML report: test-reports/python-unittest/distributed.rpc.test_tensorpipe_agent/TEST-TensorPipeDdpComparisonTestWithSpawn-20210413161559.xml
Apr 13 16:37:09 Generated XML report: test-reports/python-unittest/distributed.rpc.test_tensorpipe_agent/TEST-TensorPipeDdpUnderDistAutogradTestWithSpawn-20210413161559.xml
Apr 13 16:37:09 Generated XML report: test-reports/python-unittest/distributed.rpc.test_tensorpipe_agent/TEST-TensorPipeDistAutogradTestWithSpawn-20210413161559.xml

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

[ghstack-poisoned]

ghstack-source-id: 76b0813 Pull Request resolved: pytorch#51792

[ghstack-poisoned]

ghstack-source-id: bef766e Pull Request resolved: pytorch#51792

[ghstack-poisoned]

mruberry · 2021-04-07T07:27:36Z

    store(tmp);
    exp.store(tmp_exp);
-    for (int64_t i = 0; i < size(); i++) {
+    for (int i = 0; i < size(); i++) {


type i like size():

auto i = decltype(size()){0}

I really don't like decltype(size()) so added a using size_type to Vec256 that I can use instead.

mruberry · 2021-04-07T07:30:18Z

+namespace std {
+
+/// Used by vec256<c10::Half>::map
+inline c10::Half acos(c10::Half a) { return std::acos(float(a));}


I'm not sure what the history of half math is. @ngimel, do you know?

No, I don't. I once wondered how math overloads work, but didn't get a full picture, I wonder why it's necessary, shouldn't there be an implicit Half->float conversion (that's defined)? I thought that's how std math functions work in cuda kernels.

I assumed BFloat-math.h was an accepted pattern, but if that's not true then I've just removed the Half-math.h file and focused on copysign.

malfet

Please rebase against ToT to pick up fix for #55613
Also, do not introduce any new std::min,std::max overrides for types that have default constructor, otherwise it can lead to unexpected sideeffects.

malfet · 2021-04-12T23:49:50Z

      std::is_floating_point<T>::value ||
-      std::is_same<T, at::Half>::value> {
+      std::is_same<T, at::Half>::value ||
+      std::is_same<T, at::BFloat16>::value> {


Can it be put into separate PR (as it have nothing to do with vectorization goal)

malfet · 2021-04-12T23:51:25Z

-    });
+    cpu_kernel_vec(iter,
+      [](scalar_t a, scalar_t b) -> scalar_t {
+        return std::copysign(a, b);


Again, we can not use std::copysign otherwise it will cause an ICE

[ghstack-poisoned]

peterbell10 · 2021-04-13T20:04:48Z

PTAL @malfet, @mruberry

mruberry

This looks OK to me but not an expert on copysign or c10/util. @malfet and @ezyang, would you two like to take a look?

ezyang · 2021-04-14T15:41:02Z

Actually, going to wait on @malfet, as I don't quite understand the ICE concerns.

malfet · 2021-04-14T15:49:18Z

Looks good to me, thank you for the updates

mruberry · 2021-04-14T17:38:29Z

Thanks for taking a look, @ezyang and @malfet!

facebook-github-bot · 2021-04-18T09:16:04Z

@mruberry merged this pull request in 8c74e1b.

Summary: Pull Request resolved: pytorch#51792 Test Plan: Imported from OSS Reviewed By: agolynski Differential Revision: D27769007 Pulled By: mruberry fbshipit-source-id: 65fceb9f59ed6afee4452278992340da104ed5fe

Vectorize floating point floor_divide

1db8b5c

[ghstack-poisoned]

This was referenced Feb 5, 2021

Warn about floor_divide performing incorrect rounding #50281

Closed

Sparse support for division rounding_mode argument #50437

Merged

Support JIT shape analysis for div with rounding_mode argument #50438

Closed

facebook-github-bot added the cla signed label Feb 5, 2021

Update on "Vectorize floating point floor_divide"

c50b84a

[ghstack-poisoned]

peterbell10 added a commit to peterbell10/pytorch that referenced this pull request Feb 5, 2021

Vectorize floating point floor_divide

d334cbf

ghstack-source-id: 4089993 Pull Request resolved: pytorch#51792

pytorchbot added the open source label Feb 5, 2021

peterbell10 added 6 commits February 6, 2021 01:58

Update on "Vectorize floating point floor_divide"

56695bd

[ghstack-poisoned]

Update on "Vectorize floating point floor_divide"

a9125a8

[ghstack-poisoned]

Update on "Vectorize floating point floor_divide"

94f79aa

[ghstack-poisoned]

Update on "Vectorize floating point floor_divide"

09ec6bf

[ghstack-poisoned]

Update on "Vectorize floating point floor_divide"

defb7e6

[ghstack-poisoned]

Update on "Vectorize floating point floor_divide"

646ee5f

[ghstack-poisoned]

peterbell10 added a commit to peterbell10/pytorch that referenced this pull request Feb 8, 2021

Vectorize floating point floor_divide

275c7d7

ghstack-source-id: 76b0813 Pull Request resolved: pytorch#51792

Update on "Vectorize floating point floor_divide"

196f4b5

[ghstack-poisoned]

This was referenced Feb 9, 2021

Replace rounding_mode="true" with rounding_mode=None #51988

Closed

Sparse support for division rounding_mode argument #51989

Closed

peterbell10 added a commit to peterbell10/pytorch that referenced this pull request Feb 9, 2021

Vectorize floating point floor_divide

cbe6cfc

ghstack-source-id: bef766e Pull Request resolved: pytorch#51792

peterbell10 added 3 commits February 10, 2021 00:15

Update on "Vectorize floating point floor_divide"

860e906

[ghstack-poisoned]

Update on "Vectorize floating point floor_divide"

f048b70

[ghstack-poisoned]

Update on "Vectorize floating point floor_divide"

abfb846

[ghstack-poisoned]

peterbell10 mentioned this pull request Feb 11, 2021

Remove rounding_mode="true" option #52139

Closed

peterbell10 added 4 commits February 11, 2021 14:27

Update on "Vectorize floating point floor_divide"

d0a173a

[ghstack-poisoned]

Update on "Vectorize floating point floor_divide"

89222c9

[ghstack-poisoned]

Update on "Vectorize floating point floor_divide"

66df697

[ghstack-poisoned]

Update on "Vectorize floating point floor_divide"

3216fb9

[ghstack-poisoned]

peterbell10 commented Apr 6, 2021

View reviewed changes

Comment thread aten/src/ATen/native/cpu/BinaryOpsKernel.cpp Outdated

peterbell10 requested a review from mruberry April 6, 2021 17:47

mruberry reviewed Apr 7, 2021

View reviewed changes

Comment thread aten/src/ATen/cpu/vec256/vec256_base.h Outdated

mruberry reviewed Apr 7, 2021

View reviewed changes

Comment thread aten/src/ATen/native/quantized/cpu/kernels/QuantizedOpKernels.cpp

mruberry reviewed Apr 7, 2021

View reviewed changes

Comment thread aten/src/ATen/native/quantized/fake_quant_per_tensor_affine.cpp Outdated

mruberry reviewed Apr 7, 2021

View reviewed changes

peterbell10 requested a review from mruberry April 8, 2021 23:27

malfet requested changes Apr 12, 2021

View reviewed changes

peterbell10 added 2 commits April 13, 2021 15:45

Update on "Vectorize copysign on CPU"

8830ee5

[ghstack-poisoned]

Update on "Vectorize copysign on CPU"

cc2a5eb

[ghstack-poisoned]

mruberry reviewed Apr 14, 2021

View reviewed changes

ezyang approved these changes Apr 14, 2021

View reviewed changes

ezyang self-requested a review April 14, 2021 15:40

malfet approved these changes Apr 14, 2021

View reviewed changes

mruberry self-requested a review April 14, 2021 17:38

mruberry approved these changes Apr 14, 2021

View reviewed changes

facebook-github-bot closed this in 8c74e1b Apr 18, 2021

facebook-github-bot added the Merged label Apr 18, 2021

dncliss mentioned this pull request Apr 20, 2021

New build failure after vectorization of copysign #56474

Closed

facebook-github-bot deleted the gh/peterbell10/49/head branch April 21, 2021 14:15

Conversation

peterbell10 commented Feb 5, 2021 • edited by mruberry Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facebook-github-bot commented Feb 5, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CI failures summary and remediations

🕵️ 1 new failure recognized by patterns

pytorch_macos_10_13_py3_test (1/1)

❄️ 1 failure tentatively classified as flaky

pytorch_linux_bionic_py3_8_gcc9_coverage_test2 (1/1)

Uh oh!

Uh oh!

Uh oh!

mruberry Apr 7, 2021

Choose a reason for hiding this comment

Uh oh!

peterbell10 Apr 13, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

mruberry Apr 7, 2021

Choose a reason for hiding this comment

Uh oh!

ngimel Apr 12, 2021

Choose a reason for hiding this comment

Uh oh!

peterbell10 Apr 13, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

malfet left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

malfet Apr 12, 2021

Choose a reason for hiding this comment

Uh oh!

malfet Apr 12, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

peterbell10 commented Apr 13, 2021

Uh oh!

mruberry left a comment

Choose a reason for hiding this comment

Uh oh!

ezyang commented Apr 14, 2021

Uh oh!

malfet commented Apr 14, 2021

Uh oh!

mruberry commented Apr 14, 2021

Uh oh!

facebook-github-bot commented Apr 18, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

peterbell10 commented Feb 5, 2021 •

edited by mruberry

Loading

facebook-github-bot commented Feb 5, 2021 •

edited

Loading

peterbell10 Apr 13, 2021 •

edited

Loading