adds quantized implementation of hard sigmoid by vkuzo · Pull Request #34607 · pytorch/pytorch

vkuzo · 2020-03-11T20:00:56Z

Stack from ghstack:

adds quantized implementation of hard sigmoid #34607 adds quantized implementation of hard sigmoid

Summary:

Adds quantized version of hardsigmoid activation.

Note: not implementing the _ and .out versions is
currently intended, because the implementation changes the scale and
zp and it's nice to not allow the user to specify scale
and zp. Lmk if we should handle this differently.

Test Plan:

tests
benchmarks

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: D20480546

Summary: Adds quantized version of hardsigmoid activation. Note: not implementing the _ and .out versions is currently intended, because the implementation changes the scale and zp and it's nice to not allow the user to specify scale and zp. Lmk if we should handle this differently. Test Plan: tests benchmarks Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

Summary: Adds quantized version of hardsigmoid activation. Note: not implementing the _ and .out versions is currently intended, because the implementation changes the scale and zp and it's nice to not allow the user to specify scale and zp. Lmk if we should handle this differently. Test Plan: tests benchmarks Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: e19f098 Pull Request resolved: #34607

dr-ci · 2020-03-11T21:14:18Z

💊 CircleCI build failures summary and remediations

As of commit 5fee38c (more details on the Dr. CI page):

✅ None of the build failures appear to be your fault 💚

1/1 broken upstream at merge base ae0c88d since Mar 16
Please rebase on the viable/strict branch (expand for instructions)

If your commit is newer than viable/strict, you can try basing on an older, stable commit:
```
git fetch https://github.com/pytorch/pytorch viable/strict
git rebase --onto FETCH_HEAD $(git merge-base origin/master HEAD)
```
If your commit is older than viable/strict:
```
git fetch https://github.com/pytorch/pytorch viable/strict
git rebase FETCH_HEAD
```
Check out the recency history of this "viable master" tracking branch.

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker.

This comment has been revised 44 times.

supriyar · 2020-03-12T03:55:07Z

LGTM, do we have some perf numbers as well for this?
I wonder if it would be possible to get some benchmark numbers on mobile as well for the elementwise operators you've implemented so far. Vec256 wouldn't give any speedup on mobile and it would be nice to track what the perf is on mobile as well for future optimizations.

Summary: Adds quantized version of hardsigmoid activation. Note: not implementing the _ and .out versions is currently intended, because the implementation changes the scale and zp and it's nice to not allow the user to specify scale and zp. Lmk if we should handle this differently. Test Plan: tests benchmarks Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

Summary: Adds quantized version of hardsigmoid activation. Note: not implementing the _ and .out versions is currently intended, because the implementation changes the scale and zp and it's nice to not allow the user to specify scale and zp. Lmk if we should handle this differently. Test Plan: tests benchmarks Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: d476af7 Pull Request resolved: #34607

vkuzo · 2020-03-12T17:31:19Z

LGTM, do we have some perf numbers as well for this?
I wonder if it would be possible to get some benchmark numbers on mobile as well for the elementwise operators you've implemented so far. Vec256 wouldn't give any speedup on mobile and it would be nice to track what the perf is on mobile as well for future optimizations.

sure thing, here is the output of the current benchmark: https://our.internmc.facebook.com/intern/paste/P127415136/

I will follow up offline on the best way to benchmark all of these things on mobile!

supriyar

Thanks for the benchmarks! I usually also compare against float to see the speedup.
But looks good overall

Summary: Adds quantized version of hardsigmoid activation. Note: not implementing the _ and .out versions is currently intended, because the implementation changes the scale and zp and it's nice to not allow the user to specify scale and zp. Lmk if we should handle this differently. Test Plan: tests benchmarks Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

Summary: Adds quantized version of hardsigmoid activation. Note: not implementing the _ and .out versions is currently intended, because the implementation changes the scale and zp and it's nice to not allow the user to specify scale and zp. Lmk if we should handle this differently. Test Plan: tests benchmarks Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 864c47a Pull Request resolved: #34607

Summary: Adds quantized version of hardsigmoid activation. Note: not implementing the _ and .out versions is currently intended, because the implementation changes the scale and zp and it's nice to not allow the user to specify scale and zp. Lmk if we should handle this differently. Test Plan: tests benchmarks Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D20480546](https://our.internmc.facebook.com/intern/diff/D20480546) [ghstack-poisoned]

Summary: Adds quantized version of hardsigmoid activation. Note: not implementing the _ and .out versions is currently intended, because the implementation changes the scale and zp and it's nice to not allow the user to specify scale and zp. Lmk if we should handle this differently. Test Plan: tests benchmarks Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 8a3fbbf Pull Request resolved: #34607

Summary: Adds quantized version of hardsigmoid activation. Note: not implementing the _ and .out versions is currently intended, because the implementation changes the scale and zp and it's nice to not allow the user to specify scale and zp. Lmk if we should handle this differently. Test Plan: tests benchmarks Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D20480546](https://our.internmc.facebook.com/intern/diff/D20480546) [ghstack-poisoned]

Summary: Adds quantized version of hardsigmoid activation. Note: not implementing the _ and .out versions is currently intended, because the implementation changes the scale and zp and it's nice to not allow the user to specify scale and zp. Lmk if we should handle this differently. Test Plan: tests benchmarks Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 6ddd123 Pull Request resolved: #34607

facebook-github-bot · 2020-03-18T00:21:20Z

This pull request has been merged in 58c5b6d.

mrshenli · 2020-03-18T02:10:16Z

Looks like this PR breaks master:

Mar 18 00:56:58 ======================================================================
Mar 18 00:56:58 FAIL [0.974s]: test_qhardsigmoid (__main__.TestQuantizedOps)
Mar 18 00:56:58 ----------------------------------------------------------------------
Mar 18 00:56:58 Traceback (most recent call last):
Mar 18 00:56:58   File "test_quantized.py", line 244, in test_qhardsigmoid
Mar 18 00:56:58     elements=hu.floats(-1e3, 1e3, allow_nan=False, allow_infinity=False),
Mar 18 00:56:58   File "/var/lib/jenkins/.local/lib/python3.6/site-packages/hypothesis/core.py", line 1116, in wrapped_test
Mar 18 00:56:58     raise the_error_hypothesis_found
Mar 18 00:56:58   File "test_quantized.py", line 271, in test_qhardsigmoid
Mar 18 00:56:58     message="Hardsigmoid failed: {} vs. {}".format(qY, qY_hat))
Mar 18 00:56:58   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_utils.py", line 880, in assertEqual
Mar 18 00:56:58     exact_dtype=exact_dtype)
Mar 18 00:56:58   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_utils.py", line 882, in assertEqual
Mar 18 00:56:58     assertTensorsEqual(x, y)
Mar 18 00:56:58   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_utils.py", line 848, in assertTensorsEqual
Mar 18 00:56:58     self.assertLessEqual(max_err, prec, message)
Mar 18 00:56:58 AssertionError: tensor(1, dtype=torch.int32) not less than or equal to 1e-05 : Hardsigmoid failed: tensor([0.5000], size=(1,), dtype=torch.qint32,
Mar 18 00:56:58        quantization_scheme=torch.per_tensor_affine, scale=2.3283064365386963e-10,
Mar 18 00:56:58        zero_point=0) vs. tensor([-0.5000], size=(1,), dtype=torch.qint32,
Mar 18 00:56:58        quantization_scheme=torch.per_tensor_affine, scale=2.3283064365386963e-10,
Mar 18 00:56:58        zero_point=0)

e.g.:

https://app.circleci.com/pipelines/github/pytorch/pytorch/141312/workflows/9036f70f-a41d-4968-aedc-4cab6b4307f0/jobs/4849783
https://app.circleci.com/pipelines/github/pytorch/pytorch/141312/workflows/9036f70f-a41d-4968-aedc-4cab6b4307f0/jobs/4849602

mrshenli · 2020-03-18T02:53:47Z

@vkuzo I am reverting this PR to fix test signals on master. Please resubmit with a fix. Thanks!

vkuzo · 2020-03-18T03:56:11Z

@vkuzo I am reverting this PR to fix test signals on master. Please resubmit with a fix. Thanks!

sorry about that, will do!

Summary: Adds quantized implementation of hardsigmoid. Original PR was #34607 and had to be reverted for a test breakage, trying again. Test Plan: tests benchmarks Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

Summary: Adds quantized implementation of hardsigmoid. Original PR was #34607 and had to be reverted for a test breakage, trying again. Test Plan: tests benchmarks Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: d58ebb5 Pull Request resolved: #34959

Summary: Adds quantized implementation of hardsigmoid. Original PR was #34607 and had to be reverted for a test breakage, trying again. Test Plan: tests benchmarks Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D20514212](https://our.internmc.facebook.com/intern/diff/D20514212) [ghstack-poisoned]

Summary: Adds quantized implementation of hardsigmoid. Original PR was #34607 and had to be reverted for a test breakage, trying again. Test Plan: tests benchmarks Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 66b9504 Pull Request resolved: #34959

Summary: Adds quantized implementation of hardsigmoid. Original PR was #34607 and had to be reverted for a test breakage, trying again. Test Plan: tests benchmarks Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D20514212](https://our.internmc.facebook.com/intern/diff/D20514212) [ghstack-poisoned]

Summary: Adds quantized implementation of hardsigmoid. Original PR was #34607 and had to be reverted for a test breakage, trying again. Test Plan: tests benchmarks Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 6dcd01c Pull Request resolved: #34959

Summary: Pull Request resolved: #34959 Adds quantized implementation of hardsigmoid. Original PR was #34607 and had to be reverted for a test breakage, trying again. Test Plan: tests benchmarks Imported from OSS Differential Revision: D20514212 fbshipit-source-id: cc7ae3b67757e2dde5c313c05ce60a0f2625d961

Summary: Pull Request resolved: pytorch#34607 Adds quantized version of hardsigmoid activation. Note: not implementing the _ and .out versions is currently intended, because the implementation changes the scale and zp and it's nice to not allow the user to specify scale and zp. Lmk if we should handle this differently. Test Plan: tests benchmarks Imported from OSS Differential Revision: D20480546 fbshipit-source-id: 9febcb44afd920125ed2ca4900492f0b712078ea

Summary: Pull Request resolved: pytorch#34959 Adds quantized implementation of hardsigmoid. Original PR was pytorch#34607 and had to be reverted for a test breakage, trying again. Test Plan: tests benchmarks Imported from OSS Differential Revision: D20514212 fbshipit-source-id: cc7ae3b67757e2dde5c313c05ce60a0f2625d961

vkuzo requested a review from apaszke as a code owner March 11, 2020 20:00

vkuzo mentioned this pull request Mar 11, 2020

add hardsigmoid FP operator to PyTorch #34545

Closed

vkuzo requested review from jerryzh168, raghuramank100, supriyar and z-a-f March 11, 2020 20:02

vkuzo self-assigned this Mar 11, 2020

vkuzo added the oncall: quantization Quantization support in PyTorch label Mar 11, 2020

supriyar reviewed Mar 12, 2020

View reviewed changes

Comment thread aten/src/ATen/native/quantized/cpu/kernels/QuantizedOpKernels.cpp Outdated

jamesr66a reviewed Mar 12, 2020

View reviewed changes

Comment thread aten/src/ATen/native/quantized/cpu/kernels/QuantizedOpKernels.cpp Outdated

supriyar approved these changes Mar 12, 2020

View reviewed changes

facebook-github-bot closed this in 58c5b6d Mar 17, 2020

facebook-github-bot added the merged label Mar 18, 2020

vkuzo mentioned this pull request Mar 18, 2020

quantized hardsigmoid, take 2 #34959

Closed

facebook-github-bot deleted the gh/vkuzo/7/head branch March 21, 2020 14:17

mruberry added the Merged label Oct 28, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

adds quantized implementation of hard sigmoid#34607

adds quantized implementation of hard sigmoid#34607
vkuzo wants to merge 5 commits intogh/vkuzo/7/basefrom
gh/vkuzo/7/head

vkuzo commented Mar 11, 2020 •

edited

Loading

Uh oh!

dr-ci Bot commented Mar 11, 2020 •

edited

Loading

Uh oh!

Uh oh!

supriyar commented Mar 12, 2020

Uh oh!

Uh oh!

vkuzo commented Mar 12, 2020

Uh oh!

supriyar left a comment

Uh oh!

facebook-github-bot commented Mar 18, 2020

Uh oh!

mrshenli commented Mar 18, 2020

Uh oh!

mrshenli commented Mar 18, 2020

Uh oh!

vkuzo commented Mar 18, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

vkuzo commented Mar 11, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dr-ci Bot commented Mar 11, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CircleCI build failures summary and remediations

Uh oh!

Uh oh!

supriyar commented Mar 12, 2020

Uh oh!

Uh oh!

vkuzo commented Mar 12, 2020

Uh oh!

supriyar left a comment

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Mar 18, 2020

Uh oh!

mrshenli commented Mar 18, 2020

Uh oh!

mrshenli commented Mar 18, 2020

Uh oh!

vkuzo commented Mar 18, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

vkuzo commented Mar 11, 2020 •

edited

Loading

dr-ci Bot commented Mar 11, 2020 •

edited

Loading