[quantization] Store bias in PackedLinearWeight struct in fbgemm by supriyar · Pull Request #25428 · pytorch/pytorch

supriyar · 2019-08-29T21:29:46Z

Stack from ghstack:

[quantization] Store bias in PackedConvWeight in fbgemm #25626 [quantization] Store bias in PackedConvWeight in fbgemm
[quantization] Store bias in PackedLinearWeight struct in fbgemm #25428 [quantization] Store bias in PackedLinearWeight struct in fbgemm
[quantization] Rename FBGEMM quantized operators to generic quantized ops #25678 [quantization] Rename FBGEMM quantized operators to generic quantized ops

Added bias as an optional param to the quantized_linear_prepack function.
Bias is quantized during runtime using input scale and weight scale.

Differential Revision: D17121304

Added bias as an optional param to the quantized_linear_prepack function. Bias is quantized during runtime using input scale and weight scale. Differential Revision: [D17121304](https://our.internmc.facebook.com/intern/diff/D17121304/)

…fbgemm" Added bias as an optional param to the quantized_linear_prepack function. Bias is quantized during runtime using input scale and weight scale. Differential Revision: [D17121304](https://our.internmc.facebook.com/intern/diff/D17121304/)

Pull Request resolved: #25428 Added bias as an optional param to the quantized_linear_prepack function. Bias is quantized during runtime using input scale and weight scale. ghstack-source-id: 89269220 Differential Revision: [D17121304](https://our.internmc.facebook.com/intern/diff/D17121304/)

torch/nn/quantized/modules/linear.py

aten/src/ATen/native/quantized/cpu/qlinear.cpp

aten/src/ATen/native/quantized/cpu/qlinear_prepack.cpp

torch/nn/quantized/modules/linear.py

aten/src/ATen/native/quantized/cpu/qlinear.cpp

test/test_jit.py

…fbgemm" Added bias as an optional param to the quantized_linear_prepack function. Bias is quantized during runtime using input scale and weight scale. Differential Revision: [D17121304](https://our.internmc.facebook.com/intern/diff/D17121304/)

jamesr66a

ROCM failure is true-positive. The function signature at qlinear.cpp:218 needs to be updated

…fbgemm" Added bias as an optional param to the quantized_linear_prepack function. Bias is quantized during runtime using input scale and weight scale. Differential Revision: [D17121304](https://our.internmc.facebook.com/intern/diff/D17121304/)

Pull Request resolved: #25428 Added bias as an optional param to the quantized_linear_prepack function. Bias is quantized during runtime using input scale and weight scale. ghstack-source-id: 89329368 Differential Revision: [D17121304](https://our.internmc.facebook.com/intern/diff/D17121304/)

aten/src/ATen/native/quantized/cpu/qlinear.cpp

torch/nn/quantized/modules/linear.py

aten/src/ATen/native/quantized/cpu/qlinear.cpp

aten/src/ATen/native/quantized/cpu/fbgemm_utils.h

aten/src/ATen/native/quantized/cpu/qlinear.cpp

dskhudia · 2019-09-04T16:21:54Z

I have approved it. Please take care of the tests.

aten/src/ATen/native/quantized/cpu/qlinear.cpp

…fbgemm" Added bias as an optional param to the quantized_linear_prepack function. Bias is quantized during runtime using input scale and weight scale. Differential Revision: [D17121304](https://our.internmc.facebook.com/intern/diff/D17121304/)

raghuramank100 · 2019-09-04T22:57:41Z

aten/src/ATen/native/quantized/cpu/qlinear_prepack.cpp

            /*ld=*/K,
            /*pmat=*/nullptr, // PackBMatrix manages ownership of pmat
            /*groups=*/1),
+        bias_contig,


How does this work when bias is None?

It is stored as an optional tensor - similar to the current linear op that has bias as an input argument. So if it is none then that gets taken care of.

torch/nn/quantized/modules/linear.py

raghuramank100 · 2019-09-04T23:05:07Z

We also need to make similar changes to: https://github.com/pytorch/pytorch/blob/9d06a984f866289c2acb28a379f62378c5e70454/torch/nn/quantized/functional.py

raghuramank100

Looks great, a few suggested changes.

…fbgemm" Added bias as an optional param to the quantized_linear_prepack function. Bias is quantized during runtime using input scale and weight scale. Differential Revision: [D17121304](https://our.internmc.facebook.com/intern/diff/D17121304/)

supriyar · 2019-09-06T00:37:42Z

Removed self.bias from the modules. Added bias() and weight() functions. Also renamed and updated the API for fbgemm_linear_dynamic
cc @jianyuh @raghuramank100

…fbgemm" Added bias as an optional param to the quantized_linear_prepack function. Bias is quantized during runtime using input scale and weight scale. Differential Revision: [D17121304](https://our.internmc.facebook.com/intern/diff/D17121304/)

jianyuh · 2019-09-06T01:46:51Z

Removed self.bias from the modules. Added bias() and weight() functions. Also renamed and updated the API for fbgemm_linear_dynamic
cc @jianyuh @raghuramank100

The updated API for fbgemm_linear_dynamic looks good to me!

facebook-github-bot · 2019-09-06T16:05:07Z

This pull request has been merged in 9d2d31e.

Summary: Pull Request resolved: pytorch/pytorch#25428 Added bias as an optional param to the quantized_linear_prepack function. Bias is quantized during runtime using input scale and weight scale. ghstack-source-id: 89601399 Test Plan: python test/run_test.py --exclude nn --verbose --bring-to-front quantization quantized quantized_tensor quantized_nn_mods quantizer Differential Revision: D17121304 fbshipit-source-id: 8adb0e55e4aed0a5430aaa2c8639c8ad1639c85a

supriyar requested a review from apaszke as a code owner August 29, 2019 21:29

pytorchbot added oncall: jit Add this issue/PR to JIT oncall triage queue module: nn Related to torch.nn module: operators oncall: quantization Quantization support in PyTorch labels Aug 29, 2019

supriyar mentioned this pull request Aug 29, 2019

[quantization] Rename fbgemm quantized operators to generic quantized ops #25338

Closed

supriyar requested review from dskhudia, dzhulgakov and jamesr66a August 29, 2019 21:31

supriyar mentioned this pull request Aug 29, 2019

Skip test_compare_tensor_scalar due to overflow error #25432

Closed

This comment has been minimized.

Sign in to view

dzhulgakov requested changes Aug 30, 2019

View reviewed changes

jianyuh reviewed Aug 30, 2019

View reviewed changes

test/test_jit.py Outdated Show resolved Hide resolved

jamesr66a requested changes Aug 30, 2019

View reviewed changes

supriyar requested review from dzhulgakov and jamesr66a August 30, 2019 22:52

dzhulgakov reviewed Sep 3, 2019

View reviewed changes

aten/src/ATen/native/quantized/cpu/qlinear.cpp Show resolved Hide resolved

aten/src/ATen/native/quantized/cpu/qlinear.cpp Outdated Show resolved Hide resolved

torch/nn/quantized/modules/linear.py Show resolved Hide resolved

torch/nn/quantized/modules/linear.py Outdated Show resolved Hide resolved

dskhudia reviewed Sep 3, 2019

View reviewed changes

aten/src/ATen/native/quantized/cpu/qlinear.cpp Show resolved Hide resolved

aten/src/ATen/native/quantized/cpu/qlinear.cpp Show resolved Hide resolved

dskhudia approved these changes Sep 4, 2019

View reviewed changes

aten/src/ATen/native/quantized/cpu/fbgemm_utils.h Show resolved Hide resolved

aten/src/ATen/native/quantized/cpu/qlinear.cpp Show resolved Hide resolved

dskhudia reviewed Sep 4, 2019

View reviewed changes

aten/src/ATen/native/quantized/cpu/qlinear.cpp Show resolved Hide resolved

raghuramank100 reviewed Sep 4, 2019

View reviewed changes

torch/nn/quantized/modules/linear.py Outdated Show resolved Hide resolved

raghuramank100 reviewed Sep 4, 2019

View reviewed changes

torch/nn/quantized/modules/linear.py Show resolved Hide resolved

raghuramank100 reviewed Sep 4, 2019

View reviewed changes

torch/nn/quantized/modules/linear.py Outdated Show resolved Hide resolved

raghuramank100 suggested changes Sep 4, 2019

View reviewed changes

supriyar mentioned this pull request Sep 4, 2019

[quantization] Rename FBGEMM quantized operators to generic quantized ops #25678

Closed

supriyar requested a review from raghuramank100 September 5, 2019 00:40

supriyar mentioned this pull request Sep 5, 2019

[WIP] Temp commit to add bias() method #25741

Closed

supriyar requested a review from jianyuh September 6, 2019 00:36

jianyuh approved these changes Sep 6, 2019

View reviewed changes

facebook-github-bot closed this in 9d2d31e Sep 6, 2019

facebook-github-bot added the merged label Sep 6, 2019

facebook-github-bot deleted the gh/supriyar/7/head branch October 28, 2019 22:21

mruberry added the Merged label Oct 28, 2020

Conversation

supriyar commented Aug 29, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jamesr66a left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dskhudia commented Sep 4, 2019

Uh oh!

Uh oh!

raghuramank100 Sep 4, 2019

Choose a reason for hiding this comment

Uh oh!

supriyar Sep 4, 2019

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

raghuramank100 commented Sep 4, 2019

Uh oh!

raghuramank100 left a comment

Choose a reason for hiding this comment

Uh oh!

supriyar commented Sep 6, 2019

Uh oh!

jianyuh commented Sep 6, 2019

Uh oh!

facebook-github-bot commented Sep 6, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

supriyar commented Aug 29, 2019 •

edited

Loading