Skip to content

Fix the QuantizedAVX2 build issue#26854

Closed
lly-zero-one wants to merge 4 commits intopytorch:masterfrom
lly-zero-one:fix_avx2
Closed

Fix the QuantizedAVX2 build issue#26854
lly-zero-one wants to merge 4 commits intopytorch:masterfrom
lly-zero-one:fix_avx2

Conversation

@lly-zero-one
Copy link
Copy Markdown
Contributor

@lly-zero-one lly-zero-one commented Sep 26, 2019

The QuantizedAVx2 does not support the int32 type. We switch to use at::quantize_vec function instead.

@pytorchbot pytorchbot added module: operators oncall: quantization Quantization support in PyTorch labels Sep 26, 2019
lly-zero-one
lly-zero-one commented Sep 26, 2019
View reviewed changes
Comment thread aten/src/ATen/native/quantized/cpu/kernels/QuantizedOpKernels.cpp
@jamesr66a
Copy link
Copy Markdown
Collaborator

jamesr66a commented Sep 26, 2019

Any perf checks?

facebook-github-bot
facebook-github-bot reviewed Sep 26, 2019
View reviewed changes
Copy link
Copy Markdown
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@llyfacebook has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@lly-zero-one
Copy link
Copy Markdown
Contributor Author

lly-zero-one commented Sep 26, 2019

Any perf checks?

Yes, I found it is even faster than the original one. (For bilinear2d interpolate case.)

**** torch.qint8 *****
/home/lingyiliu/local/anaconda3/lib/python3.7/site-packages/torch/nn/functional.py:2494: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
  "See the documentation of nn.Upsample for details.".format(mode))
time/iter ms (float)    time/iter ms (quant)    quant/float
2.776181697845459       0.03426074981689453     0.012340960911702442
GB/s float      GB/s quant
1.165940976598206       23.619331285066114
**** torch.quint8 *****
time/iter ms (float)    time/iter ms (quant)    quant/float
2.8192615509033203      0.032978057861328125    0.011697409859245451
GB/s float      GB/s quant
1.1481247630121 24.538012620474262
**** torch.qint32 *****
time/iter ms (float)    time/iter ms (quant)    quant/float
2.0195364952087402      1.0663747787475586      0.5280294667996766
GB/s float      GB/s quant
1.602775690203824       3.0353906192358084

@hx89
Copy link
Copy Markdown

hx89 commented Sep 26, 2019

I got some undeclared identifier errors when build locally:
stderr: caffe2/aten/src/ATen/native/quantized/cpu/kernels/QuantizedOpKernels.cpp:654:523: error: use of undeclared identifier 'area_pixel_compute_s
cale'
caffe2/aten/src/ATen/native/quantized/cpu/kernels/QuantizedOpKernels.cpp:654:553: error: expected '(' for function-style cast or type construction
caffe2/aten/src/ATen/native/quantized/cpu/kernels/QuantizedOpKernels.cpp:654:621: error: use of undeclared identifier 'area_pixel_compute_scale'

@jamesr66a
Copy link
Copy Markdown
Collaborator

jamesr66a commented Sep 26, 2019

@llyfacebook yeah, QuantizeAVX2 operates on vectors of 32, but you were feeding it vectors of 8, so it was just running scalar code :p

@jamesr66a
Copy link
Copy Markdown
Collaborator

jamesr66a commented Sep 26, 2019

@llyfacebook I'd expect another 2x speedup if you switch to doing the float operations 4-wide and using QuantizeAVX2 again

hx89
hx89 approved these changes Sep 27, 2019
View reviewed changes
Copy link
Copy Markdown

@hx89 hx89 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! My local build pass after rebasing.

facebook-github-bot
facebook-github-bot reviewed Sep 27, 2019
View reviewed changes
Copy link
Copy Markdown
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@llyfacebook is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Copy Markdown
Contributor

facebook-github-bot commented Sep 27, 2019

@llyfacebook merged this pull request in 428204d.

zdevito pushed a commit to zdevito/ATen that referenced this pull request Sep 27, 2019
@lly-zero-one @facebook-github-bot
Fix the QuantizedAVX2 build issue (#26854)
6dc2a13 
Summary:
The QuantizedAVx2 does not support the int32 type. We switch to use at::quantize_vec function instead.
Pull Request resolved: pytorch/pytorch#26854

Differential Revision: D17609872

Pulled By: llyfacebook

fbshipit-source-id: b4a77d93ce0ebfef696506b5cdbe3e91fe44bb36
pdlive215 pushed a commit to pdlive215/pytorch that referenced this pull request Nov 27, 2019
@lly-zero-one @pdlive215
Fix the QuantizedAVX2 build issue (pytorch#26854)
ada16d3 
Summary:
The QuantizedAVx2 does not support the int32 type. We switch to use at::quantize_vec function instead.
Pull Request resolved: pytorch#26854

Differential Revision: D17609872

Pulled By: llyfacebook

fbshipit-source-id: b4a77d93ce0ebfef696506b5cdbe3e91fe44bb36
laurentdupin pushed a commit to laurentdupin/pytorch that referenced this pull request Apr 24, 2026
@lly-zero-one @facebook-github-bot
Fix the QuantizedAVX2 build issue (pytorch#26854)
6c03216 
Summary:
The QuantizedAVx2 does not support the int32 type. We switch to use at::quantize_vec function instead.
Pull Request resolved: pytorch#26854

Differential Revision: D17609872

Pulled By: llyfacebook

fbshipit-source-id: b4a77d93ce0ebfef696506b5cdbe3e91fe44bb36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jamesr66a jamesr66a Awaiting requested review from jamesr66a

2 more reviewers

@facebook-github-bot facebook-github-bot facebook-github-bot left review comments

@hx89 hx89 hx89 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

Merged oncall: quantization Quantization support in PyTorch

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@lly-zero-one @jamesr66a @hx89 @facebook-github-bot @pytorchbot @mruberry