Skip to content

[Quant][PT2E][X86] Enable annotation of aten.mul.tensor with X86InductorQuantizer#150831

Closed
Xia-Weiwen wants to merge 5 commits into
gh/Xia-Weiwen/36/basefrom
gh/Xia-Weiwen/36/head
Closed

[Quant][PT2E][X86] Enable annotation of aten.mul.tensor with X86InductorQuantizer#150831
Xia-Weiwen wants to merge 5 commits into
gh/Xia-Weiwen/36/basefrom
gh/Xia-Weiwen/36/head

Conversation

@Xia-Weiwen

@Xia-Weiwen Xia-Weiwen commented Apr 8, 2025 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Stack from ghstack (oldest at bottom):

Summary
This PR adds support of annotation of aten.mul.tensor in X86InductorQuantizer.
mul is not annotated by default. Users need to set the following to enable annotation of mul:

quantizer.set_function_type_qconfig(
    torch.mul, quantizer.get_global_quantization_config()
)

After convert_pt2e, users get patterns like

quantize_per_tensor_default = torch.ops.quantized_decomposed.quantize_per_tensor.default(x, ...)
dequantize_per_tensor_default = torch.ops.quantized_decomposed.dequantize_per_tensor.default(quantize_per_tensor_default, ...)
quantize_per_tensor_default_1 = torch.ops.quantized_decomposed.quantize_per_tensor.default(y, ...);
dequantize_per_tensor_default_1 = torch.ops.quantized_decomposed.dequantize_per_tensor.default(quantize_per_tensor_default_1, ...)
mul = torch.ops.aten.mul.Tensor(dequantize_per_tensor_default, dequantize_per_tensor_default_1);

Test plan

pytest test/quantization/pt2e/test_x86inductor_quantizer.py -k test_annotate_mul_tensor

cc @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10

@Xia-Weiwen
Update
9614697 
[ghstack-poisoned]
@Xia-Weiwen Xia-Weiwen requested a review from jerryzh168 as a code owner April 8, 2025 07:37
@pytorch-bot

pytorch-bot Bot commented Apr 8, 2025 •
edited
Loading

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/150831

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 71e079b with merge base 01f226b (image):

UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@Xia-Weiwen Xia-Weiwen mentioned this pull request Apr 8, 2025
Closed
@pytorch-bot pytorch-bot Bot added the release notes: quantization release notes category label Apr 8, 2025
Xia-Weiwen added a commit that referenced this pull request Apr 8, 2025
@Xia-Weiwen
[Quant][PT2E][X86] Enable annotation of aten.mul.tensor with X86Induc…
b818f27 
…torQuantizer

ghstack-source-id: b336298
Pull Request resolved: #150831
@Xia-Weiwen Xia-Weiwen marked this pull request as draft April 8, 2025 07:38
Copilot AI reviewed Apr 8, 2025
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

Comments suppressed due to low confidence (1)

test/quantization/pt2e/test_x86inductor_quantizer.py:2877

  • [nitpick] Avoid using 'type' as a variable name because it shadows the built-in function. Consider renaming it to 'model_type' or a similar descriptive name.
for type in [0, 1, 2]:

@Xia-Weiwen Xia-Weiwen added the intel This tag is for PR from Intel label Apr 8, 2025
@Xia-Weiwen
Update
9d31e46 
[ghstack-poisoned]
Xia-Weiwen added a commit that referenced this pull request Apr 9, 2025
@Xia-Weiwen
[Quant][PT2E][X86] Enable annotation of aten.mul.tensor with X86Induc…
b0f7d64 
…torQuantizer

ghstack-source-id: 23478a7
Pull Request resolved: #150831
leslie-fang-intel
leslie-fang-intel approved these changes Apr 9, 2025
View reviewed changes

@leslie-fang-intel leslie-fang-intel left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. A small comment.

Comment thread test/quantization/pt2e/test_x86inductor_quantizer.py Outdated
@Xia-Weiwen Xia-Weiwen marked this pull request as ready for review April 10, 2025 02:29
@Xia-Weiwen

Xia-Weiwen commented Apr 11, 2025

Copy link
Copy Markdown
Collaborator Author

Hi @jerryzh168 Could you please review this PR? Thanks.

jerryzh168
jerryzh168 reviewed Apr 11, 2025
View reviewed changes
Comment thread test/quantization/pt2e/test_x86inductor_quantizer.py Outdated
else:
return x * y.sum().item()

for type in [0, 1, 2, 3]:

@jerryzh168 jerryzh168 Apr 11, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I think we can improve [0, 1, 2, 3] by defining different classes for each test case here

@Xia-Weiwen Xia-Weiwen Apr 11, 2025

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Updated.

@Xia-Weiwen
Update
a83b7c3 
[ghstack-poisoned]
@Xia-Weiwen Xia-Weiwen mentioned this pull request Apr 11, 2025
Closed
@Xia-Weiwen
Update
a52dfd2 
[ghstack-poisoned]
@Xia-Weiwen
Update
71e079b 
[ghstack-poisoned]
Xia-Weiwen added a commit that referenced this pull request Apr 14, 2025
@Xia-Weiwen
[Quant][PT2E][X86] Enable annotation of aten.mul.tensor with X86Induc…
3b72757 
…torQuantizer

ghstack-source-id: dbd0974
Pull Request resolved: #150831
@Xia-Weiwen

Xia-Weiwen commented Apr 18, 2025

Copy link
Copy Markdown
Collaborator Author

Close this PR as we need to move to Torchao.

@Xia-Weiwen Xia-Weiwen closed this Apr 18, 2025
@Xia-Weiwen Xia-Weiwen mentioned this pull request Apr 18, 2025
Merged
@github-actions github-actions Bot deleted the gh/Xia-Weiwen/36/head branch May 25, 2025 02:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@leslie-fang-intel leslie-fang-intel leslie-fang-intel approved these changes

Copilot code review Copilot Copilot left review comments

@jerryzh168 jerryzh168 jerryzh168 approved these changes

Assignees

No one assigned

Labels

intel This tag is for PR from Intel open source release notes: quantization release notes category

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@Xia-Weiwen @jerryzh168 @leslie-fang-intel @pytorchbot