Reference representation of dqlinear int4 for xnnpack by kimishpatel · Pull Request #2520 · pytorch/ao

kimishpatel · 2025-07-10T18:33:14Z

Stack from ghstack (oldest at bottom):

-> Reference representation of dqlinear int4 for xnnpack #2520

Summary:
This diff adds dynamic quantized linear's integer arithmetic
representation. This is quite close to how arithmetic is done in
xnnpack.

Basic tests added against q/dq to make things are sane.

Followups:

See if such a graph is traceable.
Optimize implementation if needed

Test Plan:
added

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: D78198154

tuples Summary: THis is needed because lists are not hashable, since they are mutable, and as a result we cannot have literals_to_ph in pattern rewrites used inside reference_representation_rewrite.py Test Plan: CI + next diff relies on this feature Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

Summary: This is necessary because sometimes the patterns found have literals include tuple of ints kind of literals. This values shouldnt be used for pattern matching since often they are based on consts derived from example inputs. THis is not exactly a safe thing to do in general so by default it is turned off Test Plan: Subsequent diff adds a pattern that relies on this Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

pytorch-bot · 2025-07-10T18:33:18Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/2520

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit ae63634 with merge base fe0ddf1 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 756a9e9 Pull Request resolved: #2520

jerryzh168 · 2025-07-10T23:33:11Z

+    "_qdq_dynamic_quantized_linear_4bit_groupwise",
+    "_reference_dynamic_quantized_linear_4bit_groupwise",


why does these needs to be exposed?

oh good point. it doesnt. ai assisted coding i guess. lol

…for xnnpack" Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 0f79f1c Pull Request resolved: #2520

…for xnnpack" Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 080923e Pull Request resolved: #2520

…for xnnpack" Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: deb3efa Pull Request resolved: #2520

jerryzh168

I'll stamp to unblock, but let me know if any review is needed

kimishpatel · 2025-07-11T23:29:55Z

@kimishpatel has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

…for xnnpack" Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D78198154](https://our.internmc.facebook.com/intern/diff/D78198154) [ghstack-poisoned]

Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D78198154](https://our.internmc.facebook.com/intern/diff/D78198154) [ghstack-poisoned]

Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 5108e2c Pull Request resolved: #2520

…for xnnpack" Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D78198154](https://our.internmc.facebook.com/intern/diff/D78198154) [ghstack-poisoned]

Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D78198154](https://our.internmc.facebook.com/intern/diff/D78198154) [ghstack-poisoned]

Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: a1e2796 Pull Request resolved: #2520

…for xnnpack" Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D78198154](https://our.internmc.facebook.com/intern/diff/D78198154) [ghstack-poisoned]

Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D78198154](https://our.internmc.facebook.com/intern/diff/D78198154) [ghstack-poisoned]

Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: f9db619 Pull Request resolved: #2520

kimishpatel · 2025-08-07T19:44:57Z

@kimishpatel has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

…for xnnpack" Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D78198154](https://our.internmc.facebook.com/intern/diff/D78198154) [ghstack-poisoned]

Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D78198154](https://our.internmc.facebook.com/intern/diff/D78198154) [ghstack-poisoned]

Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: d1a4a2c Pull Request resolved: #2520

…for xnnpack" Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D78198154](https://our.internmc.facebook.com/intern/diff/D78198154) [ghstack-poisoned]

Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D78198154](https://our.internmc.facebook.com/intern/diff/D78198154) [ghstack-poisoned]

Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 6630cc7 Pull Request resolved: #2520

…for xnnpack" Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D78198154](https://our.internmc.facebook.com/intern/diff/D78198154) [ghstack-poisoned]

Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D78198154](https://our.internmc.facebook.com/intern/diff/D78198154) [ghstack-poisoned]

Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 0f5643a Pull Request resolved: #2520

kimishpatel · 2025-08-11T23:07:41Z

@kimishpatel has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

…for xnnpack" Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D78198154](https://our.internmc.facebook.com/intern/diff/D78198154) [ghstack-poisoned]

Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D78198154](https://our.internmc.facebook.com/intern/diff/D78198154) [ghstack-poisoned]

Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: ddb2acc Pull Request resolved: #2520

kimishpatel · 2025-08-13T21:19:30Z

@kimishpatel has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

* When replacing literals with placeholders lists are always converted to tuples Summary: THis is needed because lists are not hashable, since they are mutable, and as a result we cannot have literals_to_ph in pattern rewrites used inside reference_representation_rewrite.py Test Plan: CI + next diff relies on this feature Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned] * Allow pattern replacement to ignore literals Summary: This is necessary because sometimes the patterns found have literals include tuple of ints kind of literals. This values shouldnt be used for pattern matching since often they are based on consts derived from example inputs. THis is not exactly a safe thing to do in general so by default it is turned off Test Plan: Subsequent diff adds a pattern that relies on this Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned] * Reference representation of dqlinear int4 for xnnpack Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned] * Update base for Update on "Reference representation of dqlinear int4 for xnnpack" Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned] * Update base for Update on "Reference representation of dqlinear int4 for xnnpack" Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned] * Update base for Update on "Reference representation of dqlinear int4 for xnnpack" Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned] * Update base for Update on "Reference representation of dqlinear int4 for xnnpack" Summary: This diff adds dynamic quantized linear's integer arithmetic representation. This is quite close to how arithmetic is done in xnnpack. Basic tests added against q/dq to make things are sane. Followups: - See if such a graph is traceable. - Optimize implementation if needed Test Plan: added Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D78198154](https://our.internmc.facebook.com/intern/diff/D78198154) [ghstack-poisoned]

kimishpatel added 3 commits July 10, 2025 11:33

This was referenced Jul 10, 2025

When replacing literals with placeholders lists are always converted to #2518

Merged

Allow pattern replacement to ignore literals #2519

Merged

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 10, 2025

kimishpatel mentioned this pull request Jul 10, 2025

[WIP] Reference representation of dqlinear int4 for xnnpack #2507

Closed

jerryzh168 reviewed Jul 10, 2025

View reviewed changes

kimishpatel added the topic: new feature Use this tag if this PR adds a new feature label Jul 11, 2025

kimishpatel added 2 commits July 11, 2025 07:55

kimishpatel added 2 commits July 11, 2025 07:57

kimishpatel added 2 commits July 11, 2025 08:27

jerryzh168 approved these changes Jul 11, 2025

View reviewed changes

kimishpatel added 2 commits July 11, 2025 18:03

kimishpatel added 2 commits August 6, 2025 12:51

kimishpatel added 2 commits August 7, 2025 10:26

kimishpatel added 2 commits August 7, 2025 14:02

kimishpatel added 2 commits August 8, 2025 10:33

kimishpatel added 2 commits August 11, 2025 14:29

kimishpatel added 2 commits August 11, 2025 19:36

kimishpatel changed the base branch from gh/kimishpatel/7/base to main August 12, 2025 02:36

kimishpatel merged commit 6794ef5 into main Aug 13, 2025
35 of 36 checks passed

kimishpatel deleted the gh/kimishpatel/7/head branch August 13, 2025 22:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reference representation of dqlinear int4 for xnnpack#2520

Reference representation of dqlinear int4 for xnnpack#2520
kimishpatel merged 23 commits into
mainfrom
gh/kimishpatel/7/head

kimishpatel commented Jul 10, 2025 •

edited

Loading

Uh oh!

pytorch-bot Bot commented Jul 10, 2025 •

edited

Loading

Uh oh!

jerryzh168 Jul 10, 2025

Uh oh!

kimishpatel Jul 10, 2025

Uh oh!

jerryzh168 left a comment

Uh oh!

kimishpatel commented Jul 11, 2025

Uh oh!

kimishpatel commented Aug 7, 2025

Uh oh!

kimishpatel commented Aug 11, 2025

Uh oh!

kimishpatel commented Aug 13, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		"_qdq_dynamic_quantized_linear_4bit_groupwise",
		"_reference_dynamic_quantized_linear_4bit_groupwise",

Conversation

kimishpatel commented Jul 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot Bot commented Jul 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/2520

✅ No Failures

Uh oh!

jerryzh168 Jul 10, 2025

Choose a reason for hiding this comment

Uh oh!

kimishpatel Jul 10, 2025

Choose a reason for hiding this comment

Uh oh!

jerryzh168 left a comment

Choose a reason for hiding this comment

Uh oh!

kimishpatel commented Jul 11, 2025

Uh oh!

kimishpatel commented Aug 7, 2025

Uh oh!

kimishpatel commented Aug 11, 2025

Uh oh!

kimishpatel commented Aug 13, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kimishpatel commented Jul 10, 2025 •

edited

Loading

pytorch-bot Bot commented Jul 10, 2025 •

edited

Loading