Move cutlass_int4_packed_layout to prototype by jainapurva · Pull Request #3277 · pytorch/ao

jainapurva · 2025-11-03T06:03:56Z

This pull request moves CutlassInt4PackedLayout and its implementation from torchao/dtypes/uintx/cutlass_int4_packed_layout.py to torchao/prototype/dtypes/uintx/cutlass_int4_packed_layout.py, and exposed it via torchao.prototype.dtypes while maintaining BC at torchao.prototype.dtypes.uintx with a deprecation warning for torchao v0.16.0.
Updates API documentation to reflect the changes

Reference Issue: #2752

Removed unused exports from block_sparse_layout.py

…orts - Move torchao/prototype/sparsity/block_sparse_layout.py to torchao/prototype/dtypes/uintx/block_sparse_layout.py - Update all imports to use 'from torchao.prototype.dtypes import BlockSparseLayout' - Update deprecation warning message - Create torchao/prototype/dtypes/__init__.py and torchao/prototype/dtypes/uintx/__init__.py - Remove BlockSparseLayout from torchao/prototype/sparsity/__init__.py - Update documentation

- Moved cutlass_int4_packed_layout.py from torchao/dtypes/uintx/ to torchao/prototype/dtypes/uintx/ - Created torchao/prototype/dtypes/__init__.py and uintx/__init__.py to export CutlassInt4PackedLayout - Replaced old file with backward compatibility stub that imports from new location - Added deprecation warning for old import path (to be removed in v0.16.0) - Updated torchao/dtypes/__init__.py to re-export from prototype for backward compatibility - Updated internal imports in affine_quantized_tensor_ops.py to use new prototype location - Removed CutlassInt4PackedLayout from torchao/dtypes/uintx/__init__.py to avoid circular imports - Updated documentation to move CutlassInt4PackedLayout to Prototype section All import paths work: - New: from torchao.prototype.dtypes import CutlassInt4PackedLayout - Backward compat: from torchao.dtypes import CutlassInt4PackedLayout - Deprecated: from torchao.dtypes.uintx.cutlass_int4_packed_layout import CutlassInt4PackedLayout

pytorch-bot · 2025-11-03T06:04:00Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/3277

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 5 Pending

As of commit 531b07b with merge base 42f28f7 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

Removed redundant imports of block sparse layout functions.

Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags:

Removed unused import for CutlassInt4PackedLayout.

Removed __all__ exports from cutlass_int4_packed_layout.py

jerryzh168 · 2025-11-04T01:15:48Z

    FloatxTensor
    FloatxTensorCoreLayout
    MarlinSparseLayout
-    BlockSparseLayout


looks like it's both? maybe try to split would be better, or update the summary to say it's two things

It's in two separate PRs:
Pr for BlockSparseLayout: #3276

This one is for CutlassInt4PackedLayout

…utlass_int4_packed_layout

vkuzo · 2025-11-05T01:09:11Z

-)
-from torchao.dtypes.uintx.plain_layout import (
-    _aqt_is_int8,
+warnings.warn(


add #2752 to the message

also, I'd remove torchao v0.16.0 and just say "in a future release of torchao", just in case the work gets delayed

I would still prefer adding a deprecation timeline, maybe we can add it in the issue description

yep, add it to the issue! that way it can be modified without a code change.

…utlass_int4_packed_layout

Updated deprecation warning message to indicate future removal.

jainapurva added 10 commits November 3, 2025 00:49

Move block_sparse_layout to prototype

bc67c1a

test fixes

dd2e7d6

Remove unused import in __init__.py

efa74cd

Clean up exports in block_sparse_layout.py

3722537

Removed unused exports from block_sparse_layout.py

test fixes

6c1b8ef

Fix ruff import sorting and add noqa for re-exported imports

1702bdf

Apply ruff formatting (remove trailing newlines)

de054ab

Add Prototype section to dtypes documentation with BlockSparseLayout

4ef291f

meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 3, 2025

jainapurva changed the base branch from main to move_block_sparsity November 3, 2025 06:04

jainapurva added 5 commits November 3, 2025 09:49

Clean up imports in affine_quantized_tensor_ops.py

d699ee0

Removed redundant imports of block sparse layout functions.

Update internal links

ab84799

<Replace this line with a title. Use 1 line only, 67 chars or less>

83795a6

Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags:

test fixes

40ab188

ruff fixes

894857e

jainapurva mentioned this pull request Nov 3, 2025

Migrating from AffineQuantizedTensor + Layouts to new structure of tensor subclasses #2752

Closed

17 tasks

jainapurva added 3 commits November 3, 2025 22:19

Fixes

5b45869

Remove unused import from uintx init file

1d51ebf

Removed unused import for CutlassInt4PackedLayout.

Remove __all__ exports from module

8c18d4d

Removed __all__ exports from cutlass_int4_packed_layout.py

jainapurva force-pushed the move_cutlass_int4_packed_layout branch from 6c03ed4 to 8c18d4d Compare November 3, 2025 22:29

jainapurva requested a review from jerryzh168 November 3, 2025 22:34

jainapurva added the module: bc-breaking Use this tag if this PR breaks backward compatibility label Nov 3, 2025

jainapurva marked this pull request as ready for review November 3, 2025 22:38

jainapurva changed the base branch from move_block_sparsity to main November 3, 2025 22:40

Empty commit to trigger CI

081a4ed

jerryzh168 reviewed Nov 4, 2025

View reviewed changes

Comment thread torchao/dtypes/uintx/block_sparse_layout.py

jerryzh168 reviewed Nov 4, 2025

View reviewed changes

Comment thread torchao/dtypes/uintx/block_sparse_layout.py

Lint fixes

61c1986

jainapurva changed the base branch from main to move_block_sparsity November 4, 2025 18:27

jainapurva added 2 commits November 4, 2025 11:23

Add test cases

8ebbb9c

Merge remote-tracking branch 'origin/move_block_sparsity' into move_c…

b199f7b

…utlass_int4_packed_layout

jerryzh168 approved these changes Nov 4, 2025

View reviewed changes

vkuzo reviewed Nov 5, 2025

View reviewed changes

jainapurva force-pushed the move_block_sparsity branch from 8ebbb9c to 9cce063 Compare November 5, 2025 04:03

Add test cases

78f5e4c

jainapurva force-pushed the move_block_sparsity branch from 9cce063 to 78f5e4c Compare November 5, 2025 04:53

jainapurva added 2 commits November 4, 2025 20:54

Merge remote-tracking branch 'origin/move_block_sparsity' into move_c…

ffcaca6

…utlass_int4_packed_layout

Add test cases

1a75689

jainapurva force-pushed the move_block_sparsity branch from 78f5e4c to 1a75689 Compare November 5, 2025 05:06

jainapurva added 2 commits November 4, 2025 21:08

Merge remote-tracking branch 'origin/move_block_sparsity' into move_c…

8222741

…utlass_int4_packed_layout

Updates

42acafa

jainapurva force-pushed the move_block_sparsity branch from 1a75689 to 9bff162 Compare November 5, 2025 05:58

jainapurva changed the base branch from move_block_sparsity to main November 5, 2025 20:49

jainapurva added 3 commits November 5, 2025 12:51

Merge branch 'main' into move_cutlass_int4_packed_layout

dc1e447

Update block_sparse_layout.py

40297ba

Modify deprecation warning for import path

531b07b

Updated deprecation warning message to indicate future removal.

jainapurva merged commit aadfded into main Nov 5, 2025
21 checks passed

namgyu-youn pushed a commit to namgyu-youn/ao that referenced this pull request Nov 21, 2025

Move cutlass_int4_packed_layout to prototype (pytorch#3277)

a2d442b

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move cutlass_int4_packed_layout to prototype#3277

Move cutlass_int4_packed_layout to prototype#3277
jainapurva merged 30 commits intomainfrom
move_cutlass_int4_packed_layout

jainapurva commented Nov 3, 2025 •

edited

Loading

Uh oh!

pytorch-bot Bot commented Nov 3, 2025 •

edited

Loading

Uh oh!

jerryzh168 Nov 4, 2025

Uh oh!

jainapurva Nov 4, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

vkuzo Nov 5, 2025

Uh oh!

vkuzo Nov 5, 2025

Uh oh!

jainapurva Nov 5, 2025

Uh oh!

vkuzo Nov 5, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

jainapurva commented Nov 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot Bot commented Nov 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/3277

⏳ No Failures, 5 Pending

Uh oh!

jerryzh168 Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

jainapurva Nov 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

vkuzo Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

vkuzo Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

jainapurva Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

vkuzo Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jainapurva commented Nov 3, 2025 •

edited

Loading

pytorch-bot Bot commented Nov 3, 2025 •

edited

Loading

jainapurva Nov 4, 2025 •

edited

Loading