[dtensor] Register foreach and fused ops via single_dim_strategy#177187
Closed
anshul-si wants to merge 26 commits intogh/anshul-si/107/basefrom
Closed
[dtensor] Register foreach and fused ops via single_dim_strategy#177187anshul-si wants to merge 26 commits intogh/anshul-si/107/basefrom
anshul-si wants to merge 26 commits intogh/anshul-si/107/basefrom
Conversation
Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. [ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/177187
Note: Links to docs will display an error until the docs builds have been completed. ✅ You can merge normally! (2 Unrelated Failures)As of commit 958a4d5 with merge base 549e5c7 ( BROKEN TRUNK - The following job failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
anshul-si
added a commit
that referenced
this pull request
Mar 11, 2026
Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. ghstack-source-id: cdfddce Pull Request resolved: #177187
…rategy" Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. [ghstack-poisoned]
anshul-si
added a commit
that referenced
this pull request
Mar 11, 2026
Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. ghstack-source-id: b1cca4f Pull Request resolved: #177187
…rategy" Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. [ghstack-poisoned]
anshul-si
added a commit
that referenced
this pull request
Mar 11, 2026
Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. ghstack-source-id: c46d594 Pull Request resolved: #177187
…rategy" Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. [ghstack-poisoned]
…rategy" Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. [ghstack-poisoned]
…rategy" Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. [ghstack-poisoned]
…rategy" Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. [ghstack-poisoned]
…rategy" Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. [ghstack-poisoned]
…rategy" Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. [ghstack-poisoned]
…rategy" Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. [ghstack-poisoned]
…rategy" Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. [ghstack-poisoned]
…rategy" Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. [ghstack-poisoned]
…rategy" Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. [ghstack-poisoned]
sandy-gags
pushed a commit
to sandy-gags/pytorch
that referenced
this pull request
Mar 12, 2026
Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. ghstack-source-id: 61151ad Pull Request resolved: pytorch/pytorch#177187
wconstab
reviewed
Mar 12, 2026
…rategy" Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. [ghstack-poisoned]
This was referenced Mar 12, 2026
Merge test_fused_adam, test_fused_adamw_strided_shard_cross_mesh, and test_fused_adamw_tensor_lr into a single test_fused_optimizer test that parameterizes over the variant. Reduces code duplication while preserving all five test scenarios (same mesh, cross mesh, cross mesh with amsgrad, strided shard cross mesh, and tensor lr).
…rategy" Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. [ghstack-poisoned]
…rategy" Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. [ghstack-poisoned]
…rategy" Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. [ghstack-poisoned]
…rategy" Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. [ghstack-poisoned]
…rategy" Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. [ghstack-poisoned]
…rategy" Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. [ghstack-poisoned]
…rategy" Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. [ghstack-poisoned]
…rategy" Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. [ghstack-poisoned]
Contributor
|
@claude review this PR |
|
Claude finished @wconstab's task in 7m 4s —— View job PR Review: [dtensor] Register foreach and fused ops via single_dim_strategy
SummaryThis PR unifies foreach and fused optimizer ops to use the The approach is sound — the single_dim_strategy path is the modern path with better support for Dijkstra-based expansion and concrete sharding rules. Issues Found1. Dead code:
|
wconstab
approved these changes
Mar 22, 2026
Contributor
wconstab
left a comment
There was a problem hiding this comment.
dead code is removed in the next PR i think, or if any is missed, please remove it
…rategy" Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. [ghstack-poisoned]
Contributor
Author
|
@pytorchbot merge |
Collaborator
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
pytorchmergebot
pushed a commit
that referenced
this pull request
Mar 24, 2026
The old strategy registration path (register_op_strategy-based functions and data structures) was superseded by the register_single_dim_strategy infrastructure but never cleaned up. This removes ~550 lines of dead code including: pointwise_strategy, linear_pointwise_strategy, copy_strategy, common_pointwise_strategy, single_mesh_dim_* strategy functions, list_pointwise_strategy, list_linear_pointwise_strategy, for_each_ops/for_each_linearity_ops/fused_ops lists, and their associated helper sets and unused imports. Authored with Claude. Pull Request resolved: #177208 Approved by: https://github.com/Skylion007 ghstack dependencies: #177186, #177187
anshul-si
added a commit
that referenced
this pull request
Mar 31, 2026
The old strategy registration path (register_op_strategy-based functions and data structures) was superseded by the register_single_dim_strategy infrastructure but never cleaned up. This removes ~550 lines of dead code including: pointwise_strategy, linear_pointwise_strategy, copy_strategy, common_pointwise_strategy, single_mesh_dim_* strategy functions, list_pointwise_strategy, list_linear_pointwise_strategy, for_each_ops/for_each_linearity_ops/fused_ops lists, and their associated helper sets and unused imports. Authored with Claude. Pull Request resolved: #177208 Approved by: https://github.com/Skylion007 ghstack dependencies: #177186, #177187 ghstack-source-id: f005cbc
anshul-si
added a commit
that referenced
this pull request
Mar 31, 2026
The old strategy registration path (register_op_strategy-based functions and data structures) was superseded by the register_single_dim_strategy infrastructure but never cleaned up. This removes ~550 lines of dead code including: pointwise_strategy, linear_pointwise_strategy, copy_strategy, common_pointwise_strategy, single_mesh_dim_* strategy functions, list_pointwise_strategy, list_linear_pointwise_strategy, for_each_ops/for_each_linearity_ops/fused_ops lists, and their associated helper sets and unused imports. Authored with Claude. Pull Request resolved: #177208 Approved by: https://github.com/Skylion007 ghstack dependencies: #177186, #177187 ghstack-source-id: f4e0b82
AaronWang04
pushed a commit
to AaronWang04/pytorch
that referenced
this pull request
Mar 31, 2026
…orch#177187) Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. Pull Request resolved: pytorch#177187 Approved by: https://github.com/wconstab ghstack dependencies: pytorch#177186
AaronWang04
pushed a commit
to AaronWang04/pytorch
that referenced
this pull request
Mar 31, 2026
The old strategy registration path (register_op_strategy-based functions and data structures) was superseded by the register_single_dim_strategy infrastructure but never cleaned up. This removes ~550 lines of dead code including: pointwise_strategy, linear_pointwise_strategy, copy_strategy, common_pointwise_strategy, single_mesh_dim_* strategy functions, list_pointwise_strategy, list_linear_pointwise_strategy, for_each_ops/for_each_linearity_ops/fused_ops lists, and their associated helper sets and unused imports. Authored with Claude. Pull Request resolved: pytorch#177208 Approved by: https://github.com/Skylion007 ghstack dependencies: pytorch#177186, pytorch#177187
anshul-si
added a commit
that referenced
this pull request
Mar 31, 2026
The old strategy registration path (register_op_strategy-based functions and data structures) was superseded by the register_single_dim_strategy infrastructure but never cleaned up. This removes ~550 lines of dead code including: pointwise_strategy, linear_pointwise_strategy, copy_strategy, common_pointwise_strategy, single_mesh_dim_* strategy functions, list_pointwise_strategy, list_linear_pointwise_strategy, for_each_ops/for_each_linearity_ops/fused_ops lists, and their associated helper sets and unused imports. Authored with Claude. Pull Request resolved: #177208 Approved by: https://github.com/Skylion007 ghstack dependencies: #177186, #177187 ghstack-source-id: 5a0cf25
nklshy-aws
pushed a commit
to nklshy-aws/pytorch
that referenced
this pull request
Apr 7, 2026
…orch#177187) Enable foreach and fused ops to use the single_dim_strategy path: - Add foreach ops to existing category lists (binary_additive_ops, binary_mul_ops, etc.) and the pointwise_ops list - Add fused ops to pointwise_ops list for unified registration - Add _is_list_op() helper to detect foreach/fused/amp_foreach ops - Modify _register_single_dim_pointwise to use needs_pytree for list ops and cross_mesh_indices for fused ops - Remove separate register_single_dim_strategy loop for fused ops Tests cover multi-tensor foreach lists, mixed placements across list elements, same-mesh fused adam, and cross-mesh fused adam exercising the cross_mesh_indices code path. Pull Request resolved: pytorch#177187 Approved by: https://github.com/wconstab ghstack dependencies: pytorch#177186
nklshy-aws
pushed a commit
to nklshy-aws/pytorch
that referenced
this pull request
Apr 7, 2026
The old strategy registration path (register_op_strategy-based functions and data structures) was superseded by the register_single_dim_strategy infrastructure but never cleaned up. This removes ~550 lines of dead code including: pointwise_strategy, linear_pointwise_strategy, copy_strategy, common_pointwise_strategy, single_mesh_dim_* strategy functions, list_pointwise_strategy, list_linear_pointwise_strategy, for_each_ops/for_each_linearity_ops/fused_ops lists, and their associated helper sets and unused imports. Authored with Claude. Pull Request resolved: pytorch#177208 Approved by: https://github.com/Skylion007 ghstack dependencies: pytorch#177186, pytorch#177187
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack from ghstack (oldest at bottom):
Enable foreach and fused ops to use the single_dim_strategy path:
binary_mul_ops, etc.) and the pointwise_ops list
ops and cross_mesh_indices for fused ops
Tests cover multi-tensor foreach lists, mixed placements across list
elements, same-mesh fused adam, and cross-mesh fused adam exercising
the cross_mesh_indices code path.