Skip to content

[DTensor] insert Replicate at the begining for matmul single dim#172150

Closed
weifengpy wants to merge 20 commits intogh/weifengpy/47/basefrom
gh/weifengpy/47/head
Closed

[DTensor] insert Replicate at the begining for matmul single dim#172150
weifengpy wants to merge 20 commits intogh/weifengpy/47/basefrom
gh/weifengpy/47/head

Conversation

@weifengpy
Copy link
Copy Markdown
Contributor

@weifengpy weifengpy commented Jan 10, 2026 •
edited
Loading

Stack from ghstack (oldest at bottom):

gen_einsum_strategies inserts replicate strategy first:

pytorch/torch/distributed/tensor/_ops/_einsum_strategy.py

Lines 121 to 122 in 74b6a0e

placement_list: list[Placement] = [Replicate()] * (len(input_dims) + 1)
strategies_over_one_mesh_dim.append(placement_list)

_select_min_cost_strategy choose Replicate at equal cost

This PR makes sure consistent matmul results after switching to single dim strategy #170359

@weifengpy
add assert to fix
0d352b4 
[ghstack-poisoned]
@pytorch-bot
Copy link
Copy Markdown

pytorch-bot Bot commented Jan 10, 2026 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/172150

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

❌ 3 New Failures

As of commit 335aedf with merge base 9f5d6ec (image):

NEW FAILURES - The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

weifengpy added a commit that referenced this pull request Jan 10, 2026
@weifengpy
add assert to fix
bc0cd89 
ghstack-source-id: ce85ae5
Pull Request resolved: #172150
@weifengpy
Update on "add assert to fix"
48fc46c 
[ghstack-poisoned]
weifengpy added a commit that referenced this pull request Jan 12, 2026
@weifengpy
add assert to fix
8b2fbc8 
ghstack-source-id: 33fad11
Pull Request resolved: #172150
@pytorch-bot pytorch-bot Bot added the release notes: distributed (dtensor) release notes category label Jan 12, 2026
@weifengpy
Update on "add assert to fix"
8f66ee5 
[ghstack-poisoned]
@weifengpy weifengpy mentioned this pull request Jan 12, 2026
Closed
weifengpy added a commit that referenced this pull request Jan 12, 2026
@weifengpy
add assert to fix
d7649c3 
ghstack-source-id: d0a84f6
Pull Request resolved: #172150
@weifengpy
Update on "add assert to fix"
255be03 
[ghstack-poisoned]
weifengpy added a commit that referenced this pull request Jan 12, 2026
@weifengpy
add assert to fix
052916b 
ghstack-source-id: d0a84f6
Pull Request resolved: #172150
@weifengpy
Update on "add assert to fix"
b5b1938 
[ghstack-poisoned]
weifengpy added a commit that referenced this pull request Jan 12, 2026
@weifengpy
add assert to fix
aa1f818 
ghstack-source-id: ca976b1
Pull Request resolved: #172150
@weifengpy
Update on "add assert to fix"
9dacab4 
[ghstack-poisoned]
weifengpy added a commit that referenced this pull request Jan 12, 2026
@weifengpy
add assert to fix
5e3f299 
ghstack-source-id: 2058d49
Pull Request resolved: #172150
@weifengpy
Update on "add assert to fix"
e9d7d76 
[ghstack-poisoned]
weifengpy added a commit that referenced this pull request Jan 12, 2026
@weifengpy
add assert to fix
93ae3d2 
ghstack-source-id: 9575242
Pull Request resolved: #172150
@weifengpy
Update on "add assert to fix"
e6c0927 
[ghstack-poisoned]
weifengpy added a commit that referenced this pull request Jan 12, 2026
@weifengpy
add assert to fix
85251e5 
ghstack-source-id: 5f229fb
Pull Request resolved: #172150
@weifengpy
Update on "add assert to fix"
842f94e 
[ghstack-poisoned]
weifengpy added a commit that referenced this pull request Jan 12, 2026
@weifengpy
add assert to fix
0f11122 
ghstack-source-id: 89e6922
Pull Request resolved: #172150
@weifengpy
Update on "add assert to fix"
1d80347 
[ghstack-poisoned]
@weifengpy weifengpy mentioned this pull request Jan 12, 2026
Closed
weifengpy added a commit that referenced this pull request Jan 12, 2026
@weifengpy
add assert to fix
28d4d5a 
ghstack-source-id: 61f0cee
Pull Request resolved: #172150
@weifengpy
Update on "add assert to fix"
6ed3b39 
[ghstack-poisoned]
weifengpy added a commit that referenced this pull request Jan 12, 2026
@weifengpy
add assert to fix
547247c 
ghstack-source-id: 7eed38e
Pull Request resolved: #172150
@weifengpy
Update on "add assert to fix"
8e38d4c 
[ghstack-poisoned]
weifengpy added a commit that referenced this pull request Jan 12, 2026
@weifengpy
add assert to fix
630f845 
ghstack-source-id: 324dfa9
Pull Request resolved: #172150
@weifengpy
Update on "add assert to fix"
97cf1b3 
[ghstack-poisoned]
@weifengpy
Update on "[DTensor] insert Replicate at the begining for matmul sing…
86bf108 
…le dim"


gen_einsum_strategies inserts replicate strategy first:
https://github.com/pytorch/pytorch/blob/74b6a0efa359722def4b585d9d91fbc3a4bfa530/torch/distributed/tensor/_ops/_einsum_strategy.py#L121-L122

_select_min_cost_strategy choose Replicate at equal cost

This PR makes sure consistent matmul results after switching to single dim strategy #170359

[ghstack-poisoned]
weifengpy added a commit that referenced this pull request Jan 13, 2026
@weifengpy
[DTensor] insert Replicate at the begining for matmul single dim
3f32662 
strategy

ghstack-source-id: 824df02
Pull Request resolved: #172150
@weifengpy
Update on "[DTensor] insert Replicate at the begining for matmul sing…
1de62c5 
…le dim"


gen_einsum_strategies inserts replicate strategy first:
https://github.com/pytorch/pytorch/blob/74b6a0efa359722def4b585d9d91fbc3a4bfa530/torch/distributed/tensor/_ops/_einsum_strategy.py#L121-L122

_select_min_cost_strategy choose Replicate at equal cost

This PR makes sure consistent matmul results after switching to single dim strategy #170359

[ghstack-poisoned]
weifengpy added a commit that referenced this pull request Jan 13, 2026
@weifengpy
[DTensor] insert Replicate at the begining for matmul single dim
c994e3a 
strategy

ghstack-source-id: 077fc01
Pull Request resolved: #172150
@weifengpy
Update on "[DTensor] insert Replicate at the begining for matmul sing…
64aed32 
…le dim"


gen_einsum_strategies inserts replicate strategy first:
https://github.com/pytorch/pytorch/blob/74b6a0efa359722def4b585d9d91fbc3a4bfa530/torch/distributed/tensor/_ops/_einsum_strategy.py#L121-L122

_select_min_cost_strategy choose Replicate at equal cost

This PR makes sure consistent matmul results after switching to single dim strategy #170359

[ghstack-poisoned]
weifengpy added a commit that referenced this pull request Jan 13, 2026
@weifengpy
[DTensor] insert Replicate at the begining for matmul single dim
09c9d27 
ghstack-source-id: 36eb1b5
Pull Request resolved: #172150
This was referenced Jan 13, 2026
Closed
Closed
@weifengpy
Copy link
Copy Markdown
Contributor Author

weifengpy commented Jan 13, 2026

@pytorchmergebot merge

@pytorch-bot pytorch-bot Bot added the ciflow/trunk Trigger trunk jobs on your pull request label Jan 13, 2026
@pytorchmergebot
Copy link
Copy Markdown
Collaborator

pytorchmergebot commented Jan 13, 2026

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

This was referenced Jan 14, 2026
Closed
Closed
@weifengpy
Update on "[DTensor] insert Replicate at the begining for matmul sing…
3c4fbba 
…le dim"


gen_einsum_strategies inserts replicate strategy first:
https://github.com/pytorch/pytorch/blob/74b6a0efa359722def4b585d9d91fbc3a4bfa530/torch/distributed/tensor/_ops/_einsum_strategy.py#L121-L122

_select_min_cost_strategy choose Replicate at equal cost

This PR makes sure consistent matmul results after switching to single dim strategy #170359

[ghstack-poisoned]
@weifengpy weifengpy mentioned this pull request Jan 14, 2026
Closed
@weifengpy
Update on "[DTensor] insert Replicate at the begining for matmul sing…
335aedf 
…le dim"


gen_einsum_strategies inserts replicate strategy first:
https://github.com/pytorch/pytorch/blob/74b6a0efa359722def4b585d9d91fbc3a4bfa530/torch/distributed/tensor/_ops/_einsum_strategy.py#L121-L122

_select_min_cost_strategy choose Replicate at equal cost

This PR makes sure consistent matmul results after switching to single dim strategy #170359

[ghstack-poisoned]
@pytorchmergebot
Copy link
Copy Markdown
Collaborator

pytorchmergebot commented Jan 14, 2026

Merge failed

Reason: New commits were pushed while merging. Please rerun the merge command.

Details for Dev Infra team Raised by workflow job

@weifengpy
Copy link
Copy Markdown
Contributor Author

weifengpy commented Jan 14, 2026

@pytorchmergebot merge -f "unrelated CI errors"

@pytorchmergebot
Copy link
Copy Markdown
Collaborator

pytorchmergebot commented Jan 14, 2026

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

mattteochen pushed a commit to mattteochen/pytorch that referenced this pull request Jan 15, 2026
@weifengpy @mattteochen
[DTensor] insert Replicate at the begining for matmul single dim (pyt…
47d6837 
…orch#172150)

gen_einsum_strategies inserts replicate strategy first:
https://github.com/pytorch/pytorch/blob/74b6a0efa359722def4b585d9d91fbc3a4bfa530/torch/distributed/tensor/_ops/_einsum_strategy.py#L121-L122

_select_min_cost_strategy choose Replicate at equal cost

This PR makes sure consistent matmul results after switching to single dim strategy pytorch#170359
Pull Request resolved: pytorch#172150
Approved by: https://github.com/wconstab
SergeyTyshkevich pushed a commit to SergeyTyshkevich/chart2 that referenced this pull request Jan 19, 2026
@weifengpy
[DTensor] insert Replicate at the begining for matmul single dim
9a65e4a 
ghstack-source-id: 614a5fd
Pull Request resolved: pytorch/pytorch#172150
SergeyTyshkevich pushed a commit to SergeyTyshkevich/chart2 that referenced this pull request Jan 19, 2026
@weifengpy
[DTensor] insert Replicate at the begining for matmul single dim
fe49869 
ghstack-source-id: 49ab8a7
Pull Request resolved: pytorch/pytorch#172150
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wconstab wconstab wconstab approved these changes

@zpcore zpcore Awaiting requested review from zpcore

Assignees

No one assigned

Labels

ciflow/inductor ciflow/trunk Trigger trunk jobs on your pull request Merged release notes: distributed (dtensor) release notes category

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@weifengpy @pytorchmergebot @wconstab