Skip to content

[CP] Remove the need of recording cp_dim in the global var#162540

Closed
fegin wants to merge 6 commits intogh/fegin/316/basefrom
gh/fegin/316/head
Closed

[CP] Remove the need of recording cp_dim in the global var#162540
fegin wants to merge 6 commits intogh/fegin/316/basefrom
gh/fegin/316/head

Conversation

@fegin
Copy link
Contributor

@fegin fegin commented Sep 9, 2025 •
edited
Loading

@fegin
Update
ef47b55 
[ghstack-poisoned]
@pytorch-bot
Copy link

pytorch-bot bot commented Sep 9, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/162540

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 1 Pending

As of commit 2cbd999 with merge base 7a0f933 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorch-bot pytorch-bot bot added ciflow/inductor oncall: distributed Add this issue/PR to distributed oncall triage queue labels Sep 9, 2025
This was referenced Sep 9, 2025
Closed
Closed
Closed
Closed
@fegin
Update
543e96a 
[ghstack-poisoned]
@fegin fegin added the topic: not user facing topic category label Sep 16, 2025
@fegin
Update
9c6033e 
[ghstack-poisoned]
@fegin
Update
7f6e074 
[ghstack-poisoned]
@fegin
Update
1c93ee4 
[ghstack-poisoned]
This was referenced Sep 16, 2025
Closed
Closed
@fegin fegin mentioned this pull request Sep 17, 2025
Closed
@fegin
Update
2cbd999 
[ghstack-poisoned]
@fegin fegin mentioned this pull request Sep 18, 2025
Closed
XilunWu
XilunWu approved these changes Sep 18, 2025
View reviewed changes
Copy link
Contributor

@XilunWu XilunWu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: we still have multiple hardcoding sites (seq_dim=2 or seq_dim = 2)

@fegin
Copy link
Contributor Author

fegin commented Sep 18, 2025

@pytorchbot merge

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Sep 18, 2025
@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Sep 18, 2025

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

pytorchmergebot pushed a commit that referenced this pull request Sep 19, 2025
@fegin @pytorchmergebot
[CP][BE] Correct the names of some tests (#162541)
cd4303a 
We are not doing ring attention but only using allgather to do CP for Flex.
Pull Request resolved: #162541
Approved by: https://github.com/ezyang, https://github.com/Skylion007, https://github.com/tianyu-l, https://github.com/XilunWu
ghstack dependencies: #162539, #162540
pytorchmergebot pushed a commit that referenced this pull request Sep 19, 2025
@fegin @pytorchmergebot
[CP][BE] Cosmetic refactors for CP code base (#163115)
0815091 
Summary:
This PR is extracted from #162542, to make the original PR
easier to review. This PR only contains cosmetic changes.

Pull Request resolved: #163115
Approved by: https://github.com/tianyu-l
ghstack dependencies: #162539, #162540, #162541
pytorchmergebot pushed a commit that referenced this pull request Sep 19, 2025
@fegin @pytorchmergebot
[CP][BE] Correct an incorrect docstring (#163131)
093f064 
Pull Request resolved: #163131
Approved by: https://github.com/tianyu-l, https://github.com/XilunWu
ghstack dependencies: #162539, #162540, #162541, #163115
pytorchmergebot pushed a commit that referenced this pull request Sep 20, 2025
@fegin @pytorchmergebot
[CP] Fix cuDNN CP LSE dimension bug (#163231)
d55c9d5 
We should only unsqueeze if necessary.

Fix #162743

Pull Request resolved: #163231
Approved by: https://github.com/eqy
ghstack dependencies: #162539, #162540, #162541, #163115, #163131
mansiag05 pushed a commit to mansiag05/pytorch that referenced this pull request Sep 22, 2025
@fegin @mansiag05
[CP] Remove the need of recording cp_dim in the global var (pytorch#1…
c9415f2 
…62540)

This information can be obtained during the dispatching.

Pull Request resolved: pytorch#162540
Approved by: https://github.com/ezyang, https://github.com/tianyu-l, https://github.com/XilunWu
ghstack dependencies: pytorch#162539
mansiag05 pushed a commit to mansiag05/pytorch that referenced this pull request Sep 22, 2025
@fegin @mansiag05
[CP][BE] Correct the names of some tests (pytorch#162541)
61034ce 
We are not doing ring attention but only using allgather to do CP for Flex.
Pull Request resolved: pytorch#162541
Approved by: https://github.com/ezyang, https://github.com/Skylion007, https://github.com/tianyu-l, https://github.com/XilunWu
ghstack dependencies: pytorch#162539, pytorch#162540
mansiag05 pushed a commit to mansiag05/pytorch that referenced this pull request Sep 22, 2025
@fegin @mansiag05
[CP][BE] Cosmetic refactors for CP code base (pytorch#163115)
83aa65c 
Summary:
This PR is extracted from pytorch#162542, to make the original PR
easier to review. This PR only contains cosmetic changes.

Pull Request resolved: pytorch#163115
Approved by: https://github.com/tianyu-l
ghstack dependencies: pytorch#162539, pytorch#162540, pytorch#162541
mansiag05 pushed a commit to mansiag05/pytorch that referenced this pull request Sep 22, 2025
@fegin @mansiag05
[CP][BE] Correct an incorrect docstring (pytorch#163131)
d148bd2 
Pull Request resolved: pytorch#163131
Approved by: https://github.com/tianyu-l, https://github.com/XilunWu
ghstack dependencies: pytorch#162539, pytorch#162540, pytorch#162541, pytorch#163115
mansiag05 pushed a commit to mansiag05/pytorch that referenced this pull request Sep 22, 2025
@fegin @mansiag05
[CP] Fix cuDNN CP LSE dimension bug (pytorch#163231)
b49fdb0 
We should only unsqueeze if necessary.

Fix pytorch#162743

Pull Request resolved: pytorch#163231
Approved by: https://github.com/eqy
ghstack dependencies: pytorch#162539, pytorch#162540, pytorch#162541, pytorch#163115, pytorch#163131
cleonard530 pushed a commit to cleonard530/pytorch that referenced this pull request Sep 22, 2025
@fegin @cleonard530
[CP] Remove the need of recording cp_dim in the global var (pytorch#1…
b3f1d0b 
…62540)

This information can be obtained during the dispatching.

Pull Request resolved: pytorch#162540
Approved by: https://github.com/ezyang, https://github.com/tianyu-l, https://github.com/XilunWu
ghstack dependencies: pytorch#162539
cleonard530 pushed a commit to cleonard530/pytorch that referenced this pull request Sep 22, 2025
@fegin @cleonard530
[CP][BE] Correct the names of some tests (pytorch#162541)
4221f5e 
We are not doing ring attention but only using allgather to do CP for Flex.
Pull Request resolved: pytorch#162541
Approved by: https://github.com/ezyang, https://github.com/Skylion007, https://github.com/tianyu-l, https://github.com/XilunWu
ghstack dependencies: pytorch#162539, pytorch#162540
cleonard530 pushed a commit to cleonard530/pytorch that referenced this pull request Sep 22, 2025
@fegin @cleonard530
[CP][BE] Cosmetic refactors for CP code base (pytorch#163115)
87b1690 
Summary:
This PR is extracted from pytorch#162542, to make the original PR
easier to review. This PR only contains cosmetic changes.

Pull Request resolved: pytorch#163115
Approved by: https://github.com/tianyu-l
ghstack dependencies: pytorch#162539, pytorch#162540, pytorch#162541
cleonard530 pushed a commit to cleonard530/pytorch that referenced this pull request Sep 22, 2025
@fegin @cleonard530
[CP][BE] Correct an incorrect docstring (pytorch#163131)
0d0e820 
Pull Request resolved: pytorch#163131
Approved by: https://github.com/tianyu-l, https://github.com/XilunWu
ghstack dependencies: pytorch#162539, pytorch#162540, pytorch#162541, pytorch#163115
cleonard530 pushed a commit to cleonard530/pytorch that referenced this pull request Sep 22, 2025
@fegin @cleonard530
[CP] Fix cuDNN CP LSE dimension bug (pytorch#163231)
74f16c2 
We should only unsqueeze if necessary.

Fix pytorch#162743

Pull Request resolved: pytorch#163231
Approved by: https://github.com/eqy
ghstack dependencies: pytorch#162539, pytorch#162540, pytorch#162541, pytorch#163115, pytorch#163131
dsashidh pushed a commit to dsashidh/pytorch that referenced this pull request Sep 26, 2025
@fegin @dsashidh
[CP] Remove the need of recording cp_dim in the global var (pytorch#1…
bc3824a 
…62540)

This information can be obtained during the dispatching.

Pull Request resolved: pytorch#162540
Approved by: https://github.com/ezyang, https://github.com/tianyu-l, https://github.com/XilunWu
ghstack dependencies: pytorch#162539
dsashidh pushed a commit to dsashidh/pytorch that referenced this pull request Sep 26, 2025
@fegin @dsashidh
[CP][BE] Correct the names of some tests (pytorch#162541)
035519f 
We are not doing ring attention but only using allgather to do CP for Flex.
Pull Request resolved: pytorch#162541
Approved by: https://github.com/ezyang, https://github.com/Skylion007, https://github.com/tianyu-l, https://github.com/XilunWu
ghstack dependencies: pytorch#162539, pytorch#162540
dsashidh pushed a commit to dsashidh/pytorch that referenced this pull request Sep 26, 2025
@fegin @dsashidh
[CP][BE] Cosmetic refactors for CP code base (pytorch#163115)
2c1c783 
Summary:
This PR is extracted from pytorch#162542, to make the original PR
easier to review. This PR only contains cosmetic changes.

Pull Request resolved: pytorch#163115
Approved by: https://github.com/tianyu-l
ghstack dependencies: pytorch#162539, pytorch#162540, pytorch#162541
dsashidh pushed a commit to dsashidh/pytorch that referenced this pull request Sep 26, 2025
@fegin @dsashidh
[CP][BE] Correct an incorrect docstring (pytorch#163131)
cc3feee 
Pull Request resolved: pytorch#163131
Approved by: https://github.com/tianyu-l, https://github.com/XilunWu
ghstack dependencies: pytorch#162539, pytorch#162540, pytorch#162541, pytorch#163115
dsashidh pushed a commit to dsashidh/pytorch that referenced this pull request Sep 26, 2025
@fegin @dsashidh
[CP] Fix cuDNN CP LSE dimension bug (pytorch#163231)
c49d1a8 
We should only unsqueeze if necessary.

Fix pytorch#162743

Pull Request resolved: pytorch#163231
Approved by: https://github.com/eqy
ghstack dependencies: pytorch#162539, pytorch#162540, pytorch#162541, pytorch#163115, pytorch#163131
@github-actions github-actions bot deleted the gh/fegin/316/head branch October 19, 2025 02:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ezyang ezyang ezyang approved these changes

@XilunWu XilunWu XilunWu approved these changes

@tianyu-l tianyu-l tianyu-l approved these changes

Assignees

No one assigned

Labels

ciflow/inductor ciflow/trunk Trigger trunk jobs on your pull request Merged oncall: distributed Add this issue/PR to distributed oncall triage queue topic: not user facing topic category

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@fegin @pytorchmergebot @ezyang @XilunWu @tianyu-l