Skip to content

[PP] Customize pipeline's submod name#164037

Closed
kwen2501 wants to merge 3 commits intogh/kwen2501/266/basefrom
gh/kwen2501/266/head
Closed

[PP] Customize pipeline's submod name#164037
kwen2501 wants to merge 3 commits intogh/kwen2501/266/basefrom
gh/kwen2501/266/head

Conversation

@kwen2501
Copy link
Collaborator

@kwen2501 kwen2501 commented Sep 27, 2025

[ghstack-poisoned]
@pytorch-bot
Copy link

pytorch-bot bot commented Sep 27, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/164037

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit cea5fb3 with merge base 8d81564 (image):

FLAKY - The following job failed but was likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

kwen2501 added a commit that referenced this pull request Sep 27, 2025
ghstack-source-id: a5a0426
Pull-Request: #164037
@pytorch-bot pytorch-bot bot added the oncall: distributed Add this issue/PR to distributed oncall triage queue label Sep 27, 2025
@kwen2501 kwen2501 requested a review from H-Huang September 27, 2025 05:09
@kwen2501 kwen2501 added module: pipelining Pipeline Parallelism release notes: distributed (pipeline) release notes category labels Sep 27, 2025
@kwen2501 kwen2501 requested a review from zou3519 September 27, 2025 05:12
[ghstack-poisoned]
[ghstack-poisoned]
Copy link
Member

@H-Huang H-Huang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

# TODO: what does split do with module invocations? does it move the modules
# into the submodules?
split = split_module(traced, mod, split_callback) # type: ignore[arg-type]
split = split_module(traced, mod, split_callback, partition_affix="pp") # type: ignore[arg-type]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure if we have tests validating the submod name after splitting, might be good to have. Can do it in a followup PR

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are internal settings. And users are supposed to get submodules via:
pipe.get_stage_module(i).
If existing tests passed, we should be fine.

@kwen2501
Copy link
Collaborator Author

@pytorchbot merge

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Sep 29, 2025
@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@pytorchmergebot
Copy link
Collaborator

The merge job was canceled or timed out. This most often happen if two merge requests were issued for the same PR, or if merge job was waiting for more than 6 hours for tests to finish. In later case, please do not hesitate to reissue the merge command
For more information see pytorch-bot wiki.

@kwen2501
Copy link
Collaborator Author

@pytorchbot merge -f "previous merge timed out"

@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

pytorchmergebot added a commit that referenced this pull request Oct 1, 2025
This reverts commit 704cd77.

Reverted #164037 on behalf of https://github.com/yangw-dev due to internal build failed Buck build failed for this target, and is likely caused by your changes. ([comment](#164035 (comment)))
@pytorchmergebot
Copy link
Collaborator

@kwen2501 your PR has been reverted as part of the stack under #164035.

@pytorchmergebot pytorchmergebot added Reverted ci-no-td Do not run TD on this PR labels Oct 1, 2025
@pytorchmergebot
Copy link
Collaborator

@pytorchbot successfully started a revert job. Check the current status here.
Questions? Feedback? Please reach out to the PyTorch DevX Team

@pytorchmergebot
Copy link
Collaborator

Reverting PR 164037 failed

Reason: Comment with id 3357166994 not found

Details for Dev Infra team Raised by workflow job

@kwen2501
Copy link
Collaborator Author

kwen2501 commented Oct 1, 2025

@pytorchbot merge -f "internal signal succeeded upon retry"

@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

Chao1Han pushed a commit to Chao1Han/pytorch that referenced this pull request Oct 21, 2025
This reverts commit 704cd77.

Reverted pytorch#164037 on behalf of https://github.com/yangw-dev due to internal build failed Buck build failed for this target, and is likely caused by your changes. ([comment](pytorch#164035 (comment)))
Chao1Han pushed a commit to Chao1Han/pytorch that referenced this pull request Oct 21, 2025
Changing PP submodules' name from `submod_i` to `submod_pp_i` to distinguish from the submodule created by HOP.

Pull Request resolved: pytorch#164037
Approved by: https://github.com/H-Huang
ghstack dependencies: pytorch#164045, pytorch#164035
@github-actions github-actions bot deleted the gh/kwen2501/266/head branch November 1, 2025 02:19
Khanaksahu pushed a commit to Khanaksahu/pytorch-fork that referenced this pull request Nov 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-no-td Do not run TD on this PR ciflow/trunk Trigger trunk jobs on your pull request Merged module: pipelining Pipeline Parallelism oncall: distributed Add this issue/PR to distributed oncall triage queue release notes: distributed (pipeline) release notes category Reverted

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants