Skip to content

Backport ZeRO1: Add bucketting logic to control the size of tensors for all-gather/reduce-scatter (#6025)#6806

Merged
lsy323 merged 1 commit intor2.3from
JackCaoG/backport_aws_zero1
Mar 25, 2024
Merged

Backport ZeRO1: Add bucketting logic to control the size of tensors for all-gather/reduce-scatter (#6025)#6806
lsy323 merged 1 commit intor2.3from
JackCaoG/backport_aws_zero1

Conversation

@JackCaoG
Copy link
Copy Markdown
Collaborator

No description provided.

…ther/reduce-scatter (#6025)

Co-authored-by: Rahul Solanki <rhsoln@amazon.com>
Co-authored-by: guangtai <guangtai@amazon.com>
Co-authored-by: Amithrajith Mamidala <amithrm@amazon.com>
@JackCaoG JackCaoG requested a review from lsy323 March 22, 2024 21:25
@JackCaoG
Copy link
Copy Markdown
Collaborator Author

@lsy323 can you take a look at this pr and merge if everything looks ok?

@lsy323 lsy323 merged commit a805505 into r2.3 Mar 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants