Skip to content

add bfloat16 support for kl_div_backward_cuda#77676

Closed
Aidyn-A wants to merge 3 commits intopytorch:masterfrom
Aidyn-A:support_kl_div_grad_with_bfloat16
Closed

add bfloat16 support for kl_div_backward_cuda#77676
Aidyn-A wants to merge 3 commits intopytorch:masterfrom
Aidyn-A:support_kl_div_grad_with_bfloat16

Conversation

@Aidyn-A
Copy link
Collaborator

@Aidyn-A Aidyn-A commented May 17, 2022

This PR adds a feature requested in issue #77375.
kl_div_backward_cuda now supports bfloat16

cc @ngimel @ptrblck @rosrad

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented May 17, 2022 •
edited
Loading

🔗 Helpful links

✅ No Failures (0 Pending)

As of commit 97680c5 (more details on the Dr. CI page):

Expand to see more

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

@anjali411 anjali411 requested review from albanD, jbschlosser and ngimel and removed request for albanD May 23, 2022 03:33
@anjali411 anjali411 added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label May 23, 2022
@ngimel
Copy link
Collaborator

ngimel commented May 23, 2022

Test failures are real, please fix supported dtypes in common_methods_invocations for the failing ops.

@Aidyn-A Aidyn-A marked this pull request as draft May 23, 2022 12:29
@Aidyn-A
Copy link
Collaborator Author

Aidyn-A commented May 23, 2022

I believe now it is ready for review, the failed tests are not related to the changes in this PR

@Aidyn-A Aidyn-A marked this pull request as ready for review May 23, 2022 16:31
@Aidyn-A Aidyn-A requested a review from mruberry as a code owner May 23, 2022 16:31
jbschlosser
jbschlosser approved these changes May 24, 2022
View reviewed changes
Copy link
Contributor

@jbschlosser jbschlosser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed spurious CI failures. LGTM!

@jbschlosser
Copy link
Contributor

jbschlosser commented May 24, 2022

@pytorchbot merge this please

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented May 24, 2022

Merge failed due to Refusing to merge as mandatory check(s)linux-docs / build-docs (cpp), linux-docs / build-docs (python) failed for rule superuser
Raised by https://github.com/pytorch/pytorch/actions/runs/2379559187

@jbschlosser
Copy link
Contributor

jbschlosser commented May 24, 2022

@Aidyn-A Maybe a rebase will fix the CI failures?

@ngimel
Copy link
Collaborator

ngimel commented May 24, 2022

@pytorchbot merge on green

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented May 24, 2022

Merge failed due to Refusing to merge as mandatory check(s) linux-docs / build-docs (cpp), linux-docs / build-docs (python), win-vs2019-cuda11.3-py3 / build are pending/not yet run for rule superuser
Raised by https://github.com/pytorch/pytorch/actions/runs/2379769618

@Aidyn-A
Copy link
Collaborator Author

Aidyn-A commented May 24, 2022

@jbschlosser, rebase definitely worked. Thanks!

@ngimel
Copy link
Collaborator

ngimel commented May 24, 2022

@pytorchbot merge this

@github-actions
Copy link
Contributor

github-actions bot commented May 24, 2022

Hey @Aidyn-A.
You've committed this PR, but it does not have both a 'release notes: ...' and 'topics: ...' label. Please add one of each to the PR. The 'release notes: ...' label should represent the part of PyTorch that this PR changes (fx, autograd, distributed, etc) and the 'topics: ...' label should represent the kind of PR it is (not user facing, new feature, bug fix, perf improvement, etc). The list of valid labels can be found here for the 'release notes: ...' and here for the 'topics: ...'.
For changes that are 'topic: not user facing' there is no need for a release notes label.

facebook-github-bot pushed a commit that referenced this pull request May 26, 2022
@Aidyn-A @facebook-github-bot
add bfloat16 support for kl_div_backward_cuda (#77676) (#77676)
7237dbb 
Summary:
This PR adds a feature requested in issue #77375.
`kl_div_backward_cuda` now supports `bfloat16`

cc ngimel ptrblck rosrad

Pull Request resolved: #77676
Approved by: https://github.com/jbschlosser

Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/f37ce948ff25bf70a52ab4b327c82925ddb4aa86

Reviewed By: mehtanirav

Differential Revision: D36668740

fbshipit-source-id: 0f171ac2fdb66931f0a7ffe73a97517fe2abad02
@jbschlosser jbschlosser mentioned this pull request Jun 1, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jbschlosser jbschlosser jbschlosser approved these changes

@ngimel ngimel Awaiting requested review from ngimel

@mruberry mruberry Awaiting requested review from mruberry

Assignees

No one assigned

Labels

cla signed Merged open source triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@Aidyn-A @facebook-github-bot @ngimel @jbschlosser @pytorchmergebot @anjali411 @pytorchbot