Skip to content

properly compute batch_element_count in warp_softmax#82927

Closed
ngimel wants to merge 1 commit intomasterfrom
ngimel/softmax_fix
Closed

properly compute batch_element_count in warp_softmax#82927
ngimel wants to merge 1 commit intomasterfrom
ngimel/softmax_fix

Conversation

@ngimel
Copy link
Collaborator

@ngimel ngimel commented Aug 6, 2022

Turns out sometimes local_batches can be completely bogus (I thought for masked softmax they are guaranteed to be equal to WARP_BATCH), so to compute real number of elements it needs to be taken into account.
cc @ptrblck

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Aug 6, 2022 •
edited
Loading

🔗 Helpful links

✅ No Failures (0 Pending)

As of commit 34f64a7 (more details on the Dr. CI page):

Expand to see more

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

@ptrblck
Copy link
Collaborator

ptrblck commented Aug 6, 2022

Thank you! Local test passes on my side, too. Running the cherry-pick through CI now.

@erichan1 erichan1 self-requested a review August 8, 2022 17:22
@ngimel
Copy link
Collaborator Author

ngimel commented Aug 8, 2022

@pytorchbot merge -g

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Aug 8, 2022

@pytorchbot successfully started a merge job. Check the current status here

@github-actions
Copy link
Contributor

github-actions bot commented Aug 8, 2022

Hey @ngimel.
You've committed this PR, but it does not have both a 'release notes: ...' and 'topics: ...' label. Please add one of each to the PR. The 'release notes: ...' label should represent the part of PyTorch that this PR changes (fx, autograd, distributed, etc) and the 'topics: ...' label should represent the kind of PR it is (not user facing, new feature, bug fix, perf improvement, etc). The list of valid labels can be found here for the 'release notes: ...' and here for the 'topics: ...'.
For changes that are 'topic: not user facing' there is no need for a release notes label.

facebook-github-bot pushed a commit that referenced this pull request Aug 9, 2022
@facebook-github-bot
properly compute batch_element_count (#82927) (#82927)
e034029 
Summary:
Turns out sometimes local_batches can be completely bogus (I thought for masked softmax they are guaranteed to be equal to WARP_BATCH), so to compute real number of elements it needs to be taken into account.
cc ptrblck

Pull Request resolved: #82927
Approved by: https://github.com/erichan1

Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/88e43ca409dbd3e3da41faefe82821f38b90299a

Reviewed By: seemethere

Differential Revision: D38542295

Pulled By: ngimel

fbshipit-source-id: 2c7b6984bd1f275931cfaeee1b7e390d2a367543
@ngimel ngimel changed the title properly compute batch_element_count properly compute batch_element_count in warp_softmax Oct 13, 2022
@pytorch-bot pytorch-bot bot added the release notes: cuda release notes category label Oct 13, 2022
@github-actions github-actions bot deleted the ngimel/softmax_fix branch February 18, 2024 01:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@erichan1 erichan1 erichan1 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

cla signed Merged release notes: cuda release notes category

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@ngimel @facebook-github-bot @ptrblck @pytorchmergebot @erichan1