Update FlexAttention with masking semantic by drisspg · Pull Request #133373 · pytorch/pytorch

drisspg · 2024-08-13T22:55:39Z

Stack from ghstack (oldest at bottom):

-> Update FlexAttention with masking semantic #133373

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @ColinPeppler @amjames @desertfire @chauhang @mikaylagawarecki

[ghstack-poisoned]

pytorch-bot · 2024-08-13T22:55:42Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/133373

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 5dd6433 with merge base 14ddd93 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

test/inductor/test_flex_attention.py

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

drisspg · 2024-08-13T23:58:38Z

@pytorchbot merge

pytorchmergebot · 2024-08-14T00:00:29Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot · 2024-08-14T00:16:25Z

The merge job was canceled or timed out. This most often happen if two merge requests were issued for the same PR, or if merge job was waiting for more than 6 hours for tests to finish. In later case, please do not hesitate to reissue the merge command
For more information see pytorch-bot wiki.

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

Chillee

Can you add a test like this?

out, lse = torch.compile(flex_attention)(q, k, v, score_mod=mask, return_lse=True)
lse.sum().backward()

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

ghstack-source-id: b682009 Pull Request resolved: #133373

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

pytorchmergebot · 2024-08-22T13:33:46Z

Reverting PR 133373 failed

Reason: Command git -C /home/runner/work/pytorch/pytorch revert --no-edit 5a7b544e5c3e37bea62c6a231f6230c004a33d38 returned non-zero exit code 1

Auto-merging test/inductor/test_flex_attention.py
CONFLICT (content): Merge conflict in test/inductor/test_flex_attention.py
Auto-merging test/inductor/test_flex_decoding.py
Auto-merging torch/_higher_order_ops/flex_attention.py
Auto-merging torch/_inductor/kernel/flex_attention.py
Auto-merging torch/_inductor/kernel/flex_decoding.py
error: could not revert 5a7b544e5c... Update FlexAttention with masking semantic (#133373)
hint: After resolving the conflicts, mark them with
hint: "git add/rm <pathspec>", then run
hint: "git revert --continue".
hint: You can instead skip this commit with "git revert --skip".
hint: To abort and get back to the state before "git revert",
hint: run "git revert --abort".
hint: Disable this message with "git config advice.mergeConflict false"

Details for Dev Infra team

Raised by workflow job

This reverts commit e847b6b. Reverted #134043 on behalf of https://github.com/jeanschmidt due to Need to revert, in order to be able to revert #133373, feel free to reland this after solving conflicts ([comment](#134043 (comment)))

# Summary Adds the option for the head dims to be different between QK and V tensors. Fixes issue: #133674 V_DIM > QK_DIM is blocked by landing: triton-lang/triton#4138 / triton-lang/triton#4540 Into PyTorch's triton branch. Pull Request resolved: #134043 Approved by: https://github.com/Chillee

jeanschmidt · 2024-08-22T13:44:54Z

@pytorchbot revert -m "Broke internal test/inductor signals, see D61611729" -c ghfirst

pytorchmergebot · 2024-08-22T13:47:16Z

@pytorchbot successfully started a revert job. Check the current status here.
Questions? Feedback? Please reach out to the PyTorch DevX Team

This reverts commit 5a7b544. Reverted #133373 on behalf of https://github.com/jeanschmidt due to Broke internal test/inductor signals, see D61611729 ([comment](#133373 (comment)))

pytorchmergebot · 2024-08-22T13:47:29Z

@drisspg your PR has been successfully reverted.

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire chauhang mikaylagawarecki [ghstack-poisoned]

ghstack-source-id: 0565f1e Pull Request resolved: #133373

drisspg · 2024-08-22T17:30:03Z

@pytorchbot merge

pytorchmergebot · 2024-08-22T17:31:44Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

drisspg · 2024-08-22T22:48:33Z

@pytorchbot merge -f "very slow queue wait time for rocm jobs"

pytorchmergebot · 2024-08-22T22:48:52Z

The merge job was canceled or timed out. This most often happen if two merge requests were issued for the same PR, or if merge job was waiting for more than 6 hours for tests to finish. In later case, please do not hesitate to reissue the merge command
For more information see pytorch-bot wiki.

pytorchmergebot · 2024-08-22T22:50:24Z

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

…134043)" This reverts commit e847b6b. Reverted pytorch#134043 on behalf of https://github.com/jeanschmidt due to Need to revert, in order to be able to revert pytorch#133373, feel free to reland this after solving conflicts ([comment](pytorch#134043 (comment)))

This reverts commit 5a7b544. Reverted pytorch#133373 on behalf of https://github.com/jeanschmidt due to Broke internal test/inductor signals, see D61611729 ([comment](pytorch#133373 (comment)))

update FlexAttention with masking semantic

1c1ec63

[ghstack-poisoned]

drisspg mentioned this pull request Aug 13, 2024

Update fused kernels and call _safe_softmax from SDPA #131863

Closed

pytorch-bot bot added the module: inductor label Aug 13, 2024

drisspg requested review from Chillee and yanboliang and removed request for Chillee August 13, 2024 23:02

drisspg added topic: not user facing topic category module: multi-headed-attention labels Aug 13, 2024

yanboliang approved these changes Aug 13, 2024

View reviewed changes

test/inductor/test_flex_attention.py Show resolved Hide resolved

Update on "update FlexAttention with masking semantic"

db50b56

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Aug 13, 2024

pytorchmergebot added the merging label Aug 14, 2024

Update on "update FlexAttention with masking semantic"

90181cb

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

pytorch-bot bot added the ciflow/inductor label Aug 14, 2024

drisspg changed the title ~~update FlexAttention with masking semantic~~ Update FlexAttention with masking semantic Aug 14, 2024

Chillee reviewed Aug 14, 2024

View reviewed changes

Chillee mentioned this pull request Aug 14, 2024

made some args optional in create_mask #133413

Closed

Update on "Update FlexAttention with masking semantic"

34521cb

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

Update on "Update FlexAttention with masking semantic"

a9314f0

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

drisspg added a commit that referenced this pull request Aug 14, 2024

update FlexAttention with masking semantic

5627a91

ghstack-source-id: b682009 Pull Request resolved: #133373

Update on "Update FlexAttention with masking semantic"

47cf00d

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

jeanschmidt mentioned this pull request Aug 22, 2024

[FlexAttention] Enable different qk and v head-dims #134043

Closed

pytorchmergebot added the Reverted label Aug 22, 2024

pytorchmergebot reopened this Aug 22, 2024

Update on "Update FlexAttention with masking semantic"

5dd6433

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire chauhang mikaylagawarecki [ghstack-poisoned]

drisspg added a commit that referenced this pull request Aug 22, 2024

update FlexAttention with masking semantic

6dd4d82

ghstack-source-id: 0565f1e Pull Request resolved: #133373

pytorchmergebot added the merging label Aug 22, 2024

pytorchmergebot closed this in 629bd6f Aug 22, 2024

pytorchmergebot removed the merging label Aug 22, 2024

github-actions bot deleted the gh/drisspg/30/head branch October 1, 2024 02:14

Conversation

drisspg commented Aug 13, 2024 • edited by pytorch-bot bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Aug 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/133373

✅ No Failures

Uh oh!

Uh oh!

drisspg commented Aug 13, 2024

Uh oh!

pytorchmergebot commented Aug 14, 2024

Merge started

Uh oh!

pytorchmergebot commented Aug 14, 2024

Uh oh!

Chillee left a comment

Choose a reason for hiding this comment

Uh oh!

pytorchmergebot commented Aug 22, 2024

Reverting PR 133373 failed

Uh oh!

jeanschmidt commented Aug 22, 2024

Uh oh!

pytorchmergebot commented Aug 22, 2024

Uh oh!

pytorchmergebot commented Aug 22, 2024

Uh oh!

drisspg commented Aug 22, 2024

Uh oh!

pytorchmergebot commented Aug 22, 2024

Merge started

Uh oh!

drisspg commented Aug 22, 2024

Uh oh!

pytorchmergebot commented Aug 22, 2024

Uh oh!

pytorchmergebot commented Aug 22, 2024

Merge started

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

drisspg commented Aug 13, 2024 •

edited by pytorch-bot bot

Loading

pytorch-bot bot commented Aug 13, 2024 •

edited

Loading