Skip to content

[Cutlass] Changes to gemm template for EVT#150907

Closed
mlazos wants to merge 41 commits intogh/mlazos/25/basefrom
gh/mlazos/25/head
Closed

[Cutlass] Changes to gemm template for EVT#150907
mlazos wants to merge 41 commits intogh/mlazos/25/basefrom
gh/mlazos/25/head

Conversation

@pytorch-bot
Copy link

pytorch-bot bot commented Apr 9, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/150907

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (2 Unrelated Failures)

As of commit d209cf1 with merge base 6ae7730 (image):

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

render_evt_fn: Optional[
Callable[
[
"cutlass_library.TileDescription",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

import in a TYPE_CHECKING block

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov

[ghstack-poisoned]
cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov

[ghstack-poisoned]
cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov

[ghstack-poisoned]
cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov

[ghstack-poisoned]
cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov

[ghstack-poisoned]
cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov

[ghstack-poisoned]
cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov

[ghstack-poisoned]
mlazos added a commit that referenced this pull request May 3, 2025
ghstack-source-id: be235da
Pull Request resolved: #150907

update to gemm template

Updates to gemm_template

gemm_template

more gemm_template changes
cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov

[ghstack-poisoned]
Copy link
Collaborator

@Skylion007 Skylion007 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't erase the type unnecessarily

@mlazos mlazos requested review from eellison and henrylhtsang May 5, 2025 17:59
@henrylhtsang
Copy link
Contributor

I feel like this is going to break some tests - did you test it locally?

Also:

  1. Is there a kill switch to not use EVT?
  2. If we are not using EVT, then the template is same as before right?

):
return None

if not self._has_tma_epilogue(op):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove (I add a config for this in the next PR)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 - should we figure out another way here ?

Copy link
Contributor Author

@mlazos mlazos May 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah in the next PR, I check if the winner kernel is a TMA kernel and pass this information to the template buffer node, then during fusion reject if not TMA.

@mlazos
Copy link
Contributor Author

mlazos commented May 6, 2025

I feel like this is going to break some tests - did you test it locally?

Also:

  1. Is there a kill switch to not use EVT?

  2. If we are not using EVT, then the template is same as before right?

If there are no epilogue fusions, the template ends up being exactly the same as it was previously. I will add a killswitch.

I'll run the tests locally before merging as well.

mlazos added a commit that referenced this pull request May 7, 2025
ghstack-source-id: 12640fa
Pull Request resolved: #150907

update to gemm template

Updates to gemm_template

gemm_template

more gemm_template changes
mlazos added a commit that referenced this pull request May 8, 2025
ghstack-source-id: 12640fa
Pull Request resolved: #150907

update to gemm template

Updates to gemm_template

gemm_template

more gemm_template changes
cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov

[ghstack-poisoned]
cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov

[ghstack-poisoned]
@mlazos mlazos mentioned this pull request May 8, 2025
cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov

[ghstack-poisoned]
@pytorchmergebot
Copy link
Collaborator

Starting merge as part of PR stack under #153177

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants