[varlen_attn for inference] remove unnecessary tensor creation by liangel-02 · Pull Request #176723 · pytorch/pytorch

liangel-02 · 2026-03-06T16:38:52Z

Stack from ghstack (oldest at bottom):

[ghstack-poisoned]

pytorch-bot · 2026-03-06T16:38:57Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/176723

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (2 Unrelated Failures)

As of commit c9ed5a1 with merge base 4bc9d7f ():

FLAKY - The following job failed but was likely due to flakiness present on trunk:

pull / linux-jammy-py3.10-gcc11 / test (distributed, 1, 2, linux.2xlarge) (gh) (similar failure)
test/distributed/tensor/test_dtensor_ops.py::TestLocalDTensorOpsCPU::test_dtensor_op_db_nanmean_cpu_float32

BROKEN TRUNK - The following job failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

pull / linux-jammy-py3.10-gcc11 / test (distributed, 2, 2, linux.2xlarge) (gh) (trunk failure)
'Test'

This comment was automatically generated by Dr. CI and updates every 15 minutes.

pytorch-bot · 2026-03-06T16:39:00Z

This PR needs a `release notes:` label

If your changes are user facing and intended to be a part of release notes, please use a label starting with release notes:.

If not, please add the topic: not user facing label.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "topic: not user facing"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

ghstack-source-id: 8ac96c2 Pull Request resolved: #176723

Skylion007 · 2026-03-06T16:45:53Z

torch/nn/attention/_fa3.py

    alibi_slopes: torch.Tensor | None = None,
    out: torch.Tensor | None = None,
    block_table: torch.Tensor | None = None,
+    compute_auxiliary: bool = True,


This smells like Boolean trap... do we need to add this for every other sdpa derivative?

this is private impl so should be fine and it is specific to varlen API. Once we get fa4 wired up we should probably do teh same

liangel-02 · 2026-03-06T20:35:29Z

@pytorchbot merge -i

pytorchmergebot · 2026-03-06T20:37:47Z

Merge started

Your change will be merged while ignoring the following 2 checks: trunk / win-vs2022-cpu-py3 / build, trunk / win-vs2022-cuda12.8-py3 / build

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot · 2026-03-07T02:35:53Z

The merge job was canceled or timed out. This most often happen if two merge requests were issued for the same PR, or if merge job was waiting for more than 6 hours for tests to finish. In later case, please do not hesitate to reissue the merge command
For more information see pytorch-bot wiki.

liangel-02 · 2026-03-07T06:30:24Z

@pytorchbot merge -i

pytorchmergebot · 2026-03-07T06:32:34Z

Merge started

Your change will be merged while ignoring the following 1 checks: trunk / macos-py3-arm64 / test (mps, 1, 1, macos-m2-15)

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

#176723)" This reverts commit c405acd. Reverted #176723 on behalf of https://github.com/zou3519 due to sorry I think this broke inductor rocm ([comment](#175897 (comment)))

pytorchmergebot · 2026-03-07T16:30:23Z

@liangel-02 your PR has been reverted as part of the stack under #175897.

pytorchmergebot · 2026-03-10T01:10:18Z

@liangel-02 your PR has been reverted as part of the stack under #175924.

#176723)" This reverts commit 26dddb9. Reverted #176723 on behalf of https://github.com/huydhn due to Sorry for reverting your change but a bunch of internal builds need to be updated to unblock this change D95758397 ([comment](#175924 (comment)))

…tion" [ghstack-poisoned]

ghstack-source-id: 6c602d7 Pull Request resolved: #176723

…tion" [ghstack-poisoned]

ghstack-source-id: c7d2b3d Pull Request resolved: #176723

…tion" [ghstack-poisoned]

ghstack-source-id: c7d2b3d Pull Request resolved: #176723

liangel-02 · 2026-03-10T19:36:50Z

@liangel-02 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

…tion" Differential Revision: [D95996397](https://our.internmc.facebook.com/intern/diff/D95996397) [ghstack-poisoned]

Pull Request resolved: #176723 @imported-using-ghimport Differential Revision: [D95996397](https://our.internmc.facebook.com/intern/diff/D95996397/) ghstack-source-id: 350342133

…tion" Differential Revision: [D95996397](https://our.internmc.facebook.com/intern/diff/D95996397) [ghstack-poisoned]

ghstack-source-id: 830370c Pull Request resolved: #176723

liangel-02 · 2026-03-11T20:19:47Z

@pytorchbot merge -f "ignoring meta internal-only changes check" cc @huydhn

pytorch-bot · 2026-03-11T20:19:51Z

❌ 🤖 pytorchbot command failed:

@pytorchbot: error: unrecognized arguments: cc @huydhn

usage: @pytorchbot [-h]
                   
                   {merge,revert,rebase,label,drci,lint,fix-lint,apply-lint,cherry-pick}
                   ...

Try @pytorchbot --help for more info.

liangel-02 · 2026-03-11T20:20:07Z

@pytorchbot merge -f "ignoring meta internal-only changes check"

pytorchmergebot · 2026-03-11T20:22:01Z

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot · 2026-03-11T20:22:28Z

Merge failed

Reason: This PR has internal changes and must be landed via Phabricator! Please try reimporting/rexporting the PR!

Details for Dev Infra team

Raised by workflow job

huydhn · 2026-03-11T20:28:19Z

@pytorchbot merge -f 'Ignoring meta internal-only changes check'

pytorchmergebot · 2026-03-11T20:30:13Z

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[varlen_attn for inference] remove unnecessary tensor creation

b7996b8

[ghstack-poisoned]

liangel-02 requested review from albanD, jbschlosser and mikaylagawarecki as code owners March 6, 2026 16:38

This was referenced Mar 6, 2026

[varlen_attn for inference] add seqused_k #175897

Closed

[varlen_attn for inference] add page_table #175924

Closed

[varlen_attn for inference] add test to aot_inductor #175936

Closed

liangel-02 mentioned this pull request Mar 6, 2026

[varlen_attn for inference] add out variant #176015

Closed

liangel-02 added a commit that referenced this pull request Mar 6, 2026

[varlen_attn for inference] remove unnecessary tensor creation

a0cf336

ghstack-source-id: 8ac96c2 Pull Request resolved: #176723

liangel-02 requested a review from drisspg March 6, 2026 16:39

liangel-02 added release notes: nn release notes category ciflow/trunk Trigger trunk jobs on your pull request labels Mar 6, 2026

Skylion007 reviewed Mar 6, 2026

View reviewed changes

liangel-02 mentioned this pull request Mar 6, 2026

[rl][poc] Native PyTorch Varlen Attention for inference. pytorch/torchtitan#2364

Open

drisspg approved these changes Mar 6, 2026

View reviewed changes

pytorchmergebot added the merging label Mar 6, 2026

pytorchmergebot added the Merged label Mar 7, 2026

pytorchmergebot closed this in c405acd Mar 7, 2026

pytorchmergebot removed the merging label Mar 7, 2026

pytorchmergebot added the Reverted label Mar 7, 2026

pytorchmergebot removed the merging label Mar 8, 2026

pytorchmergebot reopened this Mar 10, 2026

Update on "[varlen_attn for inference] remove unnecessary tensor crea…

ab51f16

…tion" [ghstack-poisoned]

liangel-02 added a commit that referenced this pull request Mar 10, 2026

[varlen_attn for inference] remove unnecessary tensor creation

69ff83a

ghstack-source-id: 6c602d7 Pull Request resolved: #176723

Update on "[varlen_attn for inference] remove unnecessary tensor crea…

c63d4c2

…tion" [ghstack-poisoned]

liangel-02 added a commit that referenced this pull request Mar 10, 2026

[varlen_attn for inference] remove unnecessary tensor creation

1256bfb

ghstack-source-id: c7d2b3d Pull Request resolved: #176723

Update on "[varlen_attn for inference] remove unnecessary tensor crea…

eac52b1

…tion" [ghstack-poisoned]

liangel-02 added a commit that referenced this pull request Mar 10, 2026

[varlen_attn for inference] remove unnecessary tensor creation

18b73cf

ghstack-source-id: c7d2b3d Pull Request resolved: #176723

Update on "[varlen_attn for inference] remove unnecessary tensor crea…

89f1069

…tion" Differential Revision: [D95996397](https://our.internmc.facebook.com/intern/diff/D95996397) [ghstack-poisoned]

meta-codesync bot added fb-exported meta-exported labels Mar 11, 2026

Update on "[varlen_attn for inference] remove unnecessary tensor crea…

c9ed5a1

…tion" Differential Revision: [D95996397](https://our.internmc.facebook.com/intern/diff/D95996397) [ghstack-poisoned]

liangel-02 added a commit that referenced this pull request Mar 11, 2026

[varlen_attn for inference] remove unnecessary tensor creation

4ab09fd

ghstack-source-id: 830370c Pull Request resolved: #176723

pytorchmergebot added the merging label Mar 11, 2026

pytorchmergebot removed the merging label Mar 11, 2026

pytorchmergebot added the merging label Mar 11, 2026

pytorchmergebot closed this in f1f3d70 Mar 11, 2026

pytorchmergebot removed the merging label Mar 11, 2026

Conversation

liangel-02 commented Mar 6, 2026 • edited by huydhn Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/176723

✅ You can merge normally! (2 Unrelated Failures)

Uh oh!

pytorch-bot bot commented Mar 6, 2026

This PR needs a release notes: label

Uh oh!

Skylion007 Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

drisspg Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

liangel-02 commented Mar 6, 2026

Uh oh!

pytorchmergebot commented Mar 6, 2026

Merge started

Uh oh!

pytorchmergebot commented Mar 7, 2026

Uh oh!

liangel-02 commented Mar 7, 2026

Uh oh!

pytorchmergebot commented Mar 7, 2026

Merge started

Uh oh!

pytorchmergebot commented Mar 7, 2026

Uh oh!

pytorchmergebot commented Mar 10, 2026

Uh oh!

liangel-02 commented Mar 10, 2026

Uh oh!

liangel-02 commented Mar 11, 2026

Uh oh!

pytorch-bot bot commented Mar 11, 2026

Uh oh!

liangel-02 commented Mar 11, 2026

Uh oh!

pytorchmergebot commented Mar 11, 2026

Merge started

Uh oh!

pytorchmergebot commented Mar 11, 2026

Merge failed

Uh oh!

huydhn commented Mar 11, 2026

Uh oh!

pytorchmergebot commented Mar 11, 2026

Merge started

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

liangel-02 commented Mar 6, 2026 •

edited by huydhn

Loading

pytorch-bot bot commented Mar 6, 2026 •

edited

Loading

This PR needs a `release notes:` label