Skip to content

perf(sgl-kernel): expose get_scheduler_metadata for FA3 decode optimization#21103

Merged
Qiaolin-Yu merged 3 commits intosgl-project:mainfrom
zminglei:zminglei/fa3-scheduler-metadata-sgl-kernel
Mar 25, 2026
Merged

perf(sgl-kernel): expose get_scheduler_metadata for FA3 decode optimization#21103
Qiaolin-Yu merged 3 commits intosgl-project:mainfrom
zminglei:zminglei/fa3-scheduler-metadata-sgl-kernel

Conversation

@zminglei
Copy link
Copy Markdown
Collaborator

@zminglei zminglei commented Mar 21, 2026

Summary

Register mha_fwd_get_scheduler_metadata as a torch op in sgl_kernel namespace. This function precomputes FA3 tile scheduling metadata so that the prepare_varlen_num_blocks kernel does not need to run per-layer during decode.

The C++ symbol already exists in flash_ops.so (compiled from sgl-attn) but was not exposed as a torch op.

Changes (3 files, sgl-kernel only)

  • sgl-kernel/include/sgl_flash_kernel_ops.h: Declare mha_fwd_get_scheduler_metadata
  • sgl-kernel/csrc/flash_extension.cc: Register sgl_kernel.get_scheduler_metadata torch op
  • sgl-kernel/python/sgl_kernel/flash_attn.py: Add Python get_scheduler_metadata() wrapper

This is part 1 of 2: sgl-kernel changes only (backward compatible, no behavior change).
Part 2 (sglang Python changes to use this op): #21104

…zation

Register mha_fwd_get_scheduler_metadata as a torch op in sgl_kernel namespace.
This function precomputes FA3 tile scheduling metadata so that the
prepare_varlen_num_blocks kernel does not need to run per-layer during decode.

The C++ symbol already exists in flash_ops.so (compiled from sgl-attn) but
was not exposed as a torch op.

Changes:
- sgl_flash_kernel_ops.h: declare mha_fwd_get_scheduler_metadata
- flash_extension.cc: register sgl_kernel.get_scheduler_metadata torch op
- flash_attn.py: add Python get_scheduler_metadata() wrapper

This is part 1 of 2: sgl-kernel changes only (backward compatible).
Part 2 (sglang Python changes) will use this op when available.
@gemini-code-assist
Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a performance optimization for Flash Attention v3 (FA3) decode operations within the sgl-kernel library. By exposing a new Torch operation, get_scheduler_metadata, the system can now precompute tile scheduling metadata once per batch, significantly reducing redundant computations that previously occurred on a per-layer basis. This change lays the groundwork for more efficient decode processes in subsequent updates.

Highlights

  • Torch Op Exposure: Exposed mha_fwd_get_scheduler_metadata as a new sgl_kernel.get_scheduler_metadata Torch operation.
  • Performance Optimization: Enabled precomputation of FA3 tile scheduling metadata to prevent the prepare_varlen_num_blocks kernel from running per-layer during decode.
  • Modular Implementation: Implemented as the first part of a two-part change, focusing solely on sgl-kernel modifications without altering existing behavior.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request exposes a new operation get_scheduler_metadata to optimize FA3 decoding by precomputing tile scheduling. The changes to the C++ source and header files are well-implemented and consistent with the existing codebase. However, the new Python wrapper for this operation in sgl-kernel/python/sgl_kernel/flash_attn.py is incomplete. It omits several parameters that are available in the C++ operation. This could lead to incorrect behavior or limit the utility of the function, especially since its docstring suggests it can be used with functions that rely on these missing parameters. I've provided a suggestion to make the Python wrapper's signature complete.

Comment thread sgl-kernel/python/sgl_kernel/flash_attn.py
@zminglei zminglei marked this pull request as ready for review March 22, 2026 21:03
@zminglei
Copy link
Copy Markdown
Collaborator Author

/tag-and-rerun-ci

@zminglei zminglei force-pushed the zminglei/fa3-scheduler-metadata-sgl-kernel branch from cbfb43d to f216b9d Compare March 23, 2026 00:35
@github-actions github-actions Bot added the npu label Mar 23, 2026
@zminglei zminglei force-pushed the zminglei/fa3-scheduler-metadata-sgl-kernel branch from f216b9d to 9781ad5 Compare March 23, 2026 00:36
@zminglei
Copy link
Copy Markdown
Collaborator Author

zminglei commented Mar 23, 2026

/rerun-failed-ci retry

@zminglei zminglei removed the npu label Mar 23, 2026
@zminglei zminglei requested a review from HaiShaw as a code owner March 24, 2026 05:40
Copy link
Copy Markdown
Collaborator

@Qiaolin-Yu Qiaolin-Yu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. CI Tracked in #20943

@Qiaolin-Yu Qiaolin-Yu merged commit a12fea2 into sgl-project:main Mar 25, 2026
231 of 285 checks passed
satyamk7054 pushed a commit to satyamk7054/sglang that referenced this pull request Apr 3, 2026
JustinTong0323 pushed a commit to JustinTong0323/sglang that referenced this pull request Apr 7, 2026
yhyang201 pushed a commit to yhyang201/sglang that referenced this pull request Apr 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants