Bugfix: Change get() -> GetDLTensorPtr() in cutlass FusedMoE validations by amirkl94 · Pull Request #1995 · flashinfer-ai/flashinfer

amirkl94 · 2025-10-28T09:32:15Z

📌 Description

Using different API after apach-tvm-ffi version bump.

Summary by CodeRabbit

Bug Fixes
- Improved null-pointer validation for FP8 quantization tensors used during inference, increasing robustness and reducing risk of runtime errors.

Signed-off-by: Amir Klein <203507526+amirkl94@users.noreply.github.com>

gemini-code-assist · 2025-10-28T09:32:27Z

Summary of Changes

Hello @amirkl94, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a compatibility issue arising from an apach-tvm-ffi library version update. It specifically modifies the tensor validation logic within the Cutlass FusedMoE backend, replacing deprecated get() calls with the new GetDLTensorPtr() method. This ensures that the validation checks continue to function correctly and the system remains compatible with the updated library, preventing potential runtime errors.

Highlights

API Update: Updated the API calls from get() to GetDLTensorPtr() in response to a version bump of the apach-tvm-ffi library.
Bugfix: Resolved a bug in the Cutlass FusedMoE validations by adapting to the new tensor pointer retrieval method.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

amirkl94 · 2025-10-28T09:32:36Z

/bot run

coderabbitai · 2025-10-28T09:32:42Z

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

Walkthrough

Updated null-pointer checks in the Cutlass FusedMoE FP8 quantization validation to use GetDLTensorPtr() instead of get() for four tensors: fc1_dequant, fc2_quant, fc2_dequant, and fc1_input_dequant.

Changes

Cohort / File(s)	Summary
Quantization validation pointer access `csrc/fused_moe/cutlass_backend/flashinfer_cutlass_fused_moe_sm100_binding.cu`	Replaced null-checks that used `get()` with `GetDLTensorPtr()` for `fc1_dequant`, `fc2_quant`, `fc2_dequant`, and `fc1_input_dequant` pointers; type/rank/shape validations unchanged

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~5 minutes

Single-file, small repetitive change (pointer accessor swap).
Review focus: ensure no behavioral change and that GetDLTensorPtr() has the same lifetime/ownership assumptions as get().

Possibly related PRs

Fix: Verify scales are not None for Cutlass FP8 FusedMoE #1961 — Modifies the same FP8 quantization validation checks for fc1_dequant, fc2_quant, fc2_dequant, and fc1_input_dequant.

Suggested reviewers

yzh119
yongwww
djmmoss
cyx-6

Poem

🐰 Small paws tap keys in morning light,
Four pointers checked, now tidy and tight.
From get() to GetDLTensorPtr() they hop,
Consistent steps — a clean, gentle drop. ✨

Pre-merge checks and finishing touches

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Description Check	⚠️ Warning	The pull request description is largely incomplete compared to the repository's template requirements. While the Description section is present, it only contains a single brief line about an API version bump. The description is missing critical required sections including the Related Issues section, the entire Pull Request Checklist (Pre-commit Checks, Tests), and Reviewer Notes. The minimal description provides insufficient context about the changes and does not follow the established template structure.	The author should expand the pull request description by filling out the missing template sections. Add a Related Issues section (linking to any issue this bugfix addresses), complete the Pull Request Checklist including confirmation that pre-commit hooks were run and tests were added/updated, and optionally add Reviewer Notes highlighting that this is an API migration due to the apach-tvm-ffi version bump. The Description section could also be expanded to provide more detail about why this API change was necessary and what problem it solves.
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	You can run `@coderabbitai generate docstrings` to improve docstring coverage.

✅ Passed checks (1 passed)

Check name	Status	Explanation
Title Check	✅ Passed	The pull request title "Bugfix: Change get() -> GetDLTensorPtr() in cutlass FusedMoE validations" is directly related to the main change described in the raw summary. The title accurately reflects the primary modification—replacing get() calls with GetDLTensorPtr() in FP8 quantization tensor pointer validation—and is concise, specific, and free of vague terms or noise. A teammate scanning commit history would clearly understand the purpose of this change from the title alone.

✨ Finishing touches

📝 Generate docstrings

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 0a5a368 and c8df0a9.

📒 Files selected for processing (1)

csrc/fused_moe/cutlass_backend/flashinfer_cutlass_fused_moe_sm100_binding.cu (1 hunks)

🚧 Files skipped from review as they are similar to previous changes (1)

csrc/fused_moe/cutlass_backend/flashinfer_cutlass_fused_moe_sm100_binding.cu

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

flashinfer-bot · 2025-10-28T09:32:46Z

@amirkl94 is not authorized to trigger this CI job. cc: @yzh119, @sricketts, @yongwww

gemini-code-assist

Code Review

This pull request correctly updates the API call from .get() to .GetDLTensorPtr() for tvm::ffi::Tensor objects, which is necessary due to a version bump in the apache-tvm-ffi library. The change is straightforward and correct. I've identified a minor typo in an error message on one of the modified lines and have provided a suggestion to fix it.

Signed-off-by: Amir Klein <203507526+amirkl94@users.noreply.github.com>

coderabbitai

Actionable comments posted: 0

🧹 Nitpick comments (1)

csrc/fused_moe/cutlass_backend/flashinfer_cutlass_fused_moe_sm100_binding.cu (1)
829-829: Fix typo in error message.

The error message contains a typo: "fc2_dequant_dequant" should be "fc2_dequant". This is a pre-existing issue, not introduced by this PR.

Apply this diff to fix the typo:
-          << "Expecting fc2_dequant_dequant to be non null";
+          << "Expecting fc2_dequant to be non null";

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 135d7bc and 0a5a368.

📒 Files selected for processing (1)

csrc/fused_moe/cutlass_backend/flashinfer_cutlass_fused_moe_sm100_binding.cu (1 hunks)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

GitHub Check: Deploy Docs

🔇 Additional comments (1)

csrc/fused_moe/cutlass_backend/flashinfer_cutlass_fused_moe_sm100_binding.cu (1)

825-831: LGTM! API migration correctly applied.

The change from get() to GetDLTensorPtr() is consistent across all four null-pointer checks for the FP8 quantization parameters. This correctly addresses the API change after bumping the apache-tvm-ffi version.

bkryu · 2025-10-28T16:14:05Z

/bot run

flashinfer-bot · 2025-10-28T16:14:19Z

GitLab MR !93 has been created, and the CI pipeline #37457887 is currently running. I'll report back once the pipeline job completes.

bkryu · 2025-10-28T16:29:00Z

I can repro the previously failing unit tests now passing with this PR on B200. Waiting for results from CI bot's pipeline

yzh119

Thanks for the timely fix!

…ons (flashinfer-ai#1995) ## 📌 Description Using different API after `apach-tvm-ffi` version bump.  ## Summary by CodeRabbit * **Bug Fixes** * Improved null-pointer validation for FP8 quantization tensors used during inference, increasing robustness and reducing risk of runtime errors.  --------- Signed-off-by: Amir Klein <203507526+amirkl94@users.noreply.github.com>

Bugfix: Change get() -> GetDLTensorPtr() in cutlass FusedMoE validations

0a5a368

Signed-off-by: Amir Klein <203507526+amirkl94@users.noreply.github.com>

amirkl94 requested review from cyx-6, djmmoss, wenscarl, yongwww and yzh119 as code owners October 28, 2025 09:32

gemini-code-assist Bot reviewed Oct 28, 2025

View reviewed changes

Comment thread csrc/fused_moe/cutlass_backend/flashinfer_cutlass_fused_moe_sm100_binding.cu Outdated

Fix error string

c8df0a9

Signed-off-by: Amir Klein <203507526+amirkl94@users.noreply.github.com>

coderabbitai Bot reviewed Oct 28, 2025

View reviewed changes

tqchen approved these changes Oct 28, 2025

View reviewed changes

yzh119 approved these changes Oct 28, 2025

View reviewed changes

yzh119 merged commit 8ad64e2 into flashinfer-ai:main Oct 28, 2025
4 checks passed

coderabbitai Bot mentioned this pull request Nov 6, 2025

update trtllm cutlass moe #2020

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bugfix: Change get() -> GetDLTensorPtr() in cutlass FusedMoE validations#1995

Bugfix: Change get() -> GetDLTensorPtr() in cutlass FusedMoE validations#1995
yzh119 merged 2 commits intoflashinfer-ai:mainfrom
amirkl94:bugfix/use-correct-api

amirkl94 commented Oct 28, 2025 •

edited by coderabbitai Bot

Loading

Uh oh!

gemini-code-assist Bot commented Oct 28, 2025

Uh oh!

amirkl94 commented Oct 28, 2025

Uh oh!

coderabbitai Bot commented Oct 28, 2025 •

edited

Loading

Other AI code review bot(s) detected

Uh oh!

flashinfer-bot commented Oct 28, 2025

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Uh oh!

bkryu commented Oct 28, 2025

Uh oh!

flashinfer-bot commented Oct 28, 2025

Uh oh!

bkryu commented Oct 28, 2025

Uh oh!

yzh119 left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

amirkl94 commented Oct 28, 2025 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📌 Description

Summary by CodeRabbit

Uh oh!

gemini-code-assist Bot commented Oct 28, 2025

Summary of Changes

Highlights

Footnotes

Uh oh!

amirkl94 commented Oct 28, 2025

Uh oh!

coderabbitai Bot commented Oct 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Other AI code review bot(s) detected

Walkthrough

Changes

Estimated code review effort

Possibly related PRs

Suggested reviewers

Poem

Pre-merge checks and finishing touches

Uh oh!

flashinfer-bot commented Oct 28, 2025

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

bkryu commented Oct 28, 2025

Uh oh!

flashinfer-bot commented Oct 28, 2025

Uh oh!

bkryu commented Oct 28, 2025

Uh oh!

yzh119 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

amirkl94 commented Oct 28, 2025 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Oct 28, 2025 •

edited

Loading