Skip to content

Add Docs about attention backend, maybe we need to add a comprehensive one with Support Matrix such as: #4865

Closed
Closed
Add Docs about attention backend, maybe we need to add a comprehensive one with Support Matrix such as:#4865
@hebiao064

Description

@hebiao064
hebiao064
opened on Mar 28, 2025
Backend Page Size > 1 Spec Decoding MLA Sliding Window Speed
FA3 ✅ ❌ ✅ ✅ ⭐ ⭐ ⭐ ⭐ ⭐
FlashInfer ✅ ✅ ✅ ✅ ⭐ ⭐ ⭐ ⭐ ⭐
FlashMLA ✅ ✅ ✅ ✅ ⭐ ⭐ ⭐ ⭐ ⭐
Triton ✅ ✅ ✅ ✅ ⭐ ⭐ ⭐
Torch Native ✅ ✅ ✅ ✅ ⭐

I haven't verified the details, pls do benchmark and read the code and summarize it

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions