Skip to content

[Bug] AssertionError: Currently HybridAttentionBackend does not support speculative decoding. #9330

@zhyncs

Description

@zhyncs

Checklist

  • 1. I have searched related issues but cannot get the expected help.
  • 2. The bug has not been fixed in the latest version.
  • 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
  • 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
  • 5. Please use English, otherwise it will be closed.

Describe the bug

python3 -m sglang.launch_server --model deepseek-ai/DeepSeek-V3-0324 --tp 8 --speculative-algorithm EAGLE --attention-backend trtllm_mla --prefill-attention-backend flashinfer

Reproduction

N/A

Environment

N/A

Metadata

Metadata

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions