Skip to content

Commit 86c5617

Browse files
authored
[BUG]: fix prevent illegal memory access in Mamba SSM tracking during EAGLE speculative verification (sgl-project#19415)
Co-authored-by: ConnorLi96 <ConnorLi96@users.noreply.github.com>
1 parent 10c65df commit 86c5617

2 files changed

Lines changed: 4 additions & 0 deletions

File tree

python/sglang/srt/speculative/eagle_info.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -154,6 +154,8 @@ def prepare_for_verify(self, batch: ScheduleBatch, page_size: int):
154154
dtype=torch.int64,
155155
device=batch.device,
156156
)
157+
batch.mamba_track_mask = None
158+
batch.mamba_track_seqlens = None
157159

158160
def generate_attn_arg_prefill(
159161
self,

python/sglang/srt/speculative/eagle_info_v2.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -242,6 +242,8 @@ def prepare_for_v2_verify(
242242
dtype=torch.int64,
243243
device=device,
244244
)
245+
batch.mamba_track_mask = None
246+
batch.mamba_track_seqlens = None
245247

246248
# Get a forward batch
247249
batch.forward_mode = (

0 commit comments

Comments
 (0)