Skip to content

[Bug] LoRA buffer not correctly evicted when LoRA names are reused #8263

@lifuhuang

Description

@lifuhuang

Checklist

  • 1. I have searched related issues but cannot get the expected help.
  • 2. The bug has not been fixed in the latest version.
  • 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
  • 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
  • 5. Please use English, otherwise it will be closed.

Describe the bug

Currently we use LoRA name to identify adapters. However, with the introduction of dynamic lora update, this might result in incorrect eviction behavior when the same LoRA name is reused across different adapters, which is not uncommon at production (e.g., when "updating" an existing LoRA).

I realized this case when I was brainstorming with @Fridge003 on radix cache compatibility. I am expecting this to be fixed by #8261. Creating an issue for tracking purposes.

Reproduction

This newly added test would fail on main: test/srt/models/lora/test_lora_eviction.py

Environment

N/A

Metadata

Metadata

Assignees

Labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions