|
# FIXME: Handle lora uid with None more safely |
We should have a more robust check. @jcbjcbjc
we can use null lora slot?
Like:
- Reserve some slot in
LoraMemPool and if lora_path is None in the loop, assign the weight_indices[i] for that request to the reserved null slot index.
And set some parameters so that applying LoRA (here rank 0) has absolutely no effect. But then backend kernel should handle this case as well?
sglang/python/sglang/srt/lora/lora_manager.py
Line 139 in 7d0edf3
We should have a more robust check. @jcbjcbjc
we can use
nulllora slot?Like:
LoraMemPooland iflora_pathisNonein the loop, assign theweight_indices[i]for that request to the reservednullslot index.And set some parameters so that applying
LoRA(here rank 0) has absolutely no effect. But then backend kernel should handle this case as well?