System Info
peft==0.18.0
python==3.12.10
transformers==4.57.3
diffusers==0.36.0.dev
torch==2.9.1+rocm7.11.0a20251215
Who can help?
@BenjaminBossan @sayakpaul
Reproduction
simplest repoduction is to attempt to load lora into diffusers model since diffusers uses peft
from diffusers import ZImagePipeline
pipe = ZImagePipeline.from_pretrained("Tongyi-MAI/Z-Image-Turbo")
pipe.load_lora_weights("path/to/lora_weights.safetensors")
image = pipe("A fantasy landscape with mountains and a river").images[0]...
│E:\sd\sdnext\venv\Lib\site-packages\peft\tuners\tuners_utils.py:161 in _get_in_out_features │
│ │
│ 160 │ │ torch_supports_dtensor = version.parse(torch.__version__) >= version.parse("2.5.0") │
│❱ 161 │ │ if torch_supports_dtensor and isinstance(module.weight, torch.distributed.tensor.DTensor): │
│ 162 │ │ │ # If Tensor Parallel is used, the weight is sharded, so we need to get the local shape
AttributeError: module 'torch.distributed' has no attribute 'tensor'
this happens because torch for rocm does not come with distributed support!
this was previously an issue in diffusers as well, but the hard dependencies were since removed.
see huggingface/diffusers#12425 and huggingface/diffusers#12420 for details.
Expected behavior
not fail :)
System Info
peft==0.18.0
python==3.12.10
transformers==4.57.3
diffusers==0.36.0.dev
torch==2.9.1+rocm7.11.0a20251215
Who can help?
@BenjaminBossan @sayakpaul
Reproduction
simplest repoduction is to attempt to load lora into diffusers model since
diffusersusespeftthis happens because torch for rocm does not come with distributed support!
this was previously an issue in
diffusersas well, but the hard dependencies were since removed.see huggingface/diffusers#12425 and huggingface/diffusers#12420 for details.
Expected behavior
not fail :)