Skip to content

load_lora_weights not working in SDXL #4286

@adhikjoshi

Description

@adhikjoshi

Describe the bug

When we use load_lora_weights for SDXL lora and model it throws below error

  File "/workspace/app.py", line 18, in <module>
    pipe.load_lora_weights(lora_model)
  File "/usr/local/lib/python3.10/dist-packages/diffusers/pipelines/stable_diffusion_xl/pipeline_stable_diffusion_xl.py", line 857, in load_lora_weights
    self.load_lora_into_unet(state_dict, network_alpha=network_alpha, unet=self.unet)
  File "/usr/local/lib/python3.10/dist-packages/diffusers/loaders.py", line 1062, in load_lora_into_unet
    unet.load_attn_procs(state_dict, network_alpha=network_alpha)
  File "/usr/local/lib/python3.10/dist-packages/diffusers/loaders.py", line 349, in load_attn_procs
    attn_processor = getattr(attn_processor, sub_key)
  File "/usr/local/lib/python3.10/dist-packages/diffusers/models/modeling_utils.py", line 186, in __getattr__
    return super().__getattr__(name)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1614, in __getattr__
    raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'UNet2DConditionModel' object has no attribute ''

Reproduction

from diffusers import DiffusionPipeline
import torch
import time
# load SDXL pipeline
pipe = DiffusionPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-0.9", torch_dtype=torch.float16, variant="fp16", use_safetensors=True
).to("cuda")


lora_model = "lycoris.safetensors"
lora_strength = 1
pipe.load_lora_weights(lora_model)

# create an image
generator = torch.Generator("cuda").manual_seed(0)
prompt = "arcane style, 1girl, hoodie, arm tattoo, portrait, asymmetrical bangs, bandaid, short hair, bangs, breasts, freckles, grey eyes, large breasts, looking at viewer, neck tattoo, nose piercing, pink hair, scar, scar on face, solo, tattoo on face, upper body, detailed background, town, alley, dark alley, portrait, hood on head, night, angry, close up, closed mouth, , ((masterpiece)), absurdres "

image = pipe(prompt=prompt,generator=generator).images[0]
image.save("output/"+time.strftime("%Y%m%d_%H%M%S") + ".png")

Logs

No response

System Info

Name: diffusers
Version: 0.19.0.dev0

Who can help?

@patrickvonplaten

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingstaleIssues that haven't received updates

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions