Diffusers bug: "UNet2DConditionModel" has no attribute "weight"

Question

When using the official script "convert_lora_safetensor_to_diffusers.py", I tried to load a basemodel by using pipeline = StableDiffusionPipeline.from_pretrained(base_model_path, torch_dtype=torch.float32) and combine it with lora weights.

curr_layer = pipeline.unet

curr_layer.weight.data += alpha * torch.mm(weight_up, weight_down)

And this error appeared:

Modulist Object has no attribute weight

I use dir and the result is:

I wonder what's wrong with my diffusers or how I can fix this.

Answer 1

UNet2DConditionModel is an entire model, and not a layer that you can access its weights.

If you print out this unet, you'll see the architecture and the different layers:

  (conv_in): Conv2d(4, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (time_proj): Timesteps()
  (time_embedding): TimestepEmbedding(
    (linear_1): Linear(in_features=320, out_features=1280, bias=True)
...
  )
  (conv_norm_out): GroupNorm(32, 320, eps=1e-05, affine=True)
  (conv_act): SiLU()
  (conv_out): Conv2d(320, 4, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
)

Then, you can choose a specific layer to look at its weights, e.g. pipeline.unet.conv_out.weight.

In the script you mentioned, curr_layer is changed to a torch layer by following the given prefixes in line 55, which does the same as pipeline.unet.conv_out.weight.