leo-ke
leo-ke

Reputation: 11

When I want use "ControlNetModel.from_single_file" in diffuser, I give it correct path , but it doesn't work

code

from diffusers import StableDiffusionControlNetPipeline ,ControlNetModel
import torch
import cv2
img = cv2.imread('/home/leo/blues.png')

controlnet_model_path = "/mnt/d/models/diffusion_pytorch_model.safetensors"
controlnet = ControlNetModel.from_single_file(controlnet_model_path, torch_dtype=torch.float16)

pipe = StableDiffusionControlNetPipeline.from_pretrained("riffusion/riffusion-model-v1", controlnet=controlnet, torch_dtype=torch.float16)

torch.manual_seed(0)
generator = torch.random.manual_seed(0)

output_path = "happy.png"

out_image = pipe(
"guitar", num_inference_steps=20, generator=generator, image=img
).images[0]

out_image.save(output_path)

error log

Traceback (most recent call last):
  File "/home/leo/PhantomHarmony/src/process/diff.py", line 9, in <module>
    controlnet = ControlNetModel.from_single_file(controlnet_model_path)
  File "/home/leo/test/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "/home/leo/test/lib/python3.10/site-packages/diffusers/loaders/controlnet.py", line 120, in from_single_file
    component = create_diffusers_controlnet_model_from_ldm(
  File "/home/leo/test/lib/python3.10/site-packages/diffusers/loaders/single_file_utils.py", line 866, in create_diffusers_controlnet_model_from_ldm
    diffusers_format_controlnet_checkpoint = convert_controlnet_checkpoint(checkpoint, diffusers_config)
  File "/home/leo/test/lib/python3.10/site-packages/diffusers/loaders/single_file_utils.py", line 830, in convert_controlnet_checkpoint
    new_checkpoint["controlnet_mid_block.weight"] = controlnet_state_dict.pop("middle_block_out.0.weight")
KeyError: 'middle_block_out.0.weight'

system

diffusers version: 0.26.0
Platform: Linux-5.15.133.1-microsoft-standard-WSL2-x86_64-with-glibc2.35
Python version: 3.10.12
PyTorch version (GPU?): 2.1.2+cu121 (True)
Huggingface_hub version: 0.20.3
Transformers version: 4.37.2
Accelerate version: 0.27.0.dev0
xFormers version: not installed

When I run it print error logs

Upvotes: 1

Views: 375

Answers (0)

Related Questions