CODEWITHSUNDEEP
pytorchbfloat16
Rahul Chauhan
Rahul Chauhan

Reputation: 11

while running stable diffusion and torch on cpu RuntimeError: expected scalar type BFloat16 but found Float

Fetching 16 files: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 16/16 [00:00<?, ?it/s]
`text_config_dict` is provided which will be used to initialize `CLIPTextConfig`. The value `text_config["id2label"]` will be overriden.
C:\Users\Rahul\.conda\envs\deta_database\lib\site-packages\transformers\models\clip\feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
  warnings.warn(
  0%|                                                                                                                                              | 0/50 [00:00<?, ?it/s]
Exception in Tkinter callback
Traceback (most recent call last):
  File "C:\Users\Rahul\.conda\envs\deta_database\lib\tkinter\__init__.py", line 1921, in __call__
    return self.func(*args)
  File "C:\Users\Rahul\.conda\envs\deta_database\lib\site-packages\customtkinter\windows\widgets\ctk_button.py", line 553, in _clicked
    self._command()
  File "c:\Users\Rahul\OneDrive - Adani Institute for Education and Research\Desktop\img_py\img.py", line 56, in generate
    image = pipe(str(prompt)).images
  File "C:\Users\Rahul\.conda\envs\deta_database\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\Rahul\.conda\envs\deta_database\lib\site-packages\diffusers\pipelines\stable_diffusion\pipeline_stable_diffusion.py", line 667, in __call__
    noise_pred = self.unet(
  File "C:\Users\Rahul\.conda\envs\deta_database\lib\site-packages\torch\nn\modules\module.py", line 1190, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\Users\Rahul\.conda\envs\deta_database\lib\site-packages\diffusers\models\unet_2d_condition.py", line 582, in forward
    sample, res_samples = downsample_block(
  File "C:\Users\Rahul\.conda\envs\deta_database\lib\site-packages\torch\nn\modules\module.py", line 1190, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\Users\Rahul\.conda\envs\deta_database\lib\site-packages\diffusers\models\unet_2d_blocks.py", line 836, in forward
    hidden_states = resnet(hidden_states, temb)
  File "C:\Users\Rahul\.conda\envs\deta_database\lib\site-packages\torch\nn\modules\module.py", line 1190, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\Users\Rahul\.conda\envs\deta_database\lib\site-packages\diffusers\models\resnet.py", line 540, in forward
    hidden_states = self.norm1(hidden_states)
  File "C:\Users\Rahul\.conda\envs\deta_database\lib\site-packages\torch\nn\modules\module.py", line 1190, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\Users\Rahul\.conda\envs\deta_database\lib\site-packages\torch\nn\modules\normalization.py", line 273, in forward
    return F.group_norm(
  File "C:\Users\Rahul\.conda\envs\deta_database\lib\site-packages\torch\nn\functional.py", line 2528, in group_norm
    return torch.group_norm(input, num_groups, weight, bias, eps, torch.backends.cudnn.enabled)
RuntimeError: expected scalar type BFloat16 but found Float

MY CODE

import tkinter  as tk 
import customtkinter as ctk

from PIL import ImageTk
from auth_key import key
import torch
from torch import autocast
from diffusers import StableDiffusionPipeline
import numpy
# from torch import nn

# create app
app = tk.Tk()

app.geometry("532x622")
app.title("Stable Diffusion App")
ctk.set_appearance_mode("dark")

# try:
prompt = ctk.CTkEntry (master = app,height=40,width=512)#,text_font=("Arial",20),text_color="black",fg_color="white")
prompt.place(x=10,y=10)
# except Exception as E:
#     print(E)

lmain = ctk.CTkLabel(master=app,height=512,width=512)
lmain.place(x=10,y=110)



modelid= "CompVis/stable-diffusion-v1-4"
device ="cpu"
# cfg.MODEL.DEVICE = "cpu"
try:
    # pipe= StableDiffusionPipeline.from_pretrained(modelid, revision="fp16", torch_dtype = torch.float16 ,use_auth_token = key) 
    
    pipe= StableDiffusionPipeline.from_pretrained(modelid ,use_auth_token = key) 
    pipe.to(device)
except Exception as E:
    print(E)


def generate():
    with autocast(device):
    # with autocast():
        # image = pipe(prompt.get(),guidance_scale=8.5)["sample"][0]
        
        # image = pipe(prompt.get())["sample"][0]
        image = pipe(str(prompt)).images
    
    img.save('generatedimage.png')
    img = ImageTk.PhotoImage(image)
    lmain.configure(image=img)


trigger = ctk.CTkButton(master = app,height=40,width=120,font=("Arial",20),text_color="white", fg_color="blue", command=generate)
trigger.configure(text="Generate")
trigger.place(x=206,y=60)

app.mainloop()

I'M trying to generater image using pytorch and show in my tkinter app but i'm getting runtimeerror please help me .

Upvotes: 1

Views: 2360

Answers (1)

Harry P
Harry P

Reputation: 1

Im not an expert so take what i say with the fact it may be wrong but i believe autocast has a problem with cpu

quote from torch.autocast: 'For CPU, only lower precision floating point datatype of torch.bfloat16 is supported for now.'

I recommend having it working generating an image first, then implementing that into a UI, that way you can rule out the Diffusion part going wrong.

Upvotes: 0

Related Questions