memical
memical

Reputation: 2493

huggingface diffusers inference for flux in fp16

the Hugginface Flux documentation, links to this comment, describing how to run inference in fp16: https://github.com/huggingface/diffusers/pull/9097#issuecomment-2272292516

it says:

FP16 significantly changes the result of the images. ... Forcing FP32 inference on the text encoders thus allows FP16 DiT + VAE inference to be similar to FP32/BF16.

giving this code:

ckpt_idckpt_id = "black-forest-labs/FLUX.1-schnell"
pipe = FluxPipeline.from_pretrained(
    ckpt_id,
    torch_dtype=torch.bfloat16,
)
pipe.enable_sequential_cpu_offload()
pipe.vae.enable_tiling()
pipe.to(DTYPE)

i have tried it and it works. both for dev and schnell. the model is loaded in bf16 and the converted to fp16.

so how does it force fp32 on text encoders?

Upvotes: 0

Views: 325

Answers (0)

Related Questions