How to use SD3.5

[gerroon]

Hi

I downloaded sd3.5_medium.safetensors and put it in the checkpoints folder. When I select the SD3.5 model it doesn't generate anything it seems. Do I need to download additional stuff?

The safetensors file is corrupt or invalid. Make sure this is actually a safetensors file and not a ckpt or pt or other filetype.
Failed to load c:\RuinedFooocus\modules\..\models\checkpoints\sd3.5_medium.safetensors
Loading LoRAs: sd_xl_offset_example-lora_1.0.safetensors
Error loading LoRA: c:\RuinedFooocus\modules\..\models\loras\sd_xl_offset_example-lora_1.0.safetensors
LoRAs loaded: []
ERROR: Can only use SD1.x, SDXL, SD3, Flux or Lumina2 models
Time taken: 0.00 seconds Pipeline process
ERROR: Can only use SD1.x, SDXL, SD3, Flux or Lumina2 models
Time taken: 0.00 seconds Pipeline process
Traceback (most recent call last):
  File "c:\RuinedFooocus\venv\lib\site-packages\gradio\queueing.py", line 625, in process_events
    response = await route_utils.call_process_api(
  File "c:\RuinedFooocus\venv\lib\site-packages\gradio\route_utils.py", line 322, in call_process_api
    output = await app.get_blocks().process_api(
  File "c:\RuinedFooocus\venv\lib\site-packages\gradio\blocks.py", line 2103, in process_api
    result = await self.call_function(
  File "c:\RuinedFooocus\venv\lib\site-packages\gradio\blocks.py", line 1636, in call_function
    processed_input, progress_index, _ = special_args(
  File "c:\RuinedFooocus\venv\lib\site-packages\gradio\helpers.py", line 958, in special_args
    inputs.insert(i, type_hint(event_data.target, event_data._data))
  File "c:\RuinedFooocus\venv\lib\site-packages\gradio\events.py", line 191, in __init__
    self.value: Any = data["value"]
KeyError: 'value'

[yownas]

Hm. I should work. Do you get any other errors/output before it fails?

[gerroon]

That is all I have between the startup and the end of the error

Loading base model: sd3.5_medium.safetensors
WARNING:root:no CLIP/text encoder weights in checkpoint, the text encoder model will not be loaded.
Loading CLIP: ['clip_l.safetensors', 'clip_g.safetensors', 't5-v1_1-xxl-encoder-Q3_K_S.gguf']

ggml_sd_loader:
 GGMLQuantizationType.Q3_K     168
 GGMLQuantizationType.F32       50
 GGMLQuantizationType.Q6_K       1
WARNING:root:clip missing: ['text_projection.weight']
Loading VAE: sd3_vae.safetensors
Traceback (most recent call last):
  File "c:\RuinedFooocus\repositories\ComfyUI\comfy\utils.py", line 55, in load_torch_file
    with safetensors.safe_open(ckpt, framework="pt", device=device.type) as f:
safetensors_rust.SafetensorError: Error while deserializing header: HeaderTooLarge

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\RuinedFooocus\modules\sdxl_pipeline.py", line 326, in load_base_model
    sd = comfy.utils.load_torch_file(str(vae_path))
  File "c:\RuinedFooocus\repositories\ComfyUI\comfy\utils.py", line 65, in load_torch_file
    raise ValueError("{}\n\nFile path: {}\n\nThe safetensors file is corrupt or invalid. Make sure this is actually a safetensors file and not a ckpt or pt or other filetype.".format(message, ckpt))
ValueError: Error while deserializing header: HeaderTooLarge

File path: c:\RuinedFooocus\modules\..\models\vae\sd3_vae.safetensors

The safetensors file is corrupt or invalid. Make sure this is actually a safetensors file and not a ckpt or pt or other filetype.
Failed to load c:\RuinedFooocus\modules\..\models\checkpoints\sd3.5_medium.safetensors
Loading LoRAs: sd_xl_offset_example-lora_1.0.safetensors
Error loading LoRA: c:\RuinedFooocus\modules\..\models\loras\sd_xl_offset_example-lora_1.0.safetensors
LoRAs loaded: []
ERROR: Can only use SD1.x, SDXL, SD3, Flux or Lumina2 models
Time taken: 0.00 seconds Pipeline process
ERROR: Can only use SD1.x, SDXL, SD3, Flux or Lumina2 models
Time taken: 0.00 seconds Pipeline process
Traceback (most recent call last):
  File "c:\RuinedFooocus\venv\lib\site-packages\gradio\queueing.py", line 625, in process_events
    response = await route_utils.call_process_api(
  File "c:\RuinedFooocus\venv\lib\site-packages\gradio\route_utils.py", line 322, in call_process_api
    output = await app.get_blocks().process_api(
  File "c:\RuinedFooocus\venv\lib\site-packages\gradio\blocks.py", line 2103, in process_api
    result = await self.call_function(
  File "c:\RuinedFooocus\venv\lib\site-packages\gradio\blocks.py", line 1636, in call_function
    processed_input, progress_index, _ = special_args(
  File "c:\RuinedFooocus\venv\lib\site-packages\gradio\helpers.py", line 958, in special_args
    inputs.insert(i, type_hint(event_data.target, event_data._data))
  File "c:\RuinedFooocus\venv\lib\site-packages\gradio\events.py", line 191, in __init__
    self.value: Any = data["value"]
KeyError: 'value'

[yownas]

Oh. Looks like you can't download the SD3 VAE file directly from CivitAI anymore. :(

You can download this one instead https://huggingface.co/calcuis/sd3.5-large-gguf/blob/main/diffusion_pytorch_model.safetensors and save it over the old RuinedFoocus\models\vae\sd3_vae.safetensors.

...and I'll update the link in our code as well.

[gerroon]

Thanks, it now loads the model but the results are pretty bad. I have 12GB VRAM, I thought it should be fine for 3.5 Medium

[gerroon]

I dont see any errors on the other hand

Loading base model: sd3.5_medium.safetensors
WARNING:root:no CLIP/text encoder weights in checkpoint, the text encoder model will not be loaded.
Loading CLIP: ['clip_l.safetensors', 'clip_g.safetensors', 't5-v1_1-xxl-encoder-Q3_K_S.gguf']

ggml_sd_loader:
 GGMLQuantizationType.Q3_K     168
 GGMLQuantizationType.F32       50
 GGMLQuantizationType.Q6_K       1
WARNING:root:clip missing: ['text_projection.weight']
Loading VAE: sd3_vae.safetensors
LoRAs loaded: []

[yownas]

You might need to make a new Performance setting for it. I use:

Custom Steps: 30
CFG: 5
Sampler: dpmpp_2m
Scheduler: sgm_uniform
Clip Skip: 1

Don't think they are "the best" settings, but they seem to work.

[gerroon]

Thanks, that does definitely improve the results.