Expected performance on a RTX3090 - 3it/s

[rubenet12]

Hi,

I am very new to fooocus and AI image generation in general. I have a pc with an RTX3090 which is supposed to be one of the best options, at least as per forums as as per tomshardware benchmarks...

What is the throughput that would be good for this card?

I am getting around 3it/s with all the defaults on the realistic version (run_realistic.bat).
When I reduce the image size to 512x512 I get around 9 it/s, I guess this is expected because it is around one fourth of the default size, but I don´t like this size/quality.

I have tested the following with no success:

• Changing Nvidia drivers, to 531 and others, originally using the latest. Tried both studio and game versions, didn´t find noticeable difference.
• Wanted to test linux but I am struggling with its installation so not an option, tried several distros but installation always fails.
  In progress.
• Tried Fooocus-MRE version but got the same performance, I know this is expected as is based in fooocus but wanted to test just in case.
• Tried different models, with and without lora.
• Tried larger batch sizes in case it parallelizes but doesn´t seem to be the case.
• Read about xformers-api and other optimizations but these seem already in place and not sure how to test or apply them to fooocus.

I did run some benchmarks on my laptop and got ok results, if someone wants to look into further details on the benchmarks-hw, see:
Gigabyte GA-B550M AORUS ELITE Performance Results - UserBenchmark

Any reference of performance or suggestion to improve it is appreciated.

Would be great if parallelization could be increased, I feel only one image is generated at a time and there are plenty of resources available in the GPU(max mem 7GB and max CPU something low like 10%).

Thanks,
Ruben

[Irmagon]

Hi,

I am very new to fooocus and AI image generation in general. I have a pc with an RTX3090 which is supposed to be one of the best options, at least as per forums as as per tomshardware benchmarks...

What is the throughput that would be good for this card?

I am getting around 3it/s with all the defaults on the realistic version (run_realistic.bat). When I reduce the image size to 512x512 I get around 9 it/s, I guess this is expected because it is around one fourth of the default size, but I don´t like this size/quality.

I have tested the following with no success:

• Changing Nvidia drivers, to 531 and others, originally using the latest. Tried both studio and game versions, didn´t find noticeable difference.
• Wanted to test linux but I am struggling with its installation so not an option, tried several distros but installation always fails.
  In progress.
• Tried Fooocus-MRE version but got the same performance, I know this is expected as is based in fooocus but wanted to test just in case.
• Tried different models, with and without lora.
• Tried larger batch sizes in case it parallelizes but doesn´t seem to be the case.
• Read about xformers-api and other optimizations but these seem already in place and not sure how to test or apply them to fooocus.

I did run some benchmarks on my laptop and got ok results, if someone wants to look into further details on the benchmarks-hw, see: Gigabyte GA-B550M AORUS ELITE Performance Results - UserBenchmark

Any reference of performance or suggestion to improve it is appreciated.

Would be great if parallelization could be increased, I feel only one image is generated at a time and there are plenty of resources available in the GPU(max mem 7GB and max CPU something low like 10%).

Thanks, Ruben

Show current nvidia drivers version.
Show current fooocus install path.
Show fooocus startup logs from console window.

[rubenet12]

Sorry for the late reply, did not get a notification.

• Nvidia current driver 546.33. Tested also 531 with no improvement.

• Installed on windows desktop, on an nvme drive:
  C:\Users\Ruben\Desktop\Fooocus_win64_2-1-831>.\python_embeded\python.exe -s Fooocus\entry_with_update.py --preset realistic

• In the meantime I also tested on linux mint but I got a slight decrease in performance with all the defaults.

• See the logs below in windows:

C:\Users\Ruben\Desktop\Fooocus_win64_2-1-831>.\python_embeded\python.exe -s Fooocus\entry_with_update.py --preset realistic
Fast-forward merge
Update succeeded.
[System ARGV] ['Fooocus\\entry_with_update.py', '--preset', 'realistic']
Loaded preset: C:\Users\Ruben\Desktop\Fooocus_win64_2-1-831\Fooocus\presets\realistic.json
Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)]
Fooocus version: 2.1.861
Running on local URL:  http://127.0.0.1:7865

To create a public link, set `share=True` in `launch()`.
Total VRAM 24575 MB, total RAM 65459 MB
Set vram state to: NORMAL_VRAM
Always offload VRAM
Device: cuda:0 NVIDIA GeForce RTX 3090 : native
VAE dtype: torch.bfloat16
Using pytorch cross attention
Refiner unloaded.
model_type EPS
UNet ADM Dimension 2816
Using pytorch attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using pytorch attention in VAE
extra {'cond_stage_model.clip_l.text_projection', 'cond_stage_model.clip_g.transformer.text_model.embeddings.position_ids', 'cond_stage_model.clip_l.logit_scale'}
Base model loaded: C:\Users\Ruben\Desktop\Fooocus_win64_2-1-831\Fooocus\models\checkpoints\realisticStockPhoto_v10.safetensors
Request to load LoRAs [['SDXL_FILM_PHOTOGRAPHY_STYLE_BetaV0.4.safetensors', 0.25], ['None', 1.0], ['None', 1.0], ['None', 1.0], ['None', 1.0]] for model [C:\Users\Ruben\Desktop\Fooocus_win64_2-1-831\Fooocus\models\checkpoints\realisticStockPhoto_v10.safetensors].
Loaded LoRA [C:\Users\Ruben\Desktop\Fooocus_win64_2-1-831\Fooocus\models\loras\SDXL_FILM_PHOTOGRAPHY_STYLE_BetaV0.4.safetensors] for UNet [C:\Users\Ruben\Desktop\Fooocus_win64_2-1-831\Fooocus\models\checkpoints\realisticStockPhoto_v10.safetensors] with 788 keys at weight 0.25.
Loaded LoRA [C:\Users\Ruben\Desktop\Fooocus_win64_2-1-831\Fooocus\models\loras\SDXL_FILM_PHOTOGRAPHY_STYLE_BetaV0.4.safetensors] for CLIP [C:\Users\Ruben\Desktop\Fooocus_win64_2-1-831\Fooocus\models\checkpoints\realisticStockPhoto_v10.safetensors] with 264 keys at weight 0.25.
Fooocus V2 Expansion: Vocab with 642 words.
Fooocus Expansion engine loaded for cuda:0, use_fp16 = True.
Requested to load SDXLClipModel
Requested to load GPT2LMHeadModel
Loading 2 new models
[Fooocus Model Management] Moving model(s) has taken 1.00 seconds
App started successful. Use the app with http://127.0.0.1:7865/ or 127.0.0.1:7865

Thanks,
Ruben

[cibernicola]

RTX 3090 here.
Under Windows.
Similar results.

[lllyasviel]

3it/s is good