New User: Getting an error every time I try to generate images after the first run.

[dhosein]

Hi,
i'm new to Fooocus, having only found out about this tool in the past week. I've been running things fairly ok so far, including managing to find the basics of how to add new models/loras to get different outputs (I'm looking to generate comic art or fantasy art style pics.. so do let me know if there's extra downloads you'd recommend).

In the last couple of days, I've also discovered wildcards, so I'm using those fairly heavily to vary the images I get in a run. But I've started to get this error appearing in the log.

File "C:\Fooocus\python_embeded\lib\site-packages\torch\cuda[memory.py](https://memory.py/)", line 663, in mem_get_info

return torch.cuda.cudart().cudaMemGetInfo(device)

RuntimeError: CUDA error: device-side assert triggered

CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.

For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

The first time I load and run the 'generate' button - no problems. But for the second and any further attempts, I get this... and nothing is generated.

Shutting the session and restarting resolved this.. for one more run only.
What am I doing wrong? Any advice?

Also - I know not related to the error, but I'm getting some strange things which are probably issue with my prompts. But is there any way to improve the image output - particular eyes? I've tried using eye focused loras but they don't appear to have any effect whatsoever.

The other issue I tend to get is duplicated characters. I ask for an image of a character.. and I get twins or triplets...

[mashb1t]

CUDA_LAUNCH_BLOCKING=1 => check related issues in https://github.com/lllyasviel/Fooocus/issues?q=is%3Aissue+CUDA_LAUNCH_BLOCKING%3D1+is%3Aclosed

Eyes: use inpainting with improve detail. Fooocus only supports LoRAs for SDXL (not 1.5) when not using a 1.5 refiner.

Duplicate characters: use a different aspect ration and standard SDXL (Fooocus) resolutions + probably a negative prompt for multiple characters

Hope this helps!

[dhosein]

Yeah - I've tried that search. Nothing there quite seems to match what I'm seeing in the logs and suggested solutions don't seem to alter outcome.
It seems to be something related to using wildcards and the size of batches.
If I run only 16 images/per run - I seem to get 3 or 4 runs before the error sets in. If I run 32 - It fails on the 2nd run every time.
Running only 8 images seems to avoid the issue entirely.. as does not using wildcards.

I am trying a new aspect ratio, although I am still seeing duplicates coming up. Have only been using standard resolutions (haven't figured out custom resolutions yet). Haven't tried the negative prompt yet - will do.

[ILikeToasters]

I doubt it has anything to do with the number of images generated and wildcards only. I generated 1600 in one go with wildcards the other night without an issue. Unless it's something specific to your system.

[dhosein]

Hence why I came hoping to find answers...
But I'm not seeing anything helpful in the related issues linked above. If the answer is there, I don't understand the jargon well enough to spot it.

[ILikeToasters]

Did you add "set CUDA_LAUNCH_BLOCKING=1" to your .bat file? You can edit the .bat file used to launch Fooocus with notepad and add that and see if it helps or at least gives you more info if it still has issues.

set CUDA_LAUNCH_BLOCKING=1
.\python_embeded\python.exe -s Fooocus\entry_with_update.py
pause

[dhosein]

Yes - did that. Nothing changed.

[dhosein]

I think I may have found something else in the logs that might be why I'm struggling. This is quite a ways from the end of the log but I found this one one failed run, attempting 32 images.

"[Fooocus] Preparing Fooocus text #20 ...
Token indices sequence length is longer than the specified maximum sequence length for this model (2539 > 1024). Running this sequence through the model will result in indexing errors
C:\actions-runner_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda[Indexing.cu:1292](https://indexing.cu:1292/): block: [9,0,0], thread: [96,0,0] Assertion srcIndex < srcSelectDimSize failed."

So - it was fine up to image 20.. then falls over.
I'm thinking this may be wildcard related, since it appears to be compiling 32 different prompts by expanding out the wildcards of the original. So the number of tokens climbs dramatically.

What I don't really get is why it takes until the 2nd or 3rd run before the failure. Or if I'm running only 16 images, 4 or 5 runs. Only thing I can think of is that the token count isn't getting reset between runs?

And is there any way to adjust the token limit?

[dhosein]

I'm pretty certain there's no loops, but difficult to tell if there's a particular part that is just too crazy long... when the error shows in the log.. the prompt that actually triggers the error isn't shown. Only the prior one that actually worked.

[ILikeToasters]

So I did some experimenting and found that yes if the prompt gets long enough it will cause that error so I will bet the wildcards are creating really long prompts. But I will say I had to use extremely long prompts to get it to fail for me. 500 word prompts did not cause a problem. Once you get over 500 that is where I start seeing the problem. Those are some seriously long prompts.

[dhosein]

Once expanded - I don't think any of the prompts are coming in at over 100 words. around the 80 mark seems to be the average.
I guess with a run of 32.. that's potentially 3200 though.
The error suggests the token limit is at 1024. Not sure 1 token = 1 word... I'm guessing a lot of structural words (a, and, the, as etc...) get stripped out... but still...
Guess I should be relatively safe, long as I keep runs to 10 images max?

[ILikeToasters]

Just figured I would mention in case you are not aware, you can right click on the generate button and it will give you the option to generate forever. So you could have it just do whatever is usually safe for a number and then have it start again. So if you set it for 10 images it would generate those and then start again with the same prompt and everything with a new batch. That may allow you to generate more if you don't want to have to keep hitting generate again and again if doing a lot.

[dhosein]

I did not know that. Thank you.
I gave that a go this morning - and the error does inevitably occur down the line, but it got me 61 images before running into problems.