Are there user scripts examples how to distribute work? #20

picobyte · 2023-10-01T18:59:10Z

picobyte
Oct 1, 2023

I have 4+1 webui workers, but work doesn't seem to get distributed. Does this require user scripts? Is there an example?

I have a Tesla M10 with 4 gpus as worker, each with 8GB vram, Maxwell arch, sadly, not very fast. Speed barely outperforms --use-cpu all, which I use as webui worker. All on the same machine. If there is a better way, I'm all ears.

papuSpartan · 2023-10-01T21:59:23Z

papuSpartan
Oct 1, 2023
Maintainer

Does this require user scripts?

No, user script is an optional script you can add that is run when that button is pressed. It's not required

can you post your sdwui logs when trying to use the extension?

0 replies

picobyte · 2023-10-01T22:53:14Z

picobyte
Oct 1, 2023
Author

Thank you for your reply. (my next answer may take a while sleep+ work)
one of the workers

LD_LIBRARY_PATH=/home/pico/Downloads/TensorRT-8.6.1.6/lib python -u launch.py --xformers --no-half-vae --medvram --opt-split-attention --nowebui --api --device-id 0  --port 7860 --listen
Python 3.10.13 (main, Sep 30 2023, 13:12:56) [GCC 13.2.1 20230801]
Version: v1.6.0
Commit hash: 5ef669de080814067961f28357256e8fe27544f4
loading WD14-tagger reqs from /home/pico/dev/git/stable-diffusion-webui/extensions/stable-diffusion-webui-wd14-tagger/requirements.txt
Checking WD14-tagger requirements.
Launching API server with arguments: --xformers --no-half-vae --medvram --opt-split-attention --nowebui --api --device-id 0 --port 7860 --listen
2023-10-02 00:20:22.563314: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
Civitai Helper: Get Custom Model Folder
Tag Autocomplete: Could not locate model-keyword extension, Lora trigger word completion will be limited to those added through the extra networks menu.
[-] ADetailer initialized. version: 23.9.3, num models: 12
[AddNet] Updating model hashes...
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1357/1357 [00:00<00:00, 106801.59it/s]
[AddNet] Updating model hashes...
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1357/1357 [00:00<00:00, 159734.80it/s]
2023-10-02 00:20:28,530 - ControlNet - INFO - ControlNet v1.1.410
ControlNet preprocessor location: /home/pico/dev/git/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/downloads
2023-10-02 00:20:28,661 - ControlNet - INFO - ControlNet v1.1.410
INFO     doing initial ping sweep to see which workers are reachable                                                                                                                                                                                                                                      extension.py:63
INFO     worker 'worker1' is unreachable                                                                                                                                                                                                                                                                     World.py:604
== WD14 tagger /gpu:0, uname_result(system='Linux', node='Z', release='6.5.5-arch1-1', version='#1 SMP PREEMPT_DYNAMIC Sat, 23 Sep 2023 22:55:13 +0000', machine='x86_64') ==
Loading weights [879db523c3] from /home/pico/dev/git/stable-diffusion-webui/models/Stable-diffusion/dreamshaper_8.safetensors
[Lobe]: Initializing Lobe
Startup time: 14.4s (prepare environment: 5.1s, import torch: 3.6s, import gradio: 0.3s, setup paths: 0.9s, initialize shared: 0.1s, other imports: 0.3s, list SD models: 0.2s, load scripts: 3.6s, task_scheduler.py: 0.1s).
INFO:     Started server process [136132]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:7860 (Press CTRL+C to quit)
Creating model from config: /home/pico/dev/git/stable-diffusion-webui/configs/v1-inference.yaml
Loading VAE weights specified in settings: /home/pico/dev/git/stable-diffusion-webui/models/VAE/vae-ft-mse-840000-ema-pruned.ckpt
Applying attention optimization: Doggettx... done.
Model loaded in 2.8s (load weights from disk: 0.4s, create model: 0.3s, apply weights to model: 0.9s, apply dtype to VAE: 0.1s, load VAE: 0.5s, load textual inversion embeddings: 0.1s, calculate empty prompt: 0.4s).

Webui:

python -u launch.py --use-cpu all --skip-torch-cuda-test --no-half-vae --api --port 7865 --no-half
Python 3.10.13 (main, Sep 30 2023, 13:12:56) [GCC 13.2.1 20230801]
Version: v1.6.0
Commit hash: 5ef669de080814067961f28357256e8fe27544f4
loading WD14-tagger reqs from /home/pico/dev/git/stable-diffusion-webui/extensions/stable-diffusion-webui-wd14-tagger/requirements.txt
Checking WD14-tagger requirements.
Launching Web UI with arguments: --use-cpu all --skip-torch-cuda-test --no-half-vae --api --port 7865 --no-half
2023-10-02 00:20:29.129258: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.
2023-10-02 00:20:29.164724: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2023-10-02 00:20:29.164769: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2023-10-02 00:20:29.164809: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2023-10-02 00:20:29.172220: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.
2023-10-02 00:20:29.172436: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-10-02 00:20:29.782111: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
WARNING:xformers:WARNING[XFORMERS]: xFormers can't load C++/CUDA extensions. xFormers was built for:
    PyTorch 2.0.1+cu118 with CUDA 1108 (you have 2.0.1+cpu)
    Python  3.10.13 (you have 3.10.13)
  Please reinstall xformers (see https://github.com/facebookresearch/xformers#installing-xformers)
  Memory-efficient attention, SwiGLU, sparse and more won't be available.
  Set XFORMERS_MORE_DETAILS=1 for more details
No module 'xformers'. Proceeding without it.
Warning: caught exception 'Torch not compiled with CUDA enabled', memory monitor disabled
Civitai Helper: Get Custom Model Folder
Tag Autocomplete: Could not locate model-keyword extension, Lora trigger word completion will be limited to those added through the extra networks menu.
[-] ADetailer initialized. version: 23.9.3, num models: 12
[AddNet] Updating model hashes...
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1357/1357 [00:00<00:00, 142434.20it/s]
[AddNet] Updating model hashes...
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1357/1357 [00:00<00:00, 109930.86it/s]
2023-10-02 00:20:35,582 - ControlNet - INFO - ControlNet v1.1.410
ControlNet preprocessor location: /home/pico/dev/git/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/downloads
2023-10-02 00:20:35,723 - ControlNet - INFO - ControlNet v1.1.410
INFO     doing initial ping sweep to see which workers are reachable                                                                                                                                                                                                                                      extension.py:63
INFO     worker 'worker1' is unreachable                                                                                                                                                                                                                                                                     World.py:604
== WD14 tagger /cpu:0, uname_result(system='Linux', node='Z', release='6.5.5-arch1-1', version='#1 SMP PREEMPT_DYNAMIC Sat, 23 Sep 2023 22:55:13 +0000', machine='x86_64') ==
Loading weights [879db523c3] from /home/pico/dev/git/stable-diffusion-webui/models/Stable-diffusion/dreamshaper_8.safetensors
/home/pico/dev/git/stable-diffusion-webui/extensions/stable-diffusion-webui-distributed/scripts/spartan/UI.py:179: GradioDeprecationWarning: The `style` method is deprecated. Please set these arguments in the constructor instead.
  refresh_status_btn.style(size='sm')
/home/pico/dev/git/stable-diffusion-webui/extensions/stable-diffusion-webui-distributed/scripts/spartan/UI.py:186: GradioDeprecationWarning: The `style` method is deprecated. Please set these arguments in the constructor instead.
  refresh_checkpoints_btn.style(full_width=False)
/home/pico/dev/git/stable-diffusion-webui/extensions/stable-diffusion-webui-distributed/scripts/spartan/UI.py:186: GradioDeprecationWarning: Use `scale` in place of full_width in the constructor. scale=1 will make the button expand, whereas 0 will not.
  refresh_checkpoints_btn.style(full_width=False)
/home/pico/dev/git/stable-diffusion-webui/extensions/stable-diffusion-webui-distributed/scripts/spartan/UI.py:190: GradioDeprecationWarning: The `style` method is deprecated. Please set these arguments in the constructor instead.
  run_usr_btn.style(full_width=False)
/home/pico/dev/git/stable-diffusion-webui/extensions/stable-diffusion-webui-distributed/scripts/spartan/UI.py:190: GradioDeprecationWarning: Use `scale` in place of full_width in the constructor. scale=1 will make the button expand, whereas 0 will not.
  run_usr_btn.style(full_width=False)
/home/pico/dev/git/stable-diffusion-webui/extensions/stable-diffusion-webui-distributed/scripts/spartan/UI.py:194: GradioDeprecationWarning: The `style` method is deprecated. Please set these arguments in the constructor instead.
  interrupt_all_btn.style(full_width=False)
/home/pico/dev/git/stable-diffusion-webui/extensions/stable-diffusion-webui-distributed/scripts/spartan/UI.py:194: GradioDeprecationWarning: Use `scale` in place of full_width in the constructor. scale=1 will make the button expand, whereas 0 will not.
  interrupt_all_btn.style(full_width=False)
/home/pico/dev/git/stable-diffusion-webui/extensions/stable-diffusion-webui-distributed/scripts/spartan/UI.py:198: GradioDeprecationWarning: The `style` method is deprecated. Please set these arguments in the constructor instead.
  redo_benchmarks_btn.style(full_width=False)
/home/pico/dev/git/stable-diffusion-webui/extensions/stable-diffusion-webui-distributed/scripts/spartan/UI.py:198: GradioDeprecationWarning: Use `scale` in place of full_width in the constructor. scale=1 will make the button expand, whereas 0 will not.
  redo_benchmarks_btn.style(full_width=False)
/home/pico/dev/git/stable-diffusion-webui/extensions/stable-diffusion-webui-distributed/scripts/spartan/UI.py:202: GradioDeprecationWarning: The `style` method is deprecated. Please set these arguments in the constructor instead.
  reload_config_btn.style(full_width=False)
/home/pico/dev/git/stable-diffusion-webui/extensions/stable-diffusion-webui-distributed/scripts/spartan/UI.py:202: GradioDeprecationWarning: Use `scale` in place of full_width in the constructor. scale=1 will make the button expand, whereas 0 will not.
  reload_config_btn.style(full_width=False)
/home/pico/dev/git/stable-diffusion-webui/extensions/stable-diffusion-webui-distributed/scripts/spartan/UI.py:206: GradioDeprecationWarning: The `style` method is deprecated. Please set these arguments in the constructor instead.
  reconnect_lost_workers_btn.style(full_width=False)
/home/pico/dev/git/stable-diffusion-webui/extensions/stable-diffusion-webui-distributed/scripts/spartan/UI.py:206: GradioDeprecationWarning: Use `scale` in place of full_width in the constructor. scale=1 will make the button expand, whereas 0 will not.
  reconnect_lost_workers_btn.style(full_width=False)
Civitai Helper: Settings:
Civitai Helper: max_size_preview: True
Civitai Helper: skip_nsfw_preview: False
Civitai Helper: open_url_with_js: True
Civitai Helper: proxy: 
/home/pico/dev/git/stable-diffusion-webui/extensions/sd-webui-additional-networks/scripts/metadata_editor.py:399: GradioDeprecationWarning: The `style` method is deprecated. Please set these arguments in the constructor instead.
  with gr.Row().style(equal_height=False):
/home/pico/dev/git/stable-diffusion-webui/extensions/sd-webui-additional-networks/scripts/metadata_editor.py:521: GradioDeprecationWarning: The `style` method is deprecated. Please set these arguments in the constructor instead.
  cover_image = gr.Image(
Scanning <DirEntry 'deepdanbooru-v3-20211112-sgd-e28'> as deepdanbooru project
Scanning <DirEntry 'deepdanbooru-v4-20200814-sgd-e30'> as deepdanbooru project
Scanning <DirEntry 'wd-v1-4-convnextv2-tagger-v2'> as onnx model
Scanning <DirEntry 'Z3D-E621-Convnext'> as onnx model
Creating model from config: /home/pico/dev/git/stable-diffusion-webui/configs/v1-inference.yaml
Running on local URL:  http://127.0.0.1:7865

To create a public link, set `share=True` in `launch()`.
[Lobe]: Initializing Lobe
Startup time: 15.3s (prepare environment: 3.4s, import torch: 4.0s, import gradio: 0.4s, setup paths: 0.8s, other imports: 0.3s, list SD models: 0.2s, load scripts: 3.5s, create ui: 0.9s, gradio launch: 1.1s, add APIs: 0.1s, app_started_callback: 0.4s).
Loading VAE weights specified in settings: /home/pico/dev/git/stable-diffusion-webui/models/VAE/vae-ft-mse-840000-ema-pruned.ckpt
Applying attention optimization: InvokeAI... done.
Model loaded in 6.5s (load weights from disk: 1.3s, create model: 0.4s, apply weights to model: 1.4s, apply float(): 2.8s, load VAE: 0.4s, load textual inversion embeddings: 0.1s).

the gradio messages seem harmless, just deprecation notices. You can ignore the wd14tagger messages. it is a development branch.

the cuDNN, etc. messages are not entirely relevant, the webui is --use-cpu all and is able to txt2img, just not using the workers.

I know of the AVX2 FMA,issue, have been trying to build tensorflow from source, but its configure script is just a mess.

worker 'worker1' is unreachable is important. Selecting the script and a fresh, the worker is listed as unavailable. That was before IDLE.
touching all the variables is not enough, but removing the worker, then adding it again, and a refresh makes it appear again, as IDLE.
..however if I do the same for worker2, which is off-line it is also listed as IDLE.

building an image with prompt 'woman' The cpu is working. the gpu does nothing.
I know because I run gpustat -cp --watch, If I run webui on a gpu, the temperature severely increases. Now I see nothing, and no speedup.

Also, note: I run these from the same directory, just a different venv: the gpu venv was built with:

python3.10 -m venv venv310 
source venv310/bin/activate

# the two lack +118 on purpose (or they will not install)
pip install torchtext==0.15.2 torchdata==0.6.1 torch==2.0.1+cu118 torchvision==0.15.2+cu118 torchaudio==2.0.2+cu118 --index-url https://download.pytorch.org/whl/cu118 
pip install xformers==0.0.22 protobuf==3.20.3 torchsde==0.2.6 tensorflow[and-cuda]==2.13.1

python -c "import torch; print(torch.__version__)"

cd /home/pico/Downloads/TensorRT-8.6.1.6/python
pip install tensorrt*cp310*.whl
cd ../graphsurgeon/
pip install graphsurgeon-0.4.6-py2.py3-none-any.whl
cd ../uff
pip install uff-0.6.9-py2.py3-none-any.whl
cd ../onnx_graphsurgeon/
pip install onnx_graphsurgeon-0.3.12-py2.py3-none-any.whl
cd /home/pico/git/stable-diffusion-webui

the cpu venv was built with:

source venv310cpu/bin/activate
pip install torchtext==0.15.2 torchdata==0.6.1 torch==2.0.1+cpu torchvision==0.15.2+cpu torchaudio==2.0.2+cpu -f https://download.pytorch.org/whl/torch_stable.html
pip install tensorflow-cpu==2.13.1
python -u launch.py --use-cpu all --skip-torch-cuda-test --no-half-vae --api --port 7865 --no-half

3 replies

papuSpartan Oct 1, 2023
Maintainer

You only need the extension installed on the instance that will be receiving the images from everything else, or the "master".

papuSpartan Oct 1, 2023
Maintainer

Make sure in the worker config tab of the master instance you've set worker1's port and address properly.

picobyte Oct 1, 2023
Author

Thanks, ok I now restarted with '--disable-extra-extensions' and restarted webui.
still

INFO     doing initial ping sweep to see which workers are reachable                                                                                                                                                                                                                                      extension.py:63
INFO     worker 'worker1' is unreachable

however in webui it is listed as IDLE. starting job gives me

WARNING  Distributed doesn't yet support: Refiner, Seed, ADetailer, Tiled Diffusion, Tiled VAE, Dynamic Prompts v2.16.3, Dynamic Thresholding (CFG Scale Fix), Additional networks for generating, Agent Scheduler, Aspect Ratio Helper, Cutoff, LoRA Block Weight, Regional Prompter, Segment Anything, extension.py:277
         Composable Lora, Interactive Tokenizer, Extra options                                                                                                                                                                                                                                                           
INFO     Job distribution:                                                                                                                                                                                                                                                                                   World.py:479
         1 * 1 iteration(s): 1 images total                                                                                                                                                                                                                                                                              
         'master' - 1 image(s) @ 0.30 ipm                                                                                                                                                                                                                                                                                
         'worker1' - 0 image(s) @ 1.00 ipm

I still see no gpu activity, though.

picobyte · 2023-10-01T23:23:19Z

picobyte
Oct 1, 2023
Author

pretty sure ports are as printed on the console, though the IP address is a bit odd:

worker1
http://0.0.0.0/
port 7860

Sorry I will have to call it a day. Thanks so far.

1 reply

papuSpartan Oct 4, 2023
Maintainer

did you have any more luck with this or are there still problems?

picobyte · 2023-10-04T20:50:55Z

picobyte
Oct 4, 2023
Author

no, I have tried but haven't gotten it to work. It is working in comfyUI, though that has other issues (cancel not working - I will notify them as well).
If I search for the ip address 0.0.0.0 I get:

The Internet Engineering Task Force defines the IP address 0.0. 0.0 as a reserved, special-purpose address for
"this host, this network." Its IPv6 equivalent is expressed as ::/0. Although 0.0. 0.0 is valid address syntax, a
client device using it as a source IP address cannot communicate on a network.

which I read as, it is impossible to communicate if this IP address is assigned. In comfyui I get IP address http://127.0.0.1/, 192.168.x.x might also work.

1 reply

papuSpartan Oct 4, 2023
Maintainer

For the worker, try removing --listen. So then 127.0.0.1 (loopback) will be the address, and 7860 will be the port. You should be able to visit http://127.0.0.1:7860/docs and see a page showing api information. If you can't see that then the extension won't be able to see the worker either.

Once you confirm that's working, then the configuration should reflect that on the master instance like this:

picobyte · 2023-10-04T23:49:16Z

picobyte
Oct 4, 2023
Author

I don't get a /distributed/ entry in the api. This is listed in the console when I txt2img, no activity on worker2:

INFO     doing initial ping sweep to see which workers are reachable                                                                                                       extension.py:63
INFO     worker 'worker2' is unreachable                                                                                                                                      World.py:604
INFO     worker 'test' is unreachable

I did remove --listen, and did receive the IP & port http://127.0.0.1:7861/
BTW I did not have a worker for the main webui before, but still, that didn't seem to make the difference.
should the worker be up before the main webui?
restarted the worker and webui, still get the failled ping test issue notice

1 reply

papuSpartan Oct 5, 2023
Maintainer

You wont see any new api routes for the extension, I just asked for you to check that page because then you can be sure that the api is available on the worker. It's needed by the master instance for interfacing with the worker. You need to change the port to 7861 if that's what the worker is using now. If you're still having trouble you can join the discord and I can try to assist you further.

picobyte · 2023-10-05T00:26:35Z

picobyte
Oct 5, 2023
Author

The api is there, I was quite sure, but also double checked the ip addresses and ports. It is just not functioning for me. It could also be an extension I have, of course. The api does show a possible issue with the agent scheduler extension: crossed out routes for that extension.

have to call it a day again..

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Are there user scripts examples how to distribute work? #20

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 6 comments 6 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

Are there user scripts examples how to distribute work? #20

picobyte Oct 1, 2023

Replies: 6 comments · 6 replies

papuSpartan Oct 1, 2023 Maintainer

picobyte Oct 1, 2023 Author

papuSpartan Oct 1, 2023 Maintainer

papuSpartan Oct 1, 2023 Maintainer

picobyte Oct 1, 2023 Author

picobyte Oct 1, 2023 Author

papuSpartan Oct 4, 2023 Maintainer

picobyte Oct 4, 2023 Author

papuSpartan Oct 4, 2023 Maintainer

picobyte Oct 4, 2023 Author

papuSpartan Oct 5, 2023 Maintainer

picobyte Oct 5, 2023 Author

picobyte
Oct 1, 2023

Replies: 6 comments 6 replies

papuSpartan
Oct 1, 2023
Maintainer

picobyte
Oct 1, 2023
Author

papuSpartan Oct 1, 2023
Maintainer

papuSpartan Oct 1, 2023
Maintainer

picobyte Oct 1, 2023
Author

picobyte
Oct 1, 2023
Author

papuSpartan Oct 4, 2023
Maintainer

picobyte
Oct 4, 2023
Author

papuSpartan Oct 4, 2023
Maintainer

picobyte
Oct 4, 2023
Author

papuSpartan Oct 5, 2023
Maintainer

picobyte
Oct 5, 2023
Author