Support ML-Danbooru

[CCRcmcpe]

https://github.com/7eu7d7/ML-Danbooru

[CyberLykan]

Hi @CCRcmcpe, since your repo doesn't have a issues tab, I just wanted to let you know that sometimes the extension throws a 404 error due to the system path sometimes being included in the model url for some reason. This is on the vladmandic repo after doing the fix mentioned in vladmandic/sdnext#335 (comment).

Loading ML-Danbooru Caformer dec-5-97527 model file from deepghs/ml-danbooru-onnx
gradio call: EntryNotFoundError
┌───────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────┐
│ D:\My Stuff\Extra Stuff\automatic\venv\lib\site-packages\huggingface_hub\utils\_errors.py:259 in                    │
│ hf_raise_for_status                                                                                                 │
│                                                                                                                     │
│   258 │   try:                                                                                                      │
│ > 259 │   │   response.raise_for_status()                                                                           │
│   260 │   except HTTPError as e:                                                                                    │
│                                                                                                                     │
│ D:\My Stuff\Extra Stuff\automatic\venv\lib\site-packages\requests\models.py:1021 in raise_for_status                │
│                                                                                                                     │
│   1020 │   │   if http_error_msg:                                                                                   │
│ > 1021 │   │   │   raise HTTPError(http_error_msg, response=self)                                                   │
│   1022                                                                                                              │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
HTTPError: 404 Client Error: Not Found for url:
https://huggingface.co/deepghs/ml-danbooru-onnx/resolve/main/C%3A%5CUsers%5Cusernamehere%5C.cache%5Chuggingface%5Chub%5Cmodels
--deepghs--ml-danbooru-onnx%5Csnapshots%5C60009d1a5989970203364a2b27c887e0fa2747f2%5Cml_caformer_m36_dec-5-97527.onnx

[CCRcmcpe]

That's strange. The download logic isn't handled by the extension itself though. It seems like an error of huggingface hub, but I need more information to solve it.

Try clearing the cache and pip install -U huggingface_hub.

[CyberLykan]

That's strange. The download logic isn't handled by the extension itself though. It seems like an error of huggingface hub, but I need more information to solve it.

Try clearing the cache and pip install -U huggingface_hub.

What exactly do you mean by cache? Did you mean this?
C:\Users\username\.cache\huggingface\hub\models--deepghs--ml-danbooru-onnx

[CCRcmcpe]

Yes.

[CyberLykan]

Still getting errors.

Loading ML-Danbooru Caformer dec-5-97527 model file from deepghs/ml-danbooru-onnx
gradio call: EntryNotFoundError
┌───────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────┐
│ D:\My Stuff\Extra Stuff\automatic\venv\lib\site-packages\huggingface_hub\utils\_errors.py:259 in                    │
│ hf_raise_for_status                                                                                                 │
│                                                                                                                     │
│   258 │   try:                                                                                                      │
│ > 259 │   │   response.raise_for_status()                                                                           │
│   260 │   except HTTPError as e:                                                                                    │
│                                                                                                                     │
│ D:\My Stuff\Extra Stuff\automatic\venv\lib\site-packages\requests\models.py:1021 in raise_for_status                │
│                                                                                                                     │
│   1020 │   │   if http_error_msg:                                                                                   │
│ > 1021 │   │   │   raise HTTPError(http_error_msg, response=self)                                                   │
│   1022                                                                                                              │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
HTTPError: 404 Client Error: Not Found for url:
https://huggingface.co/deepghs/ml-danbooru-onnx/resolve/main/C%3A%5CUsers%5Cusername%5C.cache%5Chuggingface%5Chub%5Cmodels
--deepghs--ml-danbooru-onnx%5Csnapshots%5C60009d1a5989970203364a2b27c887e0fa2747f2%5Cml_caformer_m36_dec-5-97527.onnx

The above exception was the direct cause of the following exception:

┌───────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────┐
│ D:\My Stuff\Extra Stuff\automatic\modules\call_queue.py:58 in f                                                     │
│                                                                                                                     │
│    57 │   │   │   │   pr.enable()                                                                                   │
│ >  58 │   │   │   res = list(func(*args, **kwargs))                                                                 │
│    59 │   │   │   if shared.cmd_opts.profile:                                                                       │
│                                                                                                                     │
│ D:\My Stuff\Extra Stuff\automatic\modules\call_queue.py:38 in f                                                     │
│                                                                                                                     │
│    37 │   │   │   try:                                                                                              │
│ >  38 │   │   │   │   res = func(*args, **kwargs)                                                                   │
│    39 │   │   │   finally:                                                                                          │
│                                                                                                                     │
│                                               ... 7 frames hidden ...                                               │
│                                                                                                                     │
│ D:\My Stuff\Extra Stuff\automatic\venv\lib\site-packages\huggingface_hub\file_download.py:1541 in                   │
│ get_hf_file_metadata                                                                                                │
│                                                                                                                     │
│   1540 │   )                                                                                                        │
│ > 1541 │   hf_raise_for_status(r)                                                                                   │
│   1542                                                                                                              │
│                                                                                                                     │
│ D:\My Stuff\Extra Stuff\automatic\venv\lib\site-packages\huggingface_hub\utils\_errors.py:269 in                    │
│ hf_raise_for_status                                                                                                 │
│                                                                                                                     │
│   268 │   │   │   message = f"{response.status_code} Client Error." + "\n\n" + f"Entry Not Fou                      │
│ > 269 │   │   │   raise EntryNotFoundError(message, response) from e                                                │
│   270                                                                                                               │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
EntryNotFoundError: 404 Client Error. (Request ID: Root=1-6450124a-7457fded1f9575b64dc6a440)

Entry Not Found for url:
https://huggingface.co/deepghs/ml-danbooru-onnx/resolve/main/C%3A%5CUsers%5Cusername%5C.cache%5Chuggingface%5Chub%5Cmodels
--deepghs--ml-danbooru-onnx%5Csnapshots%5C60009d1a5989970203364a2b27c887e0fa2747f2%5Cml_caformer_m36_dec-5-97527.onnx.
Traceback (most recent call last):
  File "D:\My Stuff\Extra Stuff\automatic\venv\lib\site-packages\gradio\routes.py", line 399, in run_predict
    output = await app.get_blocks().process_api(
  File "D:\My Stuff\Extra Stuff\automatic\venv\lib\site-packages\gradio\blocks.py", line 1302, in process_api
    data = self.postprocess_data(fn_index, result["prediction"], state)
  File "D:\My Stuff\Extra Stuff\automatic\venv\lib\site-packages\gradio\blocks.py", line 1206, in postprocess_data
    self.validate_outputs(fn_index, predictions)  # type: ignore
  File "D:\My Stuff\Extra Stuff\automatic\venv\lib\site-packages\gradio\blocks.py", line 1181, in validate_outputs
    raise ValueError(
ValueError: An event handler (f) didn't receive enough output values (needed: 4, received: 3).
Wanted outputs:
    [textbox, label, label, html]
Received outputs:
    [None, "", "<div class='error'>EntryNotFoundError: 404 Client Error. (Request ID: Root=1-6450124a-7457fded1f9575b64dc6a440)

Entry Not Found for url: https://huggingface.co/deepghs/ml-danbooru-onnx/resolve/main/C%3A%5CUsers%5Cusername%5C.cache%5Chuggingface%5Chub%5Cmodels--deepghs--ml-danbooru-onnx%5Csnapshots%5C60009d1a5989970203364a2b27c887e0fa2747f2%5Cml_caformer_m36_dec-5-97527.onnx.</div><div class='performance'><p class='time'>Time taken: <wbr>0.31s</p><p class='vram'>Torch active/reserved: 2076/2082 MiB, <wbr>Sys VRAM: 3106/8192 MiB (37.92%)</p></div>"]

[CyberLykan]

@CCRcmcpe I can confirm the issue has been fixed as of commit 7f9a010.

[IrisRainbowNeko]

You can try the official ML-Danbooru webui extension ML-Danbooru-webui

[CyberLykan]

You can try the official ML-Danbooru webui extension ML-Danbooru-webui

Thanks for creating one!

[picobyte]

I'd like to pick this up in my fork, which is now the maintained one that gets installed from the webui. I've placed a pull request there. It will require some changes, let's see..

Also I would it be possible to add a description, what is the model particularly good at for tagging?
Are later revisions just better than the former (should I only list the latest) or can they also have become over-trained?
Also, in my implementation you can now combine the output of interrogations. It's just simple averaging, though,