Converting custom Models?

[Maki9009]

From Apple, I'd expect it just works approach for converting a models or even a simple UI?

to convert models we need a diffuser model.. that has to be on hugging face apparently. And the instructions arent even very clear. Most people use .ckpt models.. not diffusers so why not offer both versions for conversation...and why does it have to be downloaded from hugging face

[MDMAchine]

To be fair the initial release was not that long ago. Im sure more UI options will arise soon enough.

That said, you can grab the cpkt and convert them to diffusers yourself. The local process is covered well enough in a little writeup here.
I am not sure if you have to convert it to mlmodelc (which in that writeup would be the --bundle-resources-for-swift-cli command i believe) if not using the MochiDiffusion GUI.

[GuiyeC]

Hello! I have created a small app Guernika Model Converter to convert models into CoreML compatible models from existing local models and CKPT files.

I have been able to convert DreamBooth models and HuggingFace models that were giving me problems when loading from the identifier.

This app is essentially a pyinstaller wrapper of modified scripts with a tkinter UI. You can also check the modified scripts on the scripts.zip on the same repo.

[samheather]

@GuiyeC please could you share how you build the pyinstaller binary? Are you using conda or vanilla python? What is in your pip requirements.txt and what is the pyinstaller script? I've tried:

pyinstaller python_coreml_stable_diffusion/torch2coreml.py --noconfirm --clean

requirements.txt installed via pip install -r requirements.txt:

coremltools
diffusers[torch]
torch
transformers
scipy

But getting this error (despite tqdm IS installed):
importlib.metadata.PackageNotFoundError: No package metadata was found for tqdm

[GuiyeC]

@samheather I did not build the script directly, I created a GUI using Tkinter to make it into an .app. It's my first my using Tkinter and pyinstaller so my code is probably not the cleanest. I did have similar problems when executing the generated app, I solved them by force adding the libraries until I stopped getting errors, again, not an expert on pyinstaller so there might be better ways to do this.

[Zabriskije]

Simple GUI tool to convert ckpt/safetensors to mlmodelc, and single-model multi-resolution, are the two most important things right now to stimulate adoption.
Unfortunately, I can't help with that, but I can't wait to see them come true.

Hello! I have created a small app Guernika Model Converter to convert models into CoreML compatible models from existing local models and CKPT files.

Thanks for that. I've tried it once and converted some models successfully, but had less fortune with others. I'll try it again right now as I have memory issues converting a model to 512x768 original, and doing it via Terminal (also separately as stated in Q3 of the FAQ) kills the process.

[pandora523]

@samheather I did not build the script directly, I created a GUI using Tkinter to make it into an .app. It's my first my using Tkinter and pyinstaller so my code is probably not the cleanest. I did have similar problems when executing the generated app, I solved them by force adding the libraries until I stopped getting errors, again, not an expert on pyinstaller so there might be better ways to do this.

im trying to use your converter but im getting this error global_step key not found in model. Btw im using custom model from civitai

[jrittvo]

I get the same global step key error working straight with the python scripts in terminal, but the models convert and work fine, so I ignore the error. Does the error stop the app from completing conversions, or is it just a warning that pops up somewhere?

[pandora523]

I get the same global step key error working straight with the python scripts in terminal, but the models convert and work fine, so I ignore the error. Does the error stop the app from completing conversions, or is it just a warning that pops up somewhere?

its stops there and back from the start..... Update I think I found the root of the problem im trying to convert model with merge dreambooth model (rev+with my freaking face)

KeyError: 'encoder.conv_in.weight'
[15164] Failed to execute script 'torch2coreml' due to unhandled exception: 'encoder.conv_in.weight'
[15164] Traceback:
Traceback (most recent call last):
File "python_coreml_stable_diffusion/torch2coreml.py", line 1513, in
File "python_coreml_stable_diffusion/torch2coreml.py", line 1281, in main
File "diffusers/pipelines/stable_diffusion/convert_from_ckpt.py", line 1168, in download_from_original_stable_diffusion_ckpt
File "diffusers/pipelines/stable_diffusion/convert_from_ckpt.py", line 583, in convert_ldm_vae_checkpoint
KeyError: 'encoder.conv_in.weight'
global_step key not found in model

[jrittvo]

If you find it is an issue with a specific model, and you also happen to use Automatic1111, you can try running the model through the "Basic" tab of the Model Toolkit extension, and see if it cleans anything out. If so, the "cleaned" model may very well convert successfully afterwards. This is the extension I'm referring to: https://github.com/arenasys/stable-diffusion-webui-model-toolkit.git . With the "Basic" tab, you just select a model, hit the "Load" button, and it tells you if it found things to fix in 15 seconds or so. If it found things to fix, you then hit the "Save" button, and the fixed model is saved with a new name to your model folder. There is also an "Advanced" tab that can do other things, but I've never understood what is going on there.

[pandora523]

If you find it is an issue with a specific model, and you also happen to use Automatic1111, you can try running the model through the "Basic" tab of the Model Toolkit extension, and see if it cleans anything out. If so, the "cleaned" model may very well convert successfully afterwards. This is the extension I'm referring to: https://github.com/arenasys/stable-diffusion-webui-model-toolkit.git . With the "Basic" tab, you just select a model, hit the "Load" button, and it tells you if it found things to fix in 15 seconds or so. If it found things to fix, you then hit the "Save" button, and the fixed model is saved with a new name to your model folder. There is also an "Advanced" tab that can do other things, but I've never understood what is going on there.

it did fixed something on automatic 1111/extension but still got the error 😅 maybe il give-up putting my face in mochi diffusion

Traceback (most recent call last):
File "python_coreml_stable_diffusion/torch2coreml.py", line 1513, in
File "python_coreml_stable_diffusion/torch2coreml.py", line 1281, in main
File "diffusers/pipelines/stable_diffusion/convert_from_ckpt.py", line 1168, in download_from_original_stable_diffusion_ckpt
File "diffusers/pipelines/stable_diffusion/convert_from_ckpt.py", line 583, in convert_ldm_vae_checkpoint
KeyError: 'encoder.conv_in.weight'
[15975] Failed to execute script 'torch2coreml' due to unhandled exception: 'encoder.conv_in.weight'
[15975] Traceback:
Traceback (most recent call last):
File "python_coreml_stable_diffusion/torch2coreml.py", line 1513, in
File "python_coreml_stable_diffusion/torch2coreml.py", line 1281, in main
File "diffusers/pipelines/stable_diffusion/convert_from_ckpt.py", line 1168, in download_from_original_stable_diffusion_ckpt
File "diffusers/pipelines/stable_diffusion/convert_from_ckpt.py", line 583, in convert_ldm_vae_checkpoint
KeyError: 'encoder.conv_in.weight'
global_step key not found in model

[jrittvo]

Put a link here to the model and I'll try converting it with command line tools in the next day or two. I'll post back here after I give it a shot. If it works, I'll upload the converted models for you to grab. And I didn't ask before, but is it a SD-1.5 based model? SD-2.1 based models are hard to convert and most won't run in Mochi anyway because they need modified Schedulers/Samplers.

[pandora523]

Put a link here to the model and I'll try converting it with command line tools in the next day or two. I'll post back here after I give it a shot. If it works, I'll upload the converted models for you to grab. And I didn't ask before, but is it a SD-1.5 based model? SD-2.1 based models are hard to convert and most won't run in Mochi anyway because they need modified Schedulers/Samplers.

it turned out to be like this 😅

I don't want to bother you so ill pass and thanks for the offer I really appreciate it❤️ 🚀.
I think il stick to 1111 for a while and "--medvram" args cause I just read that to much memory swap can kill my Mac .😂 So idid look for alternatives and wallah here it is and I'm amaze how ml core model performs on m2 air but I don't have much option to customize my Creations /Deepest Desires 🤣.