New ideas, models, goodies, etc...

[Amin456789]

Hi!
here i post new stuff and ideas such as unique models that are already converted or their tech needs to be implemented first.

i will put good things here that are being posted here and there so we can have a permanent place and data wont be buried when the topic is closed

[Amin456789]

for started i found this:

https://huggingface.co/akameswa/tiny-sd-onnx-fp16

this is tiny sd, it is one of the smallest models which is from this i believe:
https://huggingface.co/segmind/tiny-sd
correct me but i think these models are listed as 'distilled', they have their own pipeline which will be great someday we have them here too:
https://huggingface.co/docs/diffusers/main/en/using-diffusers/distilled_sd

as the official page says:
"distilled models are upto 80% faster than the Base SD1.5 Models"

[Amin456789]

anime based LCM model which needs to be converted to onnx first:
https://huggingface.co/furusu/LCM-Acertainty

[Amin456789]

tutorial on how to convert sd models to ONNX fp16 for cpu and gpu by @dakenf:
https://github.com/saddam213/OnnxStack/issues/9#issuecomment-1803320472

[Amin456789]

int8 quantized sd models which are small but quality is lower sometimes:
https://huggingface.co/ClashSAN/miniSD-quantized-onnx/tree/main

https://huggingface.co/ClashSAN/miniSD-anything-v3-swapped-onnx-quantized/tree/main
[use int8 folder in it as it is smaller]

the script that is being used [needs to be updated as i remember needs older onnx versions, versions that are lower 16-15 i think:
https://github.com/LowinLi/stable-diffusion-streamlit/blob/main/src/stable-diffusion-streamlit/pages/model/quantization.py

[Amin456789]

please feel free to share if u ever found something interesting here

[Amin456789]

Real-Time LCM!

https://github.com/radames/Real-Time-Latent-Consistency-Model

this is something amazing, didn't try it, but with the looks of it u can use ur webcam and turn urself to anything in real time! will be cool someday we have it here too

[Amin456789]

fp16 model lf LCM-Dreamshaper v7:
https://huggingface.co/aislamov/lcm-dreamshaper-v7-onnx/tree/main

[jdluzen]

This one now works and is my default. Noticeably less vram usage.

[Amin456789]

me too! i love it

[Amin456789]

some more models [sd 1.5], they are directml fp16:

https://huggingface.co/sharpbai/stable-diffusion-v1-5-onnx-directml-fp16
https://huggingface.co/sharpbai/chilloutmix-onnx-directml-fp16

u can find fp32 and more versions on sharpbai's page

chilloutmix is excactly what i was looking for! it even can generate low resolution images 256x256 320x320 384x384 in very high quality

[Amin456789]

a tip to generate realistic images with lcm dreamshaper:
give it a location sometimes, and use = real, detailed human image, sharp focus
example:
a handsome man with brown hair walking, in a beautiful alley in japan, real, detailed human image, sharp focus

[saddam213]

New Model template to Import into OnnxStack

Deliberate-v3
deliberate-v3.json

Download file, Open OnnxStack, go to Model section, click Import, select the json file

[Amin456789]

my new favorite way to make new images is one of those 2x2frames with sd, have it 512x512 in lcm, activate automation to generate batch of images and:
quadtych photo of a woman talking to another man, in japan, detailed human image, sharp focus, real

now what it does is gives u an image with multiple frames in it and if u get lucky those images will be frames of same people with different motions!
u can easily crop the frames out of the image, put them together and make a gif or put it in flowframes [another ai app to generate more frames and make it more smooth]. voila, u have an animated image.

[Amin456789]

a bit off topic:
@saddam213 after ur onnxstack and maybe ur other projects, do u have any plan for bringing sound and music to c# onnx? i tried AudioCraft and it's pretty amazing! a c# onnx version will be great:
https://github.com/facebookresearch/audiocraft
https://github.com/GrandaddyShmax/audiocraft_plus

[Amin456789]

some olive optimized fp16 models [Realistic Vision, Deliberate, Photon and Dreamshaper]:
https://huggingface.co/softwareweaver/realisticvision/tree/main
https://huggingface.co/softwareweaver/deliberate/tree/main
https://huggingface.co/softwareweaver/photon/tree/main
https://huggingface.co/softwareweaver/dreamshaper/tree/main

LCM-Dreamshaper v7 Olive optimized:
https://huggingface.co/softwareweaver/LCM_Dreamshaper_v7_Olive_Onnx
more here:
https://huggingface.co/softwareweaver

[NeusZimmer]

HI, recently reached this forum... will follow it...as you are sharing resources, will like to share my own resources. a UI for onnx to low mem gpu or cpu, giving control on where the models are loaded during runtime , and also including a hi-res pipeline approach....(also, LCM support is on development and almost ready to be implemented...)
https://github.com/NeusZimmer/ONNX-ModularUI-StableDiffusion
PD: i moved from .Net to python to work with stable diffusion models good to see someone working on .Net approach, will try to contribute

[jeffward01]

Ideas0:

• Ability to use Refiners (two or more checkpoints to generate an image). More than (1) refiner would be very cool.
• Ability to convert .pth, .safetensors, and .ckpt, to .onnx from tools build into or side-by-side with the app (it will improve-usability)
• ability to more easily swap out models in the codebase. Perhaps this is already implemented and I just have not explored this yet. I will read the codebase on this now