[Question] How to use GGUF?

[phamkhactu]

Thanks for your great working!

I've used vace-1.3B to test with your example, but the quality isn't good enough when comparing with output in benchmark. I think the model is very small that the result is not good. Therefore, I want to use quantization version likes GGUF(14B) to test again.

I'm very happy for your help.