First, I'd like to thank you for the efforts.
Second, this may not be the best place to ask, but I'm trying to run MADLAD400-3B on an old hardware (4GB RAM, don't laugh), llama.cpp couldn't load these ones:
https://huggingface.co/google/madlad400-3b-mt/tree/main
But it could load this one:
https://huggingface.co/notjjustnumbers/madlad400-3b-mt-Q4_K_M-GGUF/tree/main
Though the model didn't translate my prompts, it just hallucinated.
Can you add support for smaller models, or maybe show me how to use them correctly?
Thanks in advance!
First, I'd like to thank you for the efforts.
Second, this may not be the best place to ask, but I'm trying to run MADLAD400-3B on an old hardware (4GB RAM, don't laugh), llama.cpp couldn't load these ones:
https://huggingface.co/google/madlad400-3b-mt/tree/main
But it could load this one:
https://huggingface.co/notjjustnumbers/madlad400-3b-mt-Q4_K_M-GGUF/tree/main
Though the model didn't translate my prompts, it just hallucinated.
Can you add support for smaller models, or maybe show me how to use them correctly?
Thanks in advance!