Try a different LLM

[SakthiMahendran]

As people mentioned before LLaMa is very slow due it's huge size... So why don't we try different models for example
"phi3" from microsoft.... in the context of day to day uses llama and phi3 won't have big difference.... but phi3 is lightwieght and is much more quick than llama. It has 3.8B parameters.... If we wan't to make to useable then LLaMa won't be good choice...

try "ollama run phi3"... You will get an idea

[FotieMConstant]

Hey there @SakthiMahendran thanks for the input on this. I really appreciate you suggestion. any thoughts on fine-tuning phi3? perhaps we can start from there. I have never fine-tuned it before and I suppose the fine-tuning processing is different from llama.

let me know. Thanks!

[iameijaz]

@FotieMConstant I am working on a similar project involving fine-tuning of phi3 model due to the faster responses. I will let you know if it is fruitful in this case.

[FotieMConstant]

@FotieMConstant I am working on a similar project involving fine-tuning of phi3 model due to the faster responses. I will let you know if it is fruitful in this case.

Perfect, thanks. I’ve also been thinking we could fine-tune qwen3.5-4b, the model is multimodal and works even on edge devices. Any thoughts?

[iameijaz]

Qwen is good for coding ,but not for language understanding .