Is there a comprehensive way of using differend models or is elbow grease needed?

[ErickMaRi]

Hey everyone,

I've been using private-GPT, and it's been working great so far. However, I'm curious about incorporating different models into it. I was wondering if anyone has found a comprehensive way of using multiple models with private-GPT, or if it requires some manual effort.

There are two main questions I have:

1. Is it possible to integrate models other than GPT-4all into private-GPT? If so, how can I go about doing that?

2. Are there any specific considerations or challenges when using another model within theprivate-GPT?

Looking forward to your answers, I'm amazed it actually runs in my laptop!

Best regards,
Erick Marín Rojas

[tedraymond]

My guesss today is elbow grease.

Something to redefine settings.yaml model and restart.

I've been wondering the same while exploring orca and dolphin. Giving users a different experience based on the type of interaction.

[FaizalJnu]

Did you find anything yet? I am stuck on the same issue, is there a way to simply change some files to use a different gguf format model from huggingface? Like maybe a smaller 2B parameter model.

[dbzoo]

I suggest using ollama allowing you to switch models. For example to use dolphin-mistral
$ ollama pull dolphin-mistral
And use the model like this - see settings-ollama.yaml for expanded setup details.

llm:
  mode: ollama
  
ollama:
  llm_model: dolphin-mistral