Development: Instance Manager 📖

[Jeffser]

Hi everyone, this is very early work for the new instance manager!

This will let you connect multiple online AI providers and Ollama instances.

Of course some features like the model creator will be exclusive for Ollama though.

Again, this is very early development and I'm in the middle of 6 different projects in college, so don't expect this to come out this week.

[Jeffser]

Providers I will add:

• Ollama (duh)
• OpenAI
• Gemini
• Deepseek

Please remember that Deepseek and Google provide free local models for Ollama, you can start using them today with Alpaca!

[olumolu]

https://www.together.ai/
provide a private way to use all models online
https://venice.ai/api-beta
another private ai provider
https://docs.mistral.ai/api/
https://docs.anthropic.com/en/api/getting-started

[Jeffser]

Together and Venice seem easy to integrate with the system I got going on, can't promise anything Mistral and Anthropic but I'll try.

[Jeffser]

Haven't pushed it yet to the repo because it breaks message generation but this is the first time Alpaca has connected to something other than Ollama!

The models are listed directly from OpenAI, no hardcoded list!

[Jeffser]

Gemini officially works with Alpaca!!!!

[Jeffser]

Chat generation has been ported to the new system

    def generate_chat_title(self, chat, prompt:str, model:str):
        class chat_title(BaseModel): #Pydantic
            title:str
            emoji:str

        completion = self.client.beta.chat.completions.parse(
            temperature=0.2,
            model=model,
            messages=[
                {"role": "system", "content": "You are an assistant that generates short chat titles based on the first message from a user"},
                {"role": "user", "content": "Generate a title for this prompt:\n{}".format(prompt)}
            ],
            response_format=chat_title
        )

        response = completion.choices[0].message
        if response.parsed:
            window.chat_list_box.rename_chat(chat.get_name(), '{} {}'.format(response.parsed.emoji, response.parsed.title))

Using a structured output makes it really consistent and it will always include an emoji now!

[Jeffser]

Error handling

All the ported instance types (ollama:managed only appears if Alpaca detects an Ollama binary, always in Snap and Flatpak)

[Jeffser]

Finally pushed it to the repo, there's still a lot left to do tho

8576dc6