support user-provided AI model providers (OpenAI, Anthropic, Ollama)

[AjayBandiwaddar]

Problem

While setting up Sugar-AI locally, I noticed that app/ai.py only
supports HuggingFace models through the RAGAgent class. There's no
way for users to plug in their own OpenAI, Anthropic, or Ollama models.

This means if a school has an OpenAI API key and wants to use it with
Sugar-AI, they currently can't. Passing openai/gpt-4 to the
/change-model endpoint would crash the app since it tries to load
it as a HuggingFace model.

Proposed Solution

Add provider detection in app/ai.py, specifically in the __init__
and set_model methods of RAGAgent that checks the model string
prefix and routes to the correct client.

This way run and run_chat_completion would work the same way
regardless of provider.

[Naitik120gupta]

I think a testing version of this has been already launched #45

[AjayBandiwaddar]

I think a testing version of this has been already launched #45

Thanks for pointing that out! I looked at PR #45 and it focuses
on fixing DEV_MODE handling and preventing duplicate RAGAgent
initialization during startup — which is a different problem.

Issue #51 is specifically about adding support for external AI
provider APIs (OpenAI, Anthropic, Ollama) so users can bring
their own API keys. Currently app/ai.py only supports loading
HuggingFace models locally, with no routing for external providers.

[Naitik120gupta]

I think a testing version of this has been already launched #45

Thanks for pointing that out! I looked at PR #45 and it focuses on fixing DEV_MODE handling and preventing duplicate RAGAgent initialization during startup — which is a different problem.

Issue #51 is specifically about adding support for external AI provider APIs (OpenAI, Anthropic, Ollama) so users can bring their own API keys. Currently app/ai.py only supports loading HuggingFace models locally, with no routing for external providers.

Thanks for opening this — this is actually a different scope from #45.

PR #45 improved DEV_MODE handling and made model selection configurable via .env, but it still assumes HuggingFace-based models loaded through the transformers pipeline.

What you're proposing is a provider abstraction layer (OpenAI, Anthropic, Ollama, etc.), which would require:

• Detecting provider prefixes
• Initializing appropriate API clients
• Standardizing run and run_chat_completion across providers

That would likely benefit from introducing a provider interface.

I think this is a valuable direction, especially if we want Sugar-AI to be deployment-flexible. Would you be open to drafting a minimal provider abstraction proposal first before modifying RAGAgent directly?

[AjayBandiwaddar]

I think a testing version of this has been already launched #45

Thanks for pointing that out! I looked at PR #45 and it focuses on fixing DEV_MODE handling and preventing duplicate RAGAgent initialization during startup — which is a different problem.
Issue #51 is specifically about adding support for external AI provider APIs (OpenAI, Anthropic, Ollama) so users can bring their own API keys. Currently app/ai.py only supports loading HuggingFace models locally, with no routing for external providers.

Thanks for opening this — this is actually a different scope from #45.

PR #45 improved DEV_MODE handling and made model selection configurable via .env, but it still assumes HuggingFace-based models loaded through the transformers pipeline.

What you're proposing is a provider abstraction layer (OpenAI, Anthropic, Ollama, etc.), which would require:

• Detecting provider prefixes
• Initializing appropriate API clients
• Standardizing run and run_chat_completion across providers

That would likely benefit from introducing a provider interface.

I think this is a valuable direction, especially if we want Sugar-AI to be deployment-flexible. Would you be open to drafting a minimal provider abstraction proposal first before modifying RAGAgent directly?

Thanks for the clarification and the detailed breakdown.

Agree that a provider interface makes more sense than directly
modifying RAGAgent. Something like an abstract BaseProvider with
concrete implementations for each (OpenAI, Anthropic, Ollama,
HuggingFace) — and RAGAgent just delegates to whichever is active.

Happy to draft a minimal design for this. Should I put it here
as a comment or open a separate discussion?

[Naitik120gupta]

I think a testing version of this has been already launched #45

Thanks for pointing that out! I looked at PR #45 and it focuses on fixing DEV_MODE handling and preventing duplicate RAGAgent initialization during startup — which is a different problem.
Issue #51 is specifically about adding support for external AI provider APIs (OpenAI, Anthropic, Ollama) so users can bring their own API keys. Currently app/ai.py only supports loading HuggingFace models locally, with no routing for external providers.

Thanks for opening this — this is actually a different scope from #45.
PR #45 improved DEV_MODE handling and made model selection configurable via .env, but it still assumes HuggingFace-based models loaded through the transformers pipeline.
What you're proposing is a provider abstraction layer (OpenAI, Anthropic, Ollama, etc.), which would require:

• Detecting provider prefixes
• Initializing appropriate API clients
• Standardizing run and run_chat_completion across providers

That would likely benefit from introducing a provider interface.
I think this is a valuable direction, especially if we want Sugar-AI to be deployment-flexible. Would you be open to drafting a minimal provider abstraction proposal first before modifying RAGAgent directly?

Thanks for the clarification and the detailed breakdown.

Agree that a provider interface makes more sense than directly modifying RAGAgent. Something like an abstract BaseProvider with concrete implementations for each (OpenAI, Anthropic, Ollama, HuggingFace) — and RAGAgent just delegates to whichever is active.

Happy to draft a minimal design for this. Should I put it here as a comment or open a separate discussion?

You can open a Pull request for this as a Phase1 prototype. Do not squash all the changes in a single commit, Do it step by step and makesure the read the guidelines for contributing https://github.com/sugarlabs/docs/blob/master/src/contributing.md and opening a pull request in sugar.
Thanks for contributing.

[AjayBandiwaddar]

I think a testing version of this has been already launched #45

Thanks for pointing that out! I looked at PR #45 and it focuses on fixing DEV_MODE handling and preventing duplicate RAGAgent initialization during startup — which is a different problem.
Issue #51 is specifically about adding support for external AI provider APIs (OpenAI, Anthropic, Ollama) so users can bring their own API keys. Currently app/ai.py only supports loading HuggingFace models locally, with no routing for external providers.

Thanks for opening this — this is actually a different scope from #45.
PR #45 improved DEV_MODE handling and made model selection configurable via .env, but it still assumes HuggingFace-based models loaded through the transformers pipeline.
What you're proposing is a provider abstraction layer (OpenAI, Anthropic, Ollama, etc.), which would require:

• Detecting provider prefixes
• Initializing appropriate API clients
• Standardizing run and run_chat_completion across providers

That would likely benefit from introducing a provider interface.
I think this is a valuable direction, especially if we want Sugar-AI to be deployment-flexible. Would you be open to drafting a minimal provider abstraction proposal first before modifying RAGAgent directly?

Thanks for the clarification and the detailed breakdown.
Agree that a provider interface makes more sense than directly modifying RAGAgent. Something like an abstract BaseProvider with concrete implementations for each (OpenAI, Anthropic, Ollama, HuggingFace) — and RAGAgent just delegates to whichever is active.
Happy to draft a minimal design for this. Should I put it here as a comment or open a separate discussion?

You can open a Pull request for this as a Phase1 prototype. Do not squash all the changes in a single commit, Do it step by step and makesure the read the guidelines for contributing https://github.com/sugarlabs/docs/blob/master/src/contributing.md and opening a pull request in sugar. Thanks for contributing.

Thanks! I'll read the contributing guidelines and open a Phase 1 PR implementing the BaseProvider abstraction. I'll keep commits incremental as suggested.

[AjayBandiwaddar]

I think a testing version of this has been already launched #45

Thanks for pointing that out! I looked at PR #45 and it focuses on fixing DEV_MODE handling and preventing duplicate RAGAgent initialization during startup — which is a different problem.
Issue #51 is specifically about adding support for external AI provider APIs (OpenAI, Anthropic, Ollama) so users can bring their own API keys. Currently app/ai.py only supports loading HuggingFace models locally, with no routing for external providers.

Thanks for opening this — this is actually a different scope from #45.
PR #45 improved DEV_MODE handling and made model selection configurable via .env, but it still assumes HuggingFace-based models loaded through the transformers pipeline.
What you're proposing is a provider abstraction layer (OpenAI, Anthropic, Ollama, etc.), which would require:

• Detecting provider prefixes
• Initializing appropriate API clients
• Standardizing run and run_chat_completion across providers

That would likely benefit from introducing a provider interface.
I think this is a valuable direction, especially if we want Sugar-AI to be deployment-flexible. Would you be open to drafting a minimal provider abstraction proposal first before modifying RAGAgent directly?

Thanks for the clarification and the detailed breakdown.
Agree that a provider interface makes more sense than directly modifying RAGAgent. Something like an abstract BaseProvider with concrete implementations for each (OpenAI, Anthropic, Ollama, HuggingFace) — and RAGAgent just delegates to whichever is active.
Happy to draft a minimal design for this. Should I put it here as a comment or open a separate discussion?

You can open a Pull request for this as a Phase1 prototype. Do not squash all the changes in a single commit, Do it step by step and makesure the read the guidelines for contributing https://github.com/sugarlabs/docs/blob/master/src/contributing.md and opening a pull request in sugar. Thanks for contributing.

I've opened PR #55 as the Phase 1 prototype: #55