Feature Request: Support for local AI components

[nickgnat]

As a homelabber and board game nerd, I love the project! I saw another github issue suggesting making AI features optional, but wanted to offer another option -- many selfhosted apps integrate LiteLLM to allow the use of a variety of LLMs including self-hosted Ollama, would this be an option you'd consider? On the vector DB side, Qdrant is a popular option that allows for both self-hosting and cloud use.

I don't necessarily have the know-how to integrate these myself, but I could certainly scaffold together a docker-compose.yml that provided those components and test out any integrations!

[TommyTheHuman]

Hi, it's a pleasure to hear that you like this project! I have a few minor client-side features to develop. Next on the list could be adding the option to use both the LLM and the vector database locally.
If you have any suggestions for additional features, feel free to share them.
I'll keep this issue open until I implement this new functionality

[enricollen]

@nickgnat your comment is spot on. I can try to implement LiteLLM to make the LLM choice easier if @TommyTheHuman agrees. Regarding Qdrant, you're right, usually it is the way to go, never used it btw. I could look into that as well

[TommyTheHuman]

@enricollen I'm open to new features regarding the RAG module, you are the expert here ;)