[FEAT]: A drop down for keeping endpoints info saved under Generic OpenAI LLM service.

[OzMan21]

What would you like to see?

When under the LLM global settings with the Generic OpenAI selected, it would be a great utility to have a dropdown that'd keep saved various endpoints and their respective API's and models, and make them easily selectable. Currently it is such that, we need to manually enter this information each time, and this is not only taking a lot of time to source this info from multiple different screens, but it also is creating probability for mistakes which then break operation.

In my case, I use RunPod to have various workers running different models, each a different endpoint, then there can be other servers running in the local network serving other LLM endpoints, so this utility is much desired, I imagine, to quickly and safely switch endpoints.

Thank you for this amazing UI and all the work Mindplex/Tim and all the developers have done! We love you very much for all you do.

[timothycarambat]

This is a great solution, we long since have been needing to move to "stored connections" so that we can persist them and have many separate connections - this would be under that.

Moving #2002 #1567 into this issue since it is more current

[WickedSquirrel]

Exactly what I came here to say

I use multiple models of the same external provider and have to always go back and copy and paste in the modellname I want to use.

[chaserhkj]

I second this improvement. I would also suggest to support switching between different configured providers and models in the chat interface as well, currently you can switch between the models of other providers, but not between different generic providers (or different models of the same provider)

This is relevant to my setup because I use LiteLLM proxy to do cost management and load balancing both on my own hardware and on different providers for a number of different LLM apps. And currently in AnythingLLM there is no way to use more than one model from such a setup.

[shanevcantwell]

Hi Tim! I've been using AnythingLLM for close to 2 years now, so when looking to plug my own langgraph project into an existing UI to handle the chat context accumulation and threading/editing, I knew exactly what I wanted to use. My custom LAS project works great through the Generic OpenAI adapter. However, I also use llama.cpp llama-server that already goes through the single Generic OpenAI. I need a way to have 2 (or more, since I also switch between 2 workstations that multiply the possible configurations) unique host configurations where neither fits into any other existing LLM service adapter. Cheers

[petealbertson]

Given that this issue is well over a year old, my guess is this will never be implemented, but I just wanted to add my $0.02 that it's pretty much a deal-breaker not to have the option to set up multiple generic OpenAI providers, nor be able to switch models within a given generic provider.

[Souravrajvi0]

Hi everyone thanks for the patience on this one, and especially to those who commented recently (including @petealbertson)

I opened a scoped PR that tries to deliver the core ask for Generic OpenAI: save multiple endpoint profiles (base URL, API key, model, token limits) and switch between them from a dropdown in LLM Preference, without retyping everything each time.

PR: #6065

This follows the direction @timothycarambat outlined here a first step toward broader “stored connections,” Generic OpenAI only for now (not full chat UI model switching yet).

I tested locally (unit tests, API smoke tests, frontend build, and Docker compose build/run). Happy to adjust shape/scope if the team wants something different before merge — just trying to help unblock the RunPod / LiteLLM / multi-endpoint workflows people described in the thread.

Appreciate any review or feedback.