multiple api keys for each provider

[tecworks-dev]

add multiple api keys for each provider to failback to if rate limited

[S4tyendra]

Much needed!

[deiangi]

I totally second this!
it would be great if you want to test and compare different models from openrouter for example - or even locally.

[deiangi]

its bit of overlap with #179

[samhvw8]

#240 can suitable with this

[bkutasi]

I was thinking of making a new discussion for this, but here is my proposal how this could work:
User sets up multiple configs (implemented now)
User sets up failover configs with relation to each other and within one config picks a custom amount of failover configs.

It could also be done within just one config:
Under the API provider setting with "+" button. If the call fails or an error happens, the api key would be rotated to the next one. You can get pretty creative with this: gemini free models are pretty limited, and google provides the ai studio and vertex for this, you could rotate 2 or more keys.

[mrubens]

@bkutasi how are you thinking auto failover would work? Maybe for each profile you could point to another profile as the fallback, and it would automatically switch after a few consecutive errors? What do you think of that @samhvw8 ?

[samhvw8]

@bkutasi how are you thinking auto failover would work? Maybe for each profile you could point to another profile as the fallback, and it would automatically switch after a few consecutive errors? What do you think of that @samhvw8 ?

This will be a good PR

[bkutasi]

@mrubens Having a + sign under the normal API provider within one profile would be better than profiles pointing to each other, I think. So within one profile you could set up multiple keys for one provider, that way you can keep one "mega-profile" for one provider and quickly switch between others if you need to with the profiles switch in the textbox.
The easy solution would be just adding another "API Provider" section each time the user presses the + button, it could lead to really convoluted UI but for initial implementation would be okay I guess. Later down the line maybe another tab would be better for providers like MCP servers.

[mrubens]

@bkutasi one dumb detail that I got kind of stuck on when I looked at the solution you’re suggesting before - how would you differentiate between the different keys for the provider as an end user? I assume we still want to keep them at least mostly hidden, right? Would showing the first and last few characters work, or letting the user label the keys?

[bkutasi]

@mrubens I think api keys are ephemeral by nature, for example, I have 4 deepseek keys because of multiple workstations where I work with roocline. Unfortunately, I don't think there is an easy way to differentiating between them, all info the user are the parts of the key: sk-156cd***********************182d
edit: I didn't actually answer the question. I think similar censoring than above would work, first 4 characters and last 4 characters. Adding a note would be little but more clutter in the UI, maybe I would give the option for users to manage this in a config file rather than the UI? Not sure about that. Its easy to make a new key if you mess something up.

[theguy000]

The feature is already here.

[benzntech]

Can Provide more details. There are proper documentation for it.

[benzntech]

Need to drop down all the popular LLM list under under the Open router Module

[TomLucidor]

Seconding this for everything

[samhvw8]

Seconding this for everything

did we have this feature ?