Add openrouter endpoint

[stojko-stojan]

Can you add openrouter endpoint so we can use different models with their API

git-bob implement this

[github-actions]

^{This message was generated by git-bob (version: 0.19.1, model: claude-3-5-sonnet-20241022, log), an experimental AI-based assistant. It can make mistakes and has limitations. Check its messages carefully.}

Hi @stojko-stojan,

thanks for reaching out! Unfortunately, I'm not allowed to respond to you directly.
I need approval from a person who has access.

Best,
git-bob

[haesleinhuepf]

I like the idea very much @stojko-stojan !

git-bob comment on how to achieve this.

[github-actions]

^{This message was generated by git-bob (version: 0.19.1, model: claude-3-5-sonnet-20241022, log), an experimental AI-based assistant. It can make mistakes and has limitations. Check its messages carefully.}

The OpenRouter implementation would be quite similar to the existing endpoint implementations. Here's a suggested addition to the _endpoints.py file:

def prompt_openrouter(message: str, model=None, image=None, max_accumulated_responses=10, max_response_tokens=16384):
    """A prompt helper function that sends a message to OpenRouter"""
    import os
    if model is None:
        model = "anthropic/claude-3-opus"
    return prompt_openai(
        message, 
        model=model, 
        image=image, 
        max_accumulated_responses=max_accumulated_responses, 
        max_response_tokens=max_response_tokens, 
        base_url="https://openrouter.ai/api/v1",
        api_key=os.environ.get("OPENROUTER_API_KEY")
    )

Also add this test to test_endpoints.py:

@pytest.mark.manual
def test_openrouter():
    from git_bob._endpoints import prompt_openrouter
    assert "ok" in str(prompt_openrouter("Answer with ok!")).lower()

You'll need to set the OPENROUTER_API_KEY environment variable with your API key from openrouter.ai

[haesleinhuepf]

Does this make sense @stojko-stojan ?

[stojko-stojan]

According to https://openrouter.ai/docs/api-reference/authentication#using-an-api-key
If you call API directly you need to have have bearer before API key , if you use openai SDK than you don't.

[haesleinhuepf]

Ok, great. git-bob implement this as you proposed. Furthermore, please add a new prompt-handler to setup.cfg called "openrouter" which calls the new endpoint.

[stojko-stojan]

Ok, so if he calls prompt_openai that already uses openai sdk he don't have to have bearer before token. I think he was right from the start

[stojko-stojan]

As for the response formatting in prompt_openai not sure if it will suit open routers response. Because you could choose reasoning model that have think tokens and also ordinary models that don't have them.

[haesleinhuepf]

Indeed, git-bob doesn't have any special treatment for reasoning models. Worth discussing in another issue how to approach this in a generic way...