Gemini Limitations

[sthomson-wyn]

This isn't necessarily a request to fix this, as there are workarounds. Just wanted to document some things I ran into while trying to set this plugin up with Gemini to hopefully save somebody else some time.

Gemini has an openai compatible endpoint: https://ai.google.dev/gemini-api/docs/openai however, due to the default for a "custom" provider having the "/v1" path appended to it

grafana-llm-app/packages/grafana-llm-app/pkg/plugin/openai_provider.go

Line 27 in 63c68d5

base, err := url.JoinPath(settings.URL, "/v1")

it won't work without a proxy (google expects https://generativelanguage.googleapis.com/v1beta/openai)

Also, when choosing a proxy, it's important to consider that Gemini doesn't support "role": "system" messages. I tried https://github.com/zhu327/gemini-openai-proxy but that doesn't change the role automatically. https://github.com/zuisong/gemini-openai-proxy does, however.

So in the end I got it working by using https://github.com/zuisong/gemini-openai-proxy.

It would be nice if this plugin offered a way to configure the default path and the default role, but the zuisong proxy should be good enough for most people to get going with Gemini.