Grafana LLM App Admin Page UX Issues and Bugs

[manoja1]

Please watch the recording for the UX issues and Bugs. In summary

• It's not evident what's in use - Grafana's own vs Bring your own
• MOdel name section should belong to Bring your own
• Save & Test results in wrong configuration being saved when the test fails later
• Switching between models doesn't prompt me to update the model name

grafana-llm-app-admin-page-issues.mp4

cc: @Maurice-L-R @devinrcheevers @brittenhouse @vaugun @okianus

[iamrajiv]

hi @manoja1 sorry for jumping in out of nowhere but yeah i have faced the same issues when running this locally and it totally makes sense to have validation on every check and click so we fail fast and yeah about the ui suggestion moving modelconfig inside the provider specific cards is obvious

@sd2k i can take care of these fixes basically we just need some ui tweaks and validations i’ll raise a draft pr and if others can review that’d be great since i already have this running locally so it’ll be easier if that’s okay?

[iamrajiv]

update draft pr on above issue #762

also @sd2k i was trying to use gemini using custom api but the thing is to use gemini with the custom api option we need to configure it as an openai compatible endpoint i know only one tool to make it happen that is https://docs.litellm.ai/docs/simple_proxy

so the thing is should we add and provide docs on this how to configure it as an openai compatible endpoint its easy for users to get started using this and gemini is important since we get free api usage wdyt?

[sd2k]

@iamrajiv we should use a separate issue for Gemini stuff! There is this existing one which might suffice.

I'll take a look at your PR soon, thanks!

[iamrajiv]

sure thanks @sd2k

[manoja1]

I also sometimes get this error when I do Save & Test .. and takes 5/10 hard resets before it goes away

[vaugun]

Here's an example of how we could restructure the selection and grouping of information on this page to more clearly communicate what information related to what has been selected. It's mostly the upper half of the page that needs to be refactored.

On the left, we have the "provided by Grafana" option selected with T&Cs to accept. On the right, we have a bring your own API option selected, which displays the API config form in close proximity within the same section.

PS: Ignore the incorrect data in the "models" section, I just used screenshots to fill in the rest to avoid recreating the entire page.

If you have any thoughts or questions, drop a comment. Happy to discuss more or elaborate :)

[sd2k]

@vaugun one comment based on this issue and the draft PR by @iamrajiv - it makes sense to group together the 'Model provider' and 'Models' sections more tightly, because you're likely to need to change the latter whenever you change the former. The second screenshot in #762 shows an example.

Otherwise I think it looks good 👍 thanks!

[vaugun]

@sd2k Roger that, makes sense! I'll update accordingly 🫡

[vaugun]

@sd2k it's a minor update, but just reposting the adjusted wireframes in their entirety for clarity sake – so everybody else who stumbles on this issue later knows what we know 🙂

Quick TL;DR of the update: I've removed the separate "models" section heading, and instead merged that field set in under the "model provider" section heading in the wireframes below.

[sd2k]

Reopening to track the UX update by @vaugun