Skip to content

Add support for OpenAI-compatible API endpoints#60

Merged
DiogoRDuarte merged 14 commits into
feedzai:developfrom
joaopalmeiro:feat-jp-57-add-support-open-ai-compatible-endpoints
Sep 8, 2025
Merged

Add support for OpenAI-compatible API endpoints#60
DiogoRDuarte merged 14 commits into
feedzai:developfrom
joaopalmeiro:feat-jp-57-add-support-open-ai-compatible-endpoints

Conversation

@joaopalmeiro

@joaopalmeiro joaopalmeiro commented Sep 5, 2025

Copy link
Copy Markdown
Member

Closes #57

Test plan

Install Ollama (if necessary).

Start Ollama:

ollama serve

Download an LLM (Gemma 3 270M or another):

ollama pull gemma3:270m

Install the documentation dependencies:

npm run build
npm run docs:install

Launch the documentation:

npm run docs:dev

Select Option A (Automatic Descriptions) and fill out the form with the following values:

  • API key: ollama
  • Model: gemma3:270m
  • Base URL: http://localhost:11434/v1/ or http://localhost:11434/v1

Since the page contains 8 charts, it will take a bit to get all the descriptions. Monitor them from the Ollama logs. Additionally, since models like Gemma 3 270M are small models (sometimes designed for later fine-tuning), expect some "bad descriptions."

Examples

Ollama logs

gemma3:270m

China and India are the most populated countries, Bar chart. Automatic description: China and India are the most populated countries in 2020, with a total of 1.4 billion people.

qwen2.5:0.5b

China and India are the most populated countries, Bar chart. Automatic description: The data shows that China and India are the most populous countries in terms of population, with China having a population of approximately 1425.89 million in 2020 and increasing to about 1417.17 million by 2023. The global trend is also showing an average annual growth rate of around 0.5% for both countries since 2020, indicating that their populations are growing at a steady pace.

Considerations

  • Following the sort-of convention of specifying a dummy API key even when using local models (a result of using SDKs like OpenAI's, which have a required API key parameter, for example), I kept the API key as a required prop.
  • For now, if a reasoning model is used and the output includes its thinking output (between <think> tags, as in Qwen 3, for example), the entire output is added to the chart description. Future work is needed to properly accommodate this type of models.

References

TODO

  • Add model ID and base URL props to example components
  • Update error messages
  • Add base URL param
  • Test automatic descriptions with Ollama

@joaopalmeiro joaopalmeiro self-assigned this Sep 5, 2025
@DiogoRDuarte DiogoRDuarte changed the base branch from main to develop September 8, 2025 10:55
@joaopalmeiro joaopalmeiro marked this pull request as ready for review September 8, 2025 12:40
Comment thread examples/src/KeyRequest.jsx Outdated
@DiogoRDuarte DiogoRDuarte merged commit 17def94 into feedzai:develop Sep 8, 2025
2 checks passed
@joaopalmeiro joaopalmeiro deleted the feat-jp-57-add-support-open-ai-compatible-endpoints branch September 9, 2025 05:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: Allow a AI provider URL to be set, supporting more providers other than OpenAI.

2 participants