Skip to content

litellm based custom llm provider#21

Open
staru09 wants to merge 4 commits intoRepello-AI:mainfrom
staru09:custom_llm
Open

litellm based custom llm provider#21
staru09 wants to merge 4 commits intoRepello-AI:mainfrom
staru09:custom_llm

Conversation

@staru09
Copy link
Copy Markdown

@staru09 staru09 commented Oct 7, 2025

Related to #16.

I had tested it with openai api using this script

import os
from core.llm_provider import LiteLLMProvider
from dotenv import load_dotenv
load_dotenv()

provider = LiteLLMProvider()
out = provider.chat_completion(
    messages=[{"role":"user","content":"Say 'litellm-ok'"}],
    model="gpt-4o-mini",
    max_tokens=16
)
print(out)

and got this output

Litellm-ok!

please test it with other providers as well.

@Aaditya-G Aaditya-G self-requested a review October 17, 2025 06:24
Copy link
Copy Markdown
Collaborator

@Aaditya-G Aaditya-G left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@staru09 kindly checkout requirements.txt, please rebase with main if possible, everything else lgtm.

@staru09
Copy link
Copy Markdown
Author

staru09 commented Oct 30, 2025

Done @Aaditya-G

Copy link
Copy Markdown
Collaborator

@Aaditya-G Aaditya-G left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants