Support for OpenAPI Model Endpoint's?

[TharunSivamani]

I have models deployed using vllm, huggingface-tgi, triton, ollama for serving.
Does all the evaluations supported by GENAI-EVAL Library offer openai compatible endpoints along with secured api-keys as an argument for evaluation, instead of passing a pretrained model path as arguments without downloading it again.

If supported can someone tell me which all evaluations are working and tested and which don't.

Thanks.

[joshuayao]

Hi @TharunSivamani, welcome to OPEA! GenAIEval does not currently support the use of user-provided API keys. Instead, OPEA manages API keys at the microservice level. To proceed, you will need to deploy your LLM services through OPEA. Once deployed, GenAIEval will be able to evaluate these services.

[yinghu5]

Hi @TharunSivamani @joshuayao , thank a lot for the request. Just remind if there are any updates.

[joshuayao]

Hi @TharunSivamani @joshuayao , thank a lot for the request. Just remind if there are any updates.

Hi @yinghu5 , the request is to use GenAIEval to evaluate OpenAI-compatible endpoints that are not provided by OPEA. While the request is valid, it currently falls outside the current scope of GenAIEval. As a result, I think this is not be a priority for now. I would suggest to lower the priority and add this request to backlog for tracking.