[R-227] Add an MLFlow Integration

[jjmachan]

[work in progress] - still flushing out the specifics

• @jjmachan will first build a POC

Todos

• POC
• ragas.integration and documentation
• joined blog post?

Resources

• https://mlflow.org/docs/latest/llms/rag/notebooks/mlflow-e2e-evaluation.html

_{From SyncLinear.com | R-227}

[Tiemen-Vanderstraeten]

Hi @jjmachan, have you explored this topic further?

[jjmachan]

not a lot @Tiemen-Vanderstraeten, is this something you would want? would you share a bit more about your usecase if so?

[Tiemen-Vanderstraeten]

@jjmachan I'm working on a RAG-chatbot and want to use ragas metrics for regression testing. I am currently using LangSmith as a test framework, but I want to migrate to MLFlow since they have now added tracing. It should be possible to use the metrics in MLFlow's evaluate method

I would possibly like to help enable ragas' metrics for MLFlow, but not super much software development experience myself

[B-Step62]

@Tiemen-Vanderstraeten @jjmachan MLflow just added native integration with RAGAS! https://mlflow.org/docs/latest/genai/eval-monitor/scorers/third-party/ragas/

import mlflow
from mlflow.genai.scorers.ragas import Faithfulness, ContextPrecision

traces = mlflow.search_traces()
results = mlflow.genai.evaluate(
    data=traces,
    scorers=[
        Faithfulness(model="openai:/gpt-4"),
        ContextPrecision(model="openai:/gpt-4"),
    ],
)