Multi-Modal Testset Generation

[wabeg]

Describe the Feature
Add support for multi-modal testset generation in RAGAS. While RAGAS currently supports multi-modal evaluation through MultiModalRelevance and MultiModalFaithfulness metrics, it lacks the ability to generate multi-modal test cases. This feature would enable automatic generation of test cases that combine both text and image contexts, making it easier to evaluate multi-modal RAG systems comprehensively. Ideally it would also extract images from documents.

Why is the feature important for you?
Currently, while RAGAS can evaluate multi-modal RAG systems, users must manually create test cases that include images. This creates a disconnect between the evaluation and generation capabilities. Visual RAG appears to be the future.

[wabeg]

I am also looking to make my first contribution to the library! I could give this a go and create a PR.