[BUG: Unable to fine-tune for a multi-modal inputs

[ragesh-beo]

Python Version

3.10

Pip Freeze

.

Reproduction Steps

I have tried it using the api platform to upload a dataset with multimodal inputs (contents contains a list of dicts)
[{"content": [{"image_url": "https://image_path.png", "type": "image_url"}, {"text": "Describe the image"}], "role": "user"}, {"content": [{"text": "description of the image"}], "role": "assistant"}]}

Whenever I try to upload such a dataset to the api platform, Im getting validation error. It is expecting str for content.
So How can I finetune for multimodal tasks ?

Expected Behavior

.

Additional Context

No response

Suggested Solutions

No response