fmri2vec is an fMRI encoder leveraging Transformer and Vision Transformer (ViT) architectures to process and encode the fMRI data directly in 4D. G3DViT, a 3D Grad-CAM, has been integrated to provide insights into the model's decision-making process and enhance interpretability.
This project is a subpart of a bigger research and development project focused on creating a multimodal model for medical image computing. The primary goal of that project is to integrate fMRIs, behavioral data, and blood-based markers into a joint embedding space. This common representation will then be used to classify patients based on their pain sensitivity threshold.
This R&D project aims to advance the field of medical image computing by offering a novel approach to patient classification based on pain sensitivity, utilizing the power of multimodal data integration and state-of-the-art deep learning architectures.