Skip to content

README.md

Latest commit

 

History

History
84 lines (52 loc) · 2.13 KB

README.md

File metadata and controls

84 lines (52 loc) · 2.13 KB

[ICLR 2026] A Cognitive Process-Inspired Architecture for Subject-Agnostic Brain Visual Decoding

📌 Overview

model

Architecture of the VCFlow framework

📣 Latest Updates

🟡 2026/01    VCFlow is accepted at ICLR 2026

🛠️ Installation & Setup

🖥️ Environment Setup

conda create -n train python==3.10
conda activate train
pip install -r requirements.txt

📊 Data Preparation

Download the preprocessed pretraining and main datasets.

python download_dataset.py

⚙️ Pretrained Weights Preparation

mkdir pretrained_weights
cd pretrained_weights
wget -O unclip6_epoch0_step110000.ckpt -c https://huggingface.co/datasets/pscotti/mindeyev2/resolve/main/unclip6_epoch0_step110000.ckpt\?download\=true
wget -O last.pth -c https://huggingface.co/datasets/pscotti/mindeyev2/resolve/main/train_logs/final_subj01_pretrained_40sess_24bs/last.pth\?download\=true
wget -O convnext_xlarge_alpha0.75_fullckpt.ckpt -c https://huggingface.co/datasets/pscotti/mindeyev2/resolve/main/convnext_xlarge_alpha0.75_fullckpt.pth\?download\=true
wget -O sd_image_var_autoenc.pth https://huggingface.co/datasets/pscotti/mindeyev2/resolve/main/sd_image_var_autoenc.pth\?download\=true
cd ..

🚀 Quick Start

This codebase allows train, test, and evaluate using one single bash file.

bash train_vcflow.sh 0 vcflow 0123456 enhance 1

Parameters:

$1: use which gpu to train

$2: train file postfix, e.g, train_vcflow

$3: run which stage:

  • 0: pretrain on fMRI2Image dateset
  • 1: train backbone and SARA
  • 2: train HCAM and HED
  • 3: recon keyframe and blurry videos
  • 4: caption keyframe
  • 5: recon videos
  • 6: eval results

$4: inference mode: ['normal', 'enhance']

$5: test which subject: [1,2,3]

Note that for convenience of debugging, use_wandb is set to False be default.

If you would like to use wandb, first run wandb login and set the use_wandb to True in train_vcflow.py.