Skip to content

Commit ebb59ed

Browse files
committed
update README and BOLD5000 subjects
1 parent 7aece57 commit ebb59ed

2 files changed

Lines changed: 19 additions & 4 deletions

File tree

README.md

Lines changed: 17 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,10 @@
33
<img src=assets/first_fig.png />
44
</p>
55

6+
## News
7+
- Mar. 1, 2023. Our paper is accpeted at CVPR2023!
8+
- Nov. 10, 2022. Project release.
9+
610
## MinD-Vis
711
**MinD-Vis** is a framework for decoding human visual stimuli from brain recording.
812
This document introduces the precesedures required for replicating the results in *Seeing Beyond the Brain: Masked Modeling Conditioned Diffusion Model for Human Vision Decoding*
@@ -104,9 +108,9 @@ conda activate mind-vis
104108
```
105109

106110
## Download data and checkpoints
107-
Due to size limi and license issue, the full fMRI pre-training dataset (required to replicate **Stage A**) needs to be downloaded from the [Human Connectome Projects (HCP)](https://db.humanconnectome.org/data/projects/HCP_1200) offical website. The pre-processing scripts are also included in this repo.
111+
Due to size limit and license issue, the full fMRI pre-training dataset (required to replicate **Stage A**) needs to be downloaded from the [Human Connectome Projects (HCP)](https://db.humanconnectome.org/data/projects/HCP_1200) offical website. The pre-processing scripts are also included in this repo.
108112

109-
We also provide checkpoints and finetuning data at [FigShare](https://figshare.com/s/94cd778e6afafb00946e) to run the finetuing and decoding directly. Due to the size limit, we only release the checkpoints for Subject 3 and CSI4 in the GOD and BOLD5000 respectively. Checkpoints for other subjects are also available upon request. After downloading, extract the ```data/``` and ```pretrains/``` to the project directory.
113+
We also provide checkpoints and finetuning data at [FigShare](https://figshare.com/s/94cd778e6afafb00946e) to run the finetuing and decoding directly. Due to the size limit, we only release the checkpoints for Subject 3 and CSI1 in the GOD and BOLD5000 respectively. Checkpoints for other subjects are also available upon request. After downloading, extract the ```data/``` and ```pretrains/``` to the project directory.
110114

111115

112116
## SC-MBM Pre-training on fMRI (Stage A)
@@ -170,3 +174,14 @@ python code/gen_eval.py --dataset GOD
170174
## Acknowledgement
171175
We thank [Kamitani Lab](https://github.com/KamitaniLab), [
172176
Weizmann Vision Lab](https://github.com/WeizmannVision) and [BOLD5000 team](https://bold5000-dataset.github.io/website/) for making their raw and pre-processed data public. Our Masked Brain Modeling implementation is based on the [Masked Autoencoders](https://github.com/facebookresearch/mae) by Facebook Research. Our Conditional Latent Diffusion Model implementation is based on the [Latent Diffusion Model](https://github.com/CompVis/latent-diffusion) implementation from CompVis. We thank these authors for making their codes and checkpoints publicly available!
177+
178+
## Citation
179+
```
180+
@InProceedings{Chen_2023_CVPR,
181+
author = {Chen, Zijiao and Qing, Jiaxin and Xiang, Tiange and Yue, Wan Lin and Zhou, Juan Helen},
182+
title = {Seeing Beyond the Brain: Masked Modeling Conditioned Diffusion Model for Human Vision Decoding},
183+
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
184+
year = {2023}
185+
}
186+
187+
```

code/config.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ def __init__(self):
6262

6363
self.include_nonavg_test = True
6464
self.kam_subs = ['sbj_3']
65-
self.bold5000_subs = ['CSI4']
65+
self.bold5000_subs = ['CSI1']
6666

6767
# Training Parameters
6868
self.lr = 5.3e-5
@@ -95,7 +95,7 @@ def __init__(self):
9595

9696
self.dataset = 'GOD' # GOD or BOLD5000
9797
self.kam_subs = ['sbj_3']
98-
self.bold5000_subs = ['CSI4']
98+
self.bold5000_subs = ['CSI1']
9999
self.pretrain_mbm_path = os.path.join(self.root_path, f'pretrains/{self.dataset}/fmri_encoder.pth')
100100

101101
self.img_size = 256

0 commit comments

Comments
 (0)