Skip to content

Latest commit

 

History

History
95 lines (48 loc) · 4.29 KB

README.md

File metadata and controls

95 lines (48 loc) · 4.29 KB

dream-viewer

Attempted using electroencephalogram (EEG) to decode visual dream content by leveraging an encoding model which generates EEG signals from deep neural network (DNN) feature maps of visual images. The encoding model is trained on THINGS2 EEG dataset and tested on both THINGS1 waking EEG dataset and Zhang_Wamsley dream EEG dataset. The project pipeline integrates the code from THINGS2 and derives the remaining analyses of this project from it.

Datasets

The open datasets currently used are：

The path of download:

  1. THINGS EEG1: ../project_directory/eeg_dataset/wake_data/THINGS_EEG1/

  2. THINGS EEG2:

    • Raw EEG data: ../project_directory/eeg_dataset/wake_data/THINGS_EEG2/raw_data/

    • Image set: ../project_directory/eeg_dataset/wake_data/THINGS_EEG2/image_set/

  3. THINGS images meta data:

  4. Zhang & Wamsley 2019: ../project_directory/eeg_dataset/dream_data/Zhang_Wamsley

The analysis

The files ended with '_func.py' have no needs to be run.

01_eeg_preprocessing

  • run_all_scripts.py (This preprocesses all datasets, including THINGS1, THINGS2, Zhang & Wamsley.)

  • THINGS1.py (This file preprocesses one specific subject of THINGS1.)

  • THINGS2.py (This file preprocesses one specific subject of THINGS2.)

  • ZW.py (This file preprocesses one specific subject of Zhang & Wamsley.)

02_dnn_fmaps

Step 1: run either of followings:

  • alexnet_THINGS.py (This file extracts dnn alexnet feature maps from THINGS images.)

  • alexnet_ZW.py (This file extracts dnn alexnet feature maps from Zhang & Wamsley images.)

  • sort_ZW.py + sort_ZW_REMs.py + alexnet_ZW_REMs.py --all_or_best (This file extracts dnn alexnet feature maps from Zhang & Wamsley REM images.)

Step 2: run the following:

  • pca.py --dataset [THINGS_EEG2, Zhang_Wamsley, ZW_REMs_all, ZW_REMs_best]

    (This file applies StandardScaler and PCA on the full feature maps.)

03_validation_test

  • encoding_THINGS1.py (This file trains the EEG encoding model on THINGS EEG2 dataset and tests the EEG encoding model on THINGS1 dataset. The correlation result is plotted.)

  • ZW.py (This file trains the EEG encoding model and tests the EEG encoding model on Zhang & Wamsley dream dataset. The correlation result is plotted.)

  • encoding_ZW.py (This file trains the EEG encoding model on THINGS EEG2 dataset and tests the EEG encoding model on Zhang & Wamsley dataset using the 'spatial correlation method'. The correlation scores of each dream is saved.)

  • encoding_ZW_REM.py --all_or_best (This file trains the EEG encoding model on THINGS EEG2 dataset and tests the EEG encoding model on REM dreams Zhang & Wamsley dataset using the 'spatial correlation method'. The correlation scores of each dream is saved.)

  • encoding_ZW_REM_fs.py --all_or_best (This file trains the EEG encoding model on THINGS EEG2 dataset and tests the EEG encoding model on REM dreams Zhang & Wamsley dataset using the 'spatial correlation method' with extra feature selections.)

04_plots

  • pca_cluster.py --all_or_best (This file plots the clusters of dreams.)

  • plot.py (This file plots the correlation scores for individual dream.)

  • plot_hist.py (This file plots the histogram of correlation scores for all dream.)

  • plot_meanall.py (This file plots the statistical decoding results from all dreams.)

  • plot_RDMs.py (This file plots the RDMs of dreams with feature maps.)

  • plot_RDMs_REM.py --all_or_best (This file plots the RDMs of REM dreams with feature maps.)

05_ZW

No needs to run the following scripts.

  • stats.py (This file produces the statistical decoding results from all dreams with different sleep stages.)

  • run_all_REMs --feature_selection --all_or_best (This file runs the alexnet, pca, encoding, RDMs and clusters of REM dreams.)