Library of bpipe functions for processing minc files

====================================================

minc-bpipe-library provides a set of chainable minc file processing functions to (pre)process data. At the moment is our star preprocessing pipeline. By default it will perform: Clean and Center, N4correction Simple, Cutneck, Head and Brain masks (using BEAST) and linear registration to MNI. Check the minc-library.bpipe file for details on each step.

To run in any computer it requires http://www.bic.mni.mcgill.ca/ServicesSoftware/ServicesSoftwareMincToolKit, https://github.com/ssadedin/bpipe/ and gnu-parallel.

To run it on Scinet see below.

To control which stages are run, edit pipeline.bpipe and add stage names using "+" to the "run" stage.

The default in pipeline.bpipe is what has experimentally determined to be a best-practice run.

Stages are listed in the minc-library.bpipe, correction stages such as denoising, n3 and n4, and normalize should be run before any other stages. Typically after this linear_antsRegistration would be run, which will allow other processing such as VBM, cutneck, deface and beast to be done in MNI space.

For convenience, "segments" have been defined for some processing jobs (beast, cutneck, VBM) which are multi-step. These may conflict with each other if you try to combine them, so instead you must specifiy their individual stages.

Stage options have been chosen based on best pratices from publications where applicable but can be changed.

Once you have defined your stages, you can run your pipeline on the SGE cluster with:

> git clone https://github.com/CobraLab/minc-bpipe-library.git
#Edit minc-bpipe-library/pipeline.bpipe as you see fit
> module load bpipe #needed to run bpipe
> module load minc-toolkit/1.9.11 #needed for most stages
> cd /path/to/store/outputs
#Choose n to be the smaller of (number of input files, 240)
> bpipe run -n<number> /path/to/pipeline.bpipe /path/to/inputs/*mnc

Output filenames from bpipe will be of the form <inputname>.stage1.stage2.stage3.ext where the last stage name will be the last run. commandlog.txt will be generated by bpipe describing all the commands run for the pipeline. This is a very good file to keep around as as a note of what happened to your data.

Some stages produce both mnc files and xfm files, see the $output variables for what is generated.

#Scinet Operation Inputs are split into 8 file chunks and submitted as local bpipe jobs on scinet nodes

Steps

1. git clone https://github.com/CobraLab/minc-bpipe-library.git
2. rm minc-bpipe-library/bpipe.config
3. sed -i 's#/opt/quarantine#/project/m/mchakrav/quarantine#g' minc-bpipe-library/minc-library.bpipe
4. Include the path in the .bashrc PATH=$PATH:/home/m/mchakrav/user/bin/minc-bpipe-library
5. cd /path/to/outputs
6. module load scinet
7. Use /path/to/minc-bpipe-library/bpipe-batch.sh /path/to/pipeline.bpipe /path/to/my/inputs/*.mnc > joblist to generate a joblist
8. Use /path/to/minc-bpipe-library/qbatch joblist 1 12:00:00 to submit jobs to scinet queing system

Example:

1. Create a folder for preprocessing (i.e. mkdir preproc) and cd into it cd preproc.

2. Point the bpipe-batch.sh to where pipeline.bpipe is installed and then to where the T1s you are using are.

bpipe-batch.sh /home/m/mchakrav/egarza/bin/minc-bpipe-library/pipeline.bpipe /home/m/mchakrav/egarza/scratch/adhd_gen/preproc/files/*.mnc > joblist

3. Check the joblist file using nano

4. Submit the jobs:

qbatch joblist 1 12:00:00

5. QC the resulting files. For MAGeT or CIVET you want to use the n4corrected.cutneckapplyautocrop.mnc in native space. You can also use the .beastmask.mnc files for CIVET.

QC Generation

The script generate-bpipe-QC.sh is used to generate standardized views of the final outputs for quality control, to use:

#In your output directory
> mkdir QC
> for file in *.cutneckapplyautocrop.beastnormalize.mnc; do /path/to/minc-bpipe-library/generate-bpipe-QC.sh $file QC/$(basename $file .mnc).jpg; done

• Important: To do the QC in scinet you need to load the module scinet-dev instead of scinet.

QC example:

for file in *.cutneckapplyautocrop.beastnormalize.mnc; do ~/bin/minc-bpipe-library/generate-bpipe-QC.sh $file QC/$(basename $file .mnc).jpg;done