Group reports

[oesteban]

As @poldrack just suggested, it would be great to have a group report which:

• Builds a "paper-ready" boilerplate clearly stating things like the total number of subjects covered by the group-report and the variations in their fMRIPrep workflow. Two of these variations were mentioned: slice-timing (there could be participants with and without slice timing information) and the automated skull-stripping skip (see ENH: Add a --skull-strip-t1w argument to skip brain extraction #2039). Other variations I can think of:
  • fMRIPrep versions
  • T1w-to-bold fallbacks (bbregister/mri_coreg, flirt-bbr/flirt-vanilla).
• Shows a summary of errors (and maybe warnings?, e.g., x-form rewrites?).
• Visual summaries: possibly building off of Group reports niworkflows#450.

[oesteban]

I know @Shotgunosine will be interested in this - please feel free to add/prioritize requirements.

[Shotgunosine]

I've got some very quick and dirty code in a notebook that generates group reports and makes a directory full of symlinks to images to go with it, will try to get marginally cleaned up and posted somewhere. Having the symlinked directory is nice because you can download it with an rsync -L and have a portable package of group reports.
The basic structure is a single report for each section of the individual level reports. I found that they work better if they're chunked up into groups of about 50 entries, any bigger than that and browser load times start to get annoying. I haven't implemented it for every section of the individual level report yet, but here's an example of what one of these directories looks like:

(base) MH02086115MACLT:local_reports nielsond$ tree
.
├── anat_reg_000.html
├── anat_reg_001.html
├── anat_surf_000.html
├── anat_surf_001.html
├── func_rois_000.html
├── func_rois_001.html
└── imgs
    ├── sub-20900_desc-reconall_T1w.svg
    ├── sub-20900_ses-01_task-mid_run-1_desc-rois_bold.svg
    ├── sub-20900_ses-04_task-mid_run-1_desc-rois_bold.svg
    ├── sub-20900_space-MNI152NLin2009cAsym_T1w.svg
    ├── sub-21111_desc-reconall_T1w.svg
    ├── sub-21111_ses-01_task-mid_run-1_desc-rois_bold.svg
     ....

And here's an example of what the anat_reg report looks like:

The box at the top is a place for the reviewer to put their initials. The numbers at the top indicate how many good, bad, and unreviewed items are in the report. I've used some javascript that may have originally been in an MRIQC report somewhere to have radio button for good and bad and the ability to download a table of the qc decisions as a CSV. I also made it so that scrolling an image off the top of of the screen marks it as good, which minimizes the interaction needed if everything is ok. We've found that it's pretty quick to do an initial QC screening of fmriprep outputs this way.

@oesteban do you think you'd like us to work on this at the OHBM hackathon?

[mgxd]

@Shotgunosine that looks awesome!! Would love to see the code whenever you get around to it

[Shotgunosine]

@mgxd Sorry for the delay and the messiness, but here's my notebook:
https://github.com/Shotgunosine/make_fmriprep_group_reports/blob/master/update_fmripep_qc_sheet_cleanish.ipynb

[oesteban]

@Shotgunosine sorry for joining late - I'm moving to Europe and my last month has been pretty crazy. I just read the invitation for the hackathon :(

[Shotgunosine]

@oesteban No worries, I've ended up getting slammed with work issues so my hackathon participation was pretty limited this year. I hope the move is going ok, it's quite a time to be making a transition like that. I hope everything goes ok.

[oesteban]

After our call with @dorianps today, I've been thinking about this a little. It seems there remains a big issue here with the many reports you end up collating together, at some point navigation is impossible because there are just too many.

After using remark.js a lot last week, I would propose the following solution inspired by them.

1. fMRIPrep would add a "group" analysis level, just like MRIQC, to generate a group.html file (or similar)
2. When invoked with "group" (and taking advantage of the --bids-filters), the figures/ folder can be queried a la PyBIDS. A CSV (or JSON) table is generated with two columns - index and reportlet path
3. Instead of embedding all the reports in only one, long HTML (this is where we copy the idea from remark.js), an anchor segment of the URL* (i.e., localhost:8000/group.html#11) is used to select a row from the table in step 2 (e.g., the #11 points to sub-YXZ11/figures/sub-YXZ11_desc-preproc_T1w.svg). Navigation can be ported from remark.js (especially fwd/back with the arrows), so that changing to #12 will now point to sub-YXZ12/figures/sub-YXZ12_desc-preproc_T1w.svg. Because the HTML is updated with each change, only one of the heavy svg files is loaded at a time.

WDYT?

* this might require opening the reports via a simple HTTP server (python -m http.server)

[oesteban]

Other shortcuts that could make the report more usable:

• Rating (for instance using the digits from 1 to 9)
• Up/down to change the group report type (i.e., start with the anatomical panel, and if you hit down, it then changes to the spatial normalization panel)
• c - add a comment