Skip to content

🔧 Turn on XCP-QC by default in preconfigs #2204

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Apr 2, 2025
Merged

🔧 Turn on XCP-QC by default in preconfigs #2204

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
24e7f4b
🔧 Turn on xcp-qc file generation in all pre-configs
shnizzedy Mar 27, 2025
23ca231
:memo: Add "Turned `generate_xcpqc_files` on for all preconfiguration…
shnizzedy Mar 27, 2025
51d7eb8
:twisted_rightwards_arrows: Merge branch 'origin/develop' into 'xcpqc'
shnizzedy Mar 31, 2025
50d861d
:twisted_rightwards_arrows: Merge branch 'develop' into 'xcpqc'
shnizzedy Mar 31, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Renamed `desc-ref_bold` created in this workflow to `desc-unifized_bold`.
- `coregistration_prep_fmriprep` nodeblock now checks if `desc-unifized_bold` exists in the Resource Pool, if not it runs the `FSL-AFNI subworkflow` to create it.
- Input `desc-brain_bold` to `desc-preproc_bold` for `sbref` generation nodeblock `coregistration_prep_vol`.
- Turned `generate_xcpqc_files` on for all preconfigurations except `blank`.

### Fixed

Expand Down
4 changes: 4 additions & 0 deletions CPAC/resources/configs/pipeline_config_abcd-options.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@ pipeline_setup:
# Name for this pipeline configuration - useful for identification.
# This string will be sanitized and used in filepaths
pipeline_name: cpac_abcd-options
output_directory:
quality_control:
# Generate eXtensible Connectivity Pipeline-style quality control files
generate_xcpqc_files: On
system_config:

# The maximum amount of memory each participant's workflow can allocate.
Expand Down
4 changes: 4 additions & 0 deletions CPAC/resources/configs/pipeline_config_abcd-prep.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@ pipeline_setup:
# Name for this pipeline configuration - useful for identification.
# This string will be sanitized and used in filepaths
pipeline_name: cpac_abcd-prep
output_directory:
quality_control:
# Generate eXtensible Connectivity Pipeline-style quality control files
generate_xcpqc_files: On
system_config:

# The maximum amount of memory each participant's workflow can allocate.
Expand Down
3 changes: 3 additions & 0 deletions CPAC/resources/configs/pipeline_config_benchmark-ANTS.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ pipeline_setup:
# Generate quality control pages containing preprocessing and derivative outputs.
generate_quality_control_images: On

# Generate eXtensible Connectivity Pipeline-style quality control files
generate_xcpqc_files: On

# Include extra versions and intermediate steps of functional preprocessing in the output directory.
write_func_outputs: On

Expand Down
3 changes: 3 additions & 0 deletions CPAC/resources/configs/pipeline_config_benchmark-FNIRT.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ pipeline_setup:
# Generate quality control pages containing preprocessing and derivative outputs.
generate_quality_control_images: On

# Generate eXtensible Connectivity Pipeline-style quality control files
generate_xcpqc_files: On

# Include extra versions and intermediate steps of functional preprocessing in the output directory.
write_func_outputs: On

Expand Down
3 changes: 3 additions & 0 deletions CPAC/resources/configs/pipeline_config_ccs-options.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ pipeline_setup:
# Generate quality control pages containing preprocessing and derivative outputs.
generate_quality_control_images: On

# Generate eXtensible Connectivity Pipeline-style quality control files
generate_xcpqc_files: On

# Include extra versions and intermediate steps of functional preprocessing in the output directory.
write_func_outputs: On

Expand Down
3 changes: 3 additions & 0 deletions CPAC/resources/configs/pipeline_config_default-deprecated.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ pipeline_setup:
# Generate quality control pages containing preprocessing and derivative outputs.
generate_quality_control_images: On

# Generate eXtensible Connectivity Pipeline-style quality control files
generate_xcpqc_files: On

anatomical_preproc:
run: On
acpc_alignment:
Expand Down
4 changes: 2 additions & 2 deletions CPAC/resources/configs/pipeline_config_default.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,10 +54,10 @@ pipeline_setup:
# Quality control outputs
quality_control:
# Generate quality control pages containing preprocessing and derivative outputs.
generate_quality_control_images: True
generate_quality_control_images: On

# Generate eXtensible Connectivity Pipeline-style quality control files
generate_xcpqc_files: False
generate_xcpqc_files: On
Copy link
Contributor

@birajstha birajstha Apr 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can use both on and True right?
I searched the code repo for any other occurrences and only found this comment. Should we update it as well?

Copy link
Contributor

@nx10 nx10 Apr 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In YAML 1.1 (which CPAC uses) yes. 1.1 has a bunch of aliases for booleans:

y|Y|yes|Yes|YES|n|N|no|No|NO
|true|True|TRUE|false|False|FALSE
|on|On|ON|off|Off|OFF

https://yaml.org/type/bool.html

Starting with YAML 1.2 its only true/false though ( https://yaml.org/spec/1.2-old/spec.html#id2803629 )

Copy link
Contributor

@nx10 nx10 Apr 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Funnily enough this also leads to the "The Norway Problem": when you abbreviate Norway to its ISO 3166-1 ALPHA-2 form NO, YAML will return false when parsing list of countries f.e "[GB, IN, NO,…]"

Copy link
Member Author

@shnizzedy shnizzedy Apr 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I searched the code repo for any other occurrences and only found this comment. Should we update it as well?

That comment is a doctest for

C-PAC/CPAC/utils/utils.py

Line 2211 in 7523a5c

def update_nested_dict(d_base, d_update, fully_specified=False):
which is just dealing with Python native datastructures (i.e., dict, str, bool`, etc.; no YAML involved), so YAML parsing isn't relevant in that test.

In YAML 1.1 (which CPAC uses) yes. 1.1 has a bunch of aliases for booleans:

y|Y|yes|Yes|YES|n|N|no|No|NO
|true|True|TRUE|false|False|FALSE
|on|On|ON|off|Off|OFF

https://yaml.org/type/bool.html

Starting with YAML 1.2 its only true/false though ( https://yaml.org/spec/1.2-old/spec.html#id2803629 )

Yeah, I'm of two minds about sticking with YAML 1.1. On the one hand, "On" / "Off" makes conceptual sense for a pipeline config, and it doesn't seem to be causing any significant problems. On the other hand, if we had something that had more than two options (like if we had n/as), I could see needing to go beyond true/false, but since we're really just using on/off to mean true/false, is the toggle switch metaphor (we're turning options on and off, not making options true or false) worth the nonintuitively broad YAML < 1.2 boolean space? If we upgrade our YAML syntax, we don't need special handling for validating bools coming from the configs.


working_directory:

Expand Down
3 changes: 3 additions & 0 deletions CPAC/resources/configs/pipeline_config_fmriprep-ingress.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ pipeline_setup:
# Generate quality control pages containing preprocessing and derivative outputs.
generate_quality_control_images: On

# Generate eXtensible Connectivity Pipeline-style quality control files
generate_xcpqc_files: On

outdir_ingress:
run: On
Template: MNI152NLin2009cAsym
Expand Down
4 changes: 4 additions & 0 deletions CPAC/resources/configs/pipeline_config_fmriprep-options.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@ pipeline_setup:
# Name for this pipeline configuration - useful for identification.
# This string will be sanitized and used in filepaths
pipeline_name: cpac_fmriprep-options
output_directory:
quality_control:
# Generate eXtensible Connectivity Pipeline-style quality control files
generate_xcpqc_files: On
system_config:

# Select Off if you intend to run CPAC on a single machine.
Expand Down
3 changes: 3 additions & 0 deletions CPAC/resources/configs/pipeline_config_monkey-ABCD.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ pipeline_setup:
# Generate quality control pages containing preprocessing and derivative outputs.
generate_quality_control_images: On

# Generate eXtensible Connectivity Pipeline-style quality control files
generate_xcpqc_files: On

system_config:

# Select Off if you intend to run CPAC on a single machine.
Expand Down
3 changes: 3 additions & 0 deletions CPAC/resources/configs/pipeline_config_monkey.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ pipeline_setup:
# Generate quality control pages containing preprocessing and derivative outputs.
generate_quality_control_images: On

# Generate eXtensible Connectivity Pipeline-style quality control files
generate_xcpqc_files: On

system_config:

# Select Off if you intend to run CPAC on a single machine.
Expand Down
3 changes: 3 additions & 0 deletions CPAC/resources/configs/pipeline_config_ndmg.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ pipeline_setup:
# Generate quality control pages containing preprocessing and derivative outputs.
generate_quality_control_images: On

# Generate eXtensible Connectivity Pipeline-style quality control files
generate_xcpqc_files: On

system_config:

# The number of cores to allocate to ANTS-based anatomical registration per participant.
Expand Down
3 changes: 3 additions & 0 deletions CPAC/resources/configs/pipeline_config_regtest-1.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ pipeline_setup:
# Generate quality control pages containing preprocessing and derivative outputs.
generate_quality_control_images: On

# Generate eXtensible Connectivity Pipeline-style quality control files
generate_xcpqc_files: On

# Include extra versions and intermediate steps of functional preprocessing in the output directory.
write_func_outputs: On

Expand Down
3 changes: 3 additions & 0 deletions CPAC/resources/configs/pipeline_config_regtest-2.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ pipeline_setup:
# Generate quality control pages containing preprocessing and derivative outputs.
generate_quality_control_images: On

# Generate eXtensible Connectivity Pipeline-style quality control files
generate_xcpqc_files: On

# Include extra versions and intermediate steps of functional preprocessing in the output directory.
write_func_outputs: On

Expand Down
3 changes: 3 additions & 0 deletions CPAC/resources/configs/pipeline_config_regtest-3.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ pipeline_setup:
# Generate quality control pages containing preprocessing and derivative outputs.
generate_quality_control_images: On

# Generate eXtensible Connectivity Pipeline-style quality control files
generate_xcpqc_files: On

# Include extra versions and intermediate steps of functional preprocessing in the output directory.
write_func_outputs: On

Expand Down
3 changes: 3 additions & 0 deletions CPAC/resources/configs/pipeline_config_regtest-4.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ pipeline_setup:
# Generate quality control pages containing preprocessing and derivative outputs.
generate_quality_control_images: On

# Generate eXtensible Connectivity Pipeline-style quality control files
generate_xcpqc_files: On

# Include extra versions and intermediate steps of functional preprocessing in the output directory.
write_func_outputs: On

Expand Down
3 changes: 3 additions & 0 deletions CPAC/resources/configs/pipeline_config_rodent.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ pipeline_setup:
# Generate quality control pages containing preprocessing and derivative outputs.
generate_quality_control_images: On

# Generate eXtensible Connectivity Pipeline-style quality control files
generate_xcpqc_files: On

system_config:

# The maximum amount of memory each participant's workflow can allocate.
Expand Down