Merge remote-tracking branch 'upstream/master' into eyetacking_eeg

Author: scott-huberty

.github/workflows/validate_datasets.yml

@@ -148,7 +148,7 @@ jobs:
     - name: Skip main validation for datasets with unreleased spec features
       # Replace ${EMPTY} with dataset patterns, when this is needed
       # Reset to "for DS in ${EMPTY}; ..." after a spec release
-      run: for DS in eyetracking_* atlas-* emg_*; do touch $DS/.SKIP_VALIDATION; done
+      run: for DS in ${EMPTY}; do touch $DS/.SKIP_VALIDATION; done
       if: matrix.bids-validator != 'dev'
 
     - name: Set BIDS_SCHEMA variable for dev version

.github/workflows/validation.yml

@@ -17,4 +17,19 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v6
-      - uses: codespell-project/actions-codespell@master
+      - uses: codespell-project/actions-codespell@master
+
+  build:
+      name: validata dataset listing
+      runs-on: ubuntu-latest
+      steps:
+      -   name: Checkout
+          uses: actions/checkout@v6
+
+      -   name: Set up Python
+          uses: astral-sh/setup-uv@v7
+          with:
+              python-version: 3.12
+
+      -   name: check listing datasets
+          run: uv run tools/print_dataset_listing.py

CONTRIBUTING.md

@@ -69,7 +69,13 @@ We release `bids-examples` in sync with `bids-specification`.
     configured as a git remote called "upstream")
 1. Tag the `master` branch: `git tag -a -m "X.X.X" X.X.X upstream/master`
    (replace `X.X.X` with the version to be released)
-1. Push the tag upstream: `git push upstream X.X.X`
+1. Make a schema tag: `git tag -a -m "BIDS Schema Y.Y.Y" schema-Y.Y.Y upstream/master`
+   (replace `Y.Y.Y` with the version to be released)
+1. Push the tags upstream: `git push upstream --tags`
+1. Create a maintenance branch to track changes induced by updates to the
+   validator or schema: `git push upstream upstream/master:refs/heads/maint/X.X.X`
+   This branch allows for future schema releases on the corresponding specification
+   branch, without accumulating datasets with new features.
 1. Create a GitHub release using the new tag. Fill the title of the release
    with the name of the tag. Fill the description of the release with a sentence like
    > "Microscopy" BEP was merged into BIDS-specification (2022-02-15).

README.md

@@ -83,16 +83,27 @@ mrs_biggaba	MEGA-PRESS and PRESS MRS data from 12 subjects from one site from th
 mrs_fmrs	Functional MRS data involving a pain stimulus task from 15 subjects	[link](https://www.nitrc.org/projects/fmrs_2020)	[@markmikkelsen](https://github.com/markmikkelsen)	anat, mrs	T1w, events, mrsref, svs
 2d_mb_pcasl	Siemens 2D MultiBand Multi-delay PCASL (m0 and noRF included within timeseries)	[link](https://osf.io/xrkc4/)	[@aptinis](https://github.com/aptinis)	anat, fmap, perf	T1w, asl, aslcontext, epi
 xeeg_hed_score	EEG and iEEG data with annotations of artifacts, seizures and modulators using HED-SCORE 		[@dorahermes](https://github.com/dorahermes)	anat, eeg, ieeg	T1w, channels, coordsystem, eeg, electrodes, events, ieeg
-dwi_deriv	exemplifies the storage of diffusion MRI derivates that may be generated on the Siemens XA platform.			dwi	dwi
+dwi_deriv	exemplifies the storage of diffusion MRI derivates that may be generated on the Siemens XA platform.			dwi	ADC, FA, S0map, colFA, dwi, expADC, trace
 pheno004	Minimal dataset with subjects with imaging and/or phenotype data		[@ericearl](https://github.com/ericearl)	phenotype, anat	T1w
 emg_ConcurrentIndependentUnits	Concurrent EMG recording with multiple independent recording units at different sampling rates	n/a	[@neuromechanist](https://github.com/neuromechanist)	emg	channels, electrodes, coordsystem, emg, events
 emg_CustomBipolar	Custom-made bipolar EMG recording setup with electrodes on flexors of the lower arm	n/a	[@neuromechanist](https://github.com/neuromechanist)	emg	channels, emg
 emg_CustomBipolarFace	EMG recording from facial muscles with many-to-many mapping between sensors and muscles	n/a	[@neuromechanist](https://github.com/neuromechanist)	emg	channels, electrodes, coordsystem, emg
 emg_IndependentMod	Commercial bipolar EMG modules recording multiple muscles with wireless sensors	n/a	[@neuromechanist](https://github.com/neuromechanist)	emg	channels, electrodes, coordsystem, emg
 emg_Multimodal	Integration of EEG, EMG, and motion capture data	n/a	[@neuromechanist](https://github.com/neuromechanist)	eeg, emg, motion	channels, electrodes, coordsystem, eeg, emg, motion, scans, events
-emg_MultiBodyParts	EMG recording from multiple body parts with different electrode types	n/a	[@neuromechanist](https://github.com/neuromechanist)	emg	channels, electrodes, coordsystem, emg
+emg_MultiBodyParts	EMG recording from multiple body parts with different electrode types	n/a	[@neuromechanist](https://github.com/neuromechanist)	emg	channels, electrodes, coordsystem, emg, physio
 emg_TwoHDsEMG	High-density EMG grid recordings from two body parts demonstrating grid placement documentation	n/a	[@neuromechanist](https://github.com/neuromechanist)	emg	channels, electrodes, coordsystem, emg
 emg_TwoWristbands	EMG recordings using two wristbands with dry electrodes to capture forearm muscle activity	n/a	[@neuromechanist](https://github.com/neuromechanist)	emg	channels, electrodes, coordsystem, emg
 mri_chunk	Example MRI dataset to illustrate BIDS chunk entity. A single subject, two chunks.		[@valosekj](https://github.com/valosekj)	anat	T1w
+atlas-Destrieux				n/a	n/a
+atlas-Schaefer				n/a	n/a
+eyetracking_binocular				beh	events, physio
 eyetracking_eeg	Example dataset of simultaneously collected EEG and eyetreacking.		[@scott-huberty](https://github.com/scott-huberty)	eeg, eyetracking	eeg, eyetracking, channels, electrodes
-
+atlas-HarvardOxford				n/a	n/a
+atlas-Talairach				n/a	n/a
+atlas-4S				n/a	n/a
+atlas-HOSPA				n/a	n/a
+atlas-AAL				n/a	n/a
+atlas-DiFuMo				n/a	n/a
+eyetracking_fmri				anat, fmap, func	T1w, T2w, bold, epi, events, fieldmap, magnitude, physio
+atlas-suit				n/a	T1w
+atlas-Juelich				n/a	n/a

dataset_listing.tsv

@@ -0,0 +1,13 @@
+{
+    "Columns": ["resp", "cardio"],
+    "StartTime": 0,
+    "SamplingFrequency": 50,
+    "resp": {
+        "Description": "Respiratory signal",
+        "Unit": "mV"
+    },
+    "cardio": {
+        "Description": "Cardiac signal",
+        "Unit": "mV"
+    }
+}

emg_MultiBodyParts/sub-01/emg/sub-01_task-mechPerturbations_physio.json

@@ -15,7 +15,7 @@
 You can pass an argument to insert the content in another file.
 Otherwise the content will be added to the README of this repository.
 """
-
+import warnings
 import sys
 from pathlib import Path
 import pandas as pd
@@ -43,6 +43,7 @@
 
 tables_order = {
     "ASL": "perf",
+    "Atlas": "",
     "Behavioral": "beh",
     "EEG": "^eeg$",
     "EMG": "emg",
@@ -56,9 +57,10 @@
     "MRS": "mrs",
     "NIRS": "nirs",
     "PET": "pet",
-    "qMRI": "",
     "Phenotype": "phenotype",
+    "Physio": "",
     "Provenance": "",
+    "qMRI": "",
 }
 
 DELIMITER = "<!-- ADD EXAMPLE LISTING HERE -->"
@@ -76,13 +78,13 @@ def main(output_file=None):
 
     names = df["name"].copy()
 
-    check_missing_folders(df, root)
-
     if update_content:
         df = update_datatypes_and_suffixes(df, root)
         df.to_csv(input_file, sep="\t", index=False)
         df = pd.read_csv(input_file, sep="\t")
 
+    check_missing_folders(df, root)
+
     df = add_links(df)
 
     clean_previous_run(output_file)
@@ -135,11 +137,12 @@ def add_links(df):
             if not isinstance(row[1][col], str):
                 continue
             if col == "name":
-                row[1][col] = f"[{row[1][col]}]({UPSTREAM_REPO}{row[1][col]})"
+                tmp = row[1][col]
+                df.loc[row[0], col]= f"[{tmp}]({UPSTREAM_REPO}{tmp})"
             if col == "link to full data" and row[1][col].startswith("http"):
-                row[1][col] = f"[link]({row[1][col]})"
+                df.loc[row[0], col] = f"[link]({row[1][col]})"
             if col == "maintained by" and row[1][col].startswith("@"):
-                row[1][col] = f"[{row[1][col]}](https://github.com/{row[1][col][1:]})"
+                df.loc[row[0], col] = f"[{row[1][col]}](https://github.com/{row[1][col][1:]})"
     return df
 
 
@@ -170,16 +173,17 @@ def add_tables(df: pd.DataFrame, output_file: Path, names) -> None:
     print("Writing markdown tables...")
     df.fillna("n/a", inplace=True)
     for table_name, table_datatypes in tables_order.items():
-        with output_file.open("a") as f:
-            f.write(f"\n### {table_name}\n\n")
-            add_warning(f)
 
         if table_name == "qMRI":
             mask = names.str.contains("qmri_")
         elif table_name == "HED":
             mask = names.str.contains("_hed_")
+        elif table_name == "Physio":
+            mask = df["suffixes"].str.contains("physio", regex=True)
         elif table_name == "Provenance":
             mask = names.str.contains("provenance_")
+        elif table_name == "Atlas":
+            mask = names.str.contains("atlas-")
         else:
             mask = df["datatypes"].str.contains(table_datatypes, regex=True)
 
@@ -188,9 +192,15 @@ def add_tables(df: pd.DataFrame, output_file: Path, names) -> None:
 
         print(sub_df)
 
-        sub_df.to_markdown(output_file, index=False, mode="a")
-        with output_file.open("a") as f:
-            f.write("\n")
+        if len(sub_df) > 0:
+            with output_file.open("a") as f:
+                f.write(f"\n### {table_name}\n\n")
+                add_warning(f)
+            with output_file.open("a") as f:
+                sub_df.to_markdown(output_file, index=False, mode="a")
+                f.write("\n")
+        else:
+            warnings.warn(f"No dataset for '{table_name}'", stacklevel=2)
 
 
 def stringify_list(l):