Run across (selection of) bids-examples for smoke + basic testing

[yarikoptic]

would be great to run across all https://github.com/bids-standard/bids-examples (e.g. which have neuroimaging for starters) and seeing script not failing and passing target schema (image03?) validation.

attn @WillForan who might be interested

[WillForan]

Looks like a really good idea!

There's an initial pytest setup in #46 (read_participant_info for per session age)
Is it reasonable to merge that here first?

[Otherwise I can separate the pytest and run refactor into their own PR. But not looking forward to potential merge conflicts when updating the other PRs/issues]

[WillForan]

I have a work in progress test that sends off to NDA for validation. Using the bids-example/synethic/ dataset to start off.

I might be using vtcmd wrong. I'm prompted for credentials, but validation-only shouldn't need them.

Some interesting errors (date too early) and lots of missingness issues (TR, TE, slice_timing, field strength, software versions, etc)

WillForan@7abdd79#diff-572a40f4ef30d2045c55beb2a6489f0b4bba6a7b1acb32800ebd345836c43fb4R126-R171

@andrewpapale are those errors similar to the ones you debugged for the recent upload?

[andrewpapale]

Yes I believe you now need to have date of scan as month since birth month. I have all the error logs I encountered saved if they are helpful.

[yarikoptic]

re age

1 -- "Units": note that although it was permitted since awhile (if not forever) to overload Units for any column, including 'age', it was not really done consistently etc. But now we even have controlled vocab to test against

• [ENH] Recommend controlled vocabulary for age Units, clarify that it can be overloaded bids-specification#2400

thus when loading "age" from a .tsv -- also get the "Units" for it from .json sidecar!

2 -- sessions. In multi session studies, if there is no single "age" for all sessions, there could be "age" defined in sub-X/sub-X_session.tsv files . https://bids-specification.readthedocs.io/en/stable/modality-agnostic-files/data-summary-files.html#sessions-file doesn't even mention that yet, but I think it is the case and thus ideally should be supported

[WillForan]

1. re. interview_age Units: current code assumes years and calculates months: interview_age = int(round(list(this_subj.age)[0]*12, 0)). Getting Units from session/participant.json is another new issue? and fun can of worms 😿
2. More than just participant.tsv vs session.tsv, I think scan.tsv could also have a different age (we have a single "session" that have acquisitions across two coils and might have someone come back days or a month later to run protocols on the second coil). feat: read sessions.tsv files + pytest #46 only resolves acq_time across session and scans. But age should probably be in there too? ... yikes

Specific to this issue: should we just add the known missing data to image03 (or the underlying bids data) before testing the validation? Or check that it fails in expected ways?

[yarikoptic]

2. I think scan.tsv could also have a different age (we have a single "session" that have acquisitions across two coils and might have someone come back days or a month later to run protocols on the second coil).

yikes... never thought about such use cases -- to me by all definitions of a "session" sounds like it should have just been different sessions ;-)

Specific to this issue: should we just add the known missing data to image03 (or the underlying bids data) before testing the validation? Or check that it fails in expected ways?

sorry, my friday melted brain might be missing, but ideally we should have sufficient data in bids. If something is missing in BIDS specification - let's propose/add; if something is missing in the dataset -- add to BIDS (.tsvs or sidecar .jsons).

[WillForan]

We didn't anticipate/appreciate how difficult the coil change could be (or how appealing participants would find breaking up the long scan). Our study file organization has since changed to session labels like ses-1ptx (anat, func, dwi) and ses-1stx (31P MRSI), ending at ses-3ptx & ses-3stx in a few years! (Still undetermined: where do we put the longitudinally paired EEG data? it's own ses-1eeg?)
A BIDS FAQ answer makes the case for the original organization

The session level in the BIDS folder hierarchy can be used to group data that go "logically" together: this means that you can put in the same session folder data that were acquired on different days, but that are "linked" to one another in a way that make sense to how you want to organize your data.

---

For NDA validation with bids-examples/synthetic, the scans.tsv acq_time is nonsense and reasonably rejected by the remote validator.

The date 01/31/1868 does not match the NDA definition for Date fields which is MM/DD/YYYY. Leading zeroes are required for single digit day and month. Years must be between 1900-2200'

In that case, a passing test could confirm that failure. But testing NDA's validation system isn't too useful. Instead, we could modify image03 before upload to have a date after 1900 and the local bids2nda test would (ideally) confirm vtcmd returned no validation errors: image03 was generated correctly with all necessary info.

...but there are a lot parameters to modify (missing or junk in bids-examples/synethic)

experiment_id flip_angle image_orientation magnetic_field_strength mri_echo_time_pd mri_repetition_time_pd scanner_manufacturer_pd scanner_software_versions_pd scanner_type_pd slice_timing

Hard coding a bunch of bespoke corrections feels a bit deflating: "bids2nda works on example datasets ... once they are totally revamped"

On the postive side, working through these is also good opportunity to enumerate commonly excluded metadata that bids2nda can give a more actionable error about. Like "update session.tsv's acq_time (row N) to a valid date for file XXX.nii.gz" or "XXXX.json missing mandatory slice_timing field". Or maybe reasonable stand in values. "WARNING: missing "magnetic_field_strength" in XXX.json. Assume 3T"

I'll hopefully find some time to explore bid-example datasets to see if others have all or at least most of the NDA image03 mandated metadata.

A note to myself: quick look at what studies have acq_time and func with SliceTiming finds only ds000117!

rg acq_time -l bids-examples | sed s:/sub-.*:: | sort -u | xargs rg -l Slice|grep bold
bids-examples/ds000117/task-facerecognition_bold.json