Improve spec/validation of participants.tsv

[sappelhoff]

TLDR

In the participants.tsv file, the age and sex columns are sometimes not well defined, and this leads to (unnecessary) issues on the side of tool developers (and thus eventually the users). We should improve either the spec or the validator, or both.

cc @jasmainak @agramfort @adam2392 @hoechenberger

came up in: mne-tools/mne-bids#396

Intro

The specification says the following about the participants.tsv file:

In case of single session studies this file has one compulsory column participant_id that consists of sub-, followed by a list of optional columns describing participants.

so strictly speaking, all columns that are not participant_id are OPTIONAL, and thus SHOULD be described in an accompanying participants.json.

For optional columns that are not described, the validator currently emits a warning such as this:

1: [WARN] Tabular file contains custom columns not described in a data dictionary (code: 82 - CUSTOM_COLUMN_WITHOUT_DESCRIPTION)
  ./participants.tsv
    Evidence: Columns: group not defined, please define in: /participants.json

Yet, the validator treats some "optional" columns differently, i.e., these columns are accepted WITHOUT warning. Examples of these are:

• age
• sex

However, the specification does not cover that these two variables are "expected optional columns". The expected behavior would be to raise a warning also for age and sex.

I could not pin down the exact part of the validator that is responsible for this behavior, but it may be this line:

https://github.com/bids-standard/bids-validator/blob/dfabbfb058daca406ed1d0897c3a25be059a5ad6/bids-validator/utils/summary/collectSubjectMetadata.js#L31

perhaps @nellh or @rwblair can help

The problem

The issue that arises from this (apart from inconsistency) is that users define their own levels for the sex column, and are NOT reminded by the validator to please define their levels further in a participant.json.

As a result, these values are hard (or impossible) to parse by software.

E.g., we may have the following participants.json:

participant_id	age	sex
sub-05	25	fem
sub-06	30	ma
sub-07	26	ma

what's fem? what's ma?

How to fix?

I think we should do one of the following:

1. fix the validator so that it emits a warning if age and sex are columns in participants.tsv but have no description in an accompanying participants.json

OR

2. Amend the participants.tsv part of specification and explicitly say that age and sex are "to-be-expected" columns ... and then also define the expected inputs:

• age MUST be a float (years since birth)
  • if a user wants to specify age differently, they must make their own custom column, e.g. age_in_months
• sex MUST be a string (here we need to discuss, which strings we accept. Most straight forward would perhaps be "male", "female", "undefined", "other", but I would like somebody with a bit more experience in inclusive language to make a suggestion here.
  • again: if a user wants to do their own sex column they can make their own custom column with a wide range of acceptable factor levels

[hoechenberger]

• sex MUST be a string (here we need to discuss, which strings we accept. Most straight forward would perhaps be "male, female, undefined", but I would like somebody with a bit more experience in inclusive language to make a suggestion here.

Typically a third option is other or diverse – undefined could potentially be perceived as disrespectful by some. diverse could potentially lead to confusion with gender (however it's the official "third sex / gender" recognized by the German gov't). So I guess, at this time, I'd be in favor of other. The latter would also capture inter, in case we decide not to add that option too.

[nicholst]

age MUST be an integer (years since birth)

I don’t understand the prohibition against decimal ages. It is not uncommon to have exact birthdate and scan date (before anonymisation) with which to compute an age with greater than integer precision.

[sappelhoff]

Thanks @hoechenberger and @nicholst I updated the post with your sensible suggestions

[satra]

@sappelhoff and @hoechenberger - i would suggest making a clearer difference between sex and gender. in bids, sex was adopted in the early days to be chromosomal, while datasets could easily add columns on gender to include and potentially differentiate preference. the language can definitely be improved to both clarify and provide support for including gender in addition to sex, when such information is collected in a study.

more broadly, #423 has some general conventions about augmenting BIDS keys. my vote would be in preference of the validator suggesting that age/sex have to have corresponding entries in participants.json. to map on to known units and levels (as is done in other parts of bids). this allows greater flexibility downstream, especially if BIDS is used for datasets containing in utero participants or non-human species where other age units are more appropriate.

[hoechenberger]

Hello @satra, thanks for your comment! I fully agree that having a separate gender key could be beneficial in certain situations.

I could imagine sth like:

• sex: female, male, other (as suggested above)
• gender: female, male, diverse (as a catch-all), or
• gender: female, male, trans-female, trans-male, inter, other (or diverse)

[sappelhoff]

I fully agree that having a separate gender key could be beneficial in certain situations.

Currently we are limiting this discussion to sex and age, which have an undocumented "special role" in the validator (see my OP). I'd rather not extend this discussion to gender. If users want to specify gender, they can already do so by adding a column to TSV and describing it in JSON.

Note: for gender, the validator WILL WARN if no description is in JSON ... while for age and sex, it will not.

my vote would be in preference of the validator suggesting that age/sex have to have corresponding entries in participants.json. to map on to known units and levels (as is done in other parts of bids). this allows greater flexibility downstream

If I understand correctly, this preference maps to my suggestion 1 in the "how to fix" part of the original post.

I think that allowing downstream flexibility is a good point.

Another point is, that there are many datasets already out there ... and if we clarify now that we want age to be a float and sex to be one of (female, male, other [or whatever we would decide]) that would break several datasets.

Therefore I think it'd be best to:

1. Clarify the specification what we mean by sex and age as well as RECOMMEND (i.e., not REQUIRE) factor levels for sex
2. Amend the bids-validator to explicitly WARN if either age or sex are NOT described in an accompanying JSON file

--> This will not break existing datasets
--> This will hopefully result in more "sane" datasets in the future

HOWEVER

--> existing datasets with age and sex not described in JSON are still "broken" (i.e., cannot be parsed automatically in most cases)

--> users who create new datasets may simply ignore a bids-validator warning ... and hence produce new datasets where age or sex cannot be parsed automatically

I think that's a fair tradeoff and a pragmatic solution.

[satra]

@sappelhoff - yes i was voting for 1. and my reference to gender was simply that the bids key sex should not be confused with gender.

regarding validator and specification interaction, each dataset has a specification version, so a validator should always be checking against the spec rules of that version. this means that for datasets released prior to any change we make to the spec, it should simply accept it as valid (may not be useful, but that's not the validator's responsibility). once the age/sex descriptions are upgraded, current validator can conform to it.

the validator should always be able to make good recommendations at any time. and a new validator could always recommend how to upgrade the dataset from version X to version Y of the spec.

[hoechenberger]

@sappelhoff These suggestions sound very sane to me.

[jasmainak]

hi guys, I would actually suggest going for error. It may sound a bit radical but I think there are parts of the BIDS specification that are too under-specified and flexible which makes it a nightmare to write software for it.

At least regarding the sex/gender issue, I think there is no good reason to leave it as optional. IMO it should be a compulsory field and if the researcher feels really compelled to leave it out, then they should fill it up as 'n/a'. I would also not introduce a new mechanism for unknown/others. We should try to be consistent across the specification.

[jasmainak]

In that sense, this should be considered a bug-fix ... the datasets that break should be retroactively fixed, otherwise we are stuck with a specification that is too flexible.

[hoechenberger]

I would also not introduce a new mechanism for unknown/others.

But there are cases where the biological sex does not clearly fall into the male/female dichotomy, and these are different from unknown (which would be n/a to me). Designing a sex key limited to only two values would be a little like having a handedness key without the option for left-handed participants…

[jasmainak]

yes sure! Sorry I read too quickly, it's fair to have options for sex/genders other than male and female of course.