BIDS Extension for Sleep Data and Annotations

[Pouya-Ta]

Your idea

Title:

BIDS Extension for Sleep Data and Annotations

Introduction:
Dear BIDS community, I am Pouya Taghipour Langrodi, an undergraduate student mentored by Ali Saberi, a doctoral candidate at the Sleep and Memory lab at the Donders Institute for Brain, Cognition, and Behaviour, and a member of Lullabyte, a multidisciplinary doctoral network that explores the effects of music on sleep using computational techniques. As part of this initiative, we are proposing a new extension to the Brain Imaging Data Structure (BIDS) to better support researchers working on sleep studies. Our goal is to provide a systematic framework for organizing, storing, and sharing sleep-related data within the neuroscience community. This proposal is currently under development, and we are opening this issue to invite feedback and discussion from the community. While our broader objective is to support general sleep data in BIDS, we will begin by focusing on the standardization of sleep staging annotations, a foundational component in many sleep research studies.

Motivation:
Sleep research is a rapidly growing field that spans neuroscience, psychology, physiology, and computational science. Despite this growth, the organization and sharing of sleep data remain inconsistent, with researchers relying on a variety of custom formats and undocumented conventions. With the growing use of AI and machine learning in sleep research, precise annotations are essential because they serve as the reliable ground truth needed to train and validate these models effectively. At present, the absence of a unified framework within BIDS for storing the full spectrum of sleep data and adapting existing annotation specifications limits reproducibility, impedes data sharing, and complicates automated analysis. Our goal is to develop a user-friendly, flexible, and accessible extension to BIDS that both extends support for a wider range of sleep data types and adapts current specifications for sleep annotations to better serve the research community.

Scope:
The scope of this extension includes:

• Develop a flexible specification for storing diverse sleep data and metadata within BIDS.
• Adapt and extend existing BIDS specifications to better support sleep annotations.
• Engage with the sleep and neuroscience research communities for continuous feedback and refinement.
• Ensure compatibility with existing BIDS standards and tools.
• Provide validation tools tailored specifically for sleep data and annotations.

Preliminary Work:
In collaboration with another BIDS extension project at the University of Stuttgart, a small-scale survey of the research community has already been carried out. The purpose of this survey was to evaluate the needs of sleep researchers, the data formats they use, and the existing limitations within BIDS, which would help shape the development of the extension. The results of the survey clearly highlighted the need for this proposed extension and offered direction for the initial design considerations.

The key findings from our survey that support this proposal are:

• Feedback from 13 respondents with past experience in sleep studies showed that 69% are likely to adopt a BIDS extension for sleep data (selecting 4 or 5 on a 5-point scale), with a mean likelihood score of 4.0 out of 5.
• Key BIDS features seen as most useful by sleep researchers include:
  • Facilitating data sharing and collaboration (70% of respondents)
  • Metadata organization and accessibility (60%)
  • Long-term data storage and reproducibility (55%)
  • Support for multimodal data formats, including EEG and other physiological data relevant to sleep research (45%)
• For sleep-specific metadata, the most prioritized items were:
  • Sleep-related events (e.g., arousals, stimuli presentation) (73%)
  • Sleep staging details (67%)
  • Behavioral or subjective sleep responses (e.g., sleep questionnaires, sleep diary) (67%)
• Additionally, 80% of sleep researchers rated standardized structures for enabling automated analysis pipelines as ‘important’ or ‘extremely important’, highlighting a strong need for standardization in sleep research data management.

Planned Deliverables

• Formal Specification: A complete document defining the file formats, naming conventions, and metadata for sleep-related annotations.
• BIDS Schema Update: Updating the official BIDS schema to incorporate the new sleep data structures, enabling validation.
• Example Dataset: A public example dataset demonstrating the practical implementation of the standard.

Use Cases

• For Data Contributors: To easily share sleep datasets in a standardized, machine-readable format that increases data value and impact.
• For Secondary Data Users: To accelerate meta-analyses by using universal scripts to automatically parse and aggregate data from multiple studies.
• For AI/ML Developers: To access a growing repository of consistently formatted data for training and validating automated scoring models.

Community Involvement:
We invite contributions and involvement from researchers, data curators, and developers engaged in sleep studies. A shared Google Doc for community input will be accessible shortly, in addition to discussions on GitHub.

Lead Contributors:

Contributor 1 :
Name: Pouya Taghipour
Affiliations: Sleep and Memory Lab at Donders Institute for Brain, Cognition, and Behaviour; Amirkabir University of Technology
Email: Pouya_Taghipour@aut.ac.ir

Contributor 2 :
Name: Ali Saberi (https://dreslerlab.org/team/)
Affiliations: Donders Institute for Brain, Cognition, and Behaviour (Radboudumc); Lullabyte MSCA Doctoral Network (https://lullabyte.eu/)
Email: ali.saberi@donders.ru.nl

We appreciate your attention in advance. We are eager for the community's insightful feedback and teamwork!

[christinerogers]

Somnography data largely falls under the EEG-BIDS spec already, including derivatives (e.g. sleep analysis pipelines).
Could your scope address this practice and argue for how this is distinct from annotated EEG (e.g.) ?
The AASM classification could be an extension to the HED ontology supported for annotations within BIDS.

[neuromechanist]

Thanks very much Pouya and Ali for brining this up, and also for sharing via the Google Groups (while people don't reply on Google Groups often, they get pinged to respond here 😉).

I agree to with the comments from @christinerogers, @yarikoptic and @agramfort here and on the Google doc that sleep data can be almost be completely annotated FAIRly using HED. See HED schema, select Schema: score, from the drop down menu and find/search for sleep:

Having said that, I think HED/BIDS can be minimally extended to include scorer-id, confidence, channels-used-for-score, etc.

CC: @VisLab and @dorahermes

[christinerogers]

Hi @Pouya-Ta @alitsaberi,
Tagging for followup response based on the above BIDS maintainer comments and community comments in your Googledoc from last year.

• Let us know your questions, corner cases and examples.
• We would very much welcome contributions of sleep data/annotations under bids-examples, as well as clarifying PRs to the BIDS website/spec to better illustrate how Sleep data and Annotations are represented in BIDS format.

Christine (BIDS Maintainer, like @neuromechanist)

cc @robertoostenveld / Donders colleagues

[robertoostenveld]

If HED/BIDS is minimally extended, I think that sleep staging could be appropriately documented in the events.tsv and events.json files.

[VisLab]

HED score library has a number of terms relevant to sleep scoring including:

• Apnea-hypopnea-index/#
• Periodic-respiration/#
• Tachypnea/#
• Other-respiration-feature/#

Sleep-architecture

• Normal-sleep-architecture
• Abnormal-sleep-architecture

Sleep-stage-reached

• Sleep-stage-N1/#
• Sleep-stage-N2/#
• Sleep-stage-N3/#
• Sleep-stage-REM/#
• Sleep-spindles
• Vertex-wave
• K-complex
• Saw-tooth-waves
• POSTS
• Hypnagogic-hypersynchrony
• Non-reactive-sleep

The # means that a value can be substituted after the tag. In BIDS these values can be in a column of the events.tsv file. There are other tags for cardiac features etc. HED score is directly based on the SCORE standard, which has been endorsed by the European Chapter of the International Federation of Clinical Neurophysiology (IFCN) and the International League Against Epilepsy (ILAE) Commission on European Affairs.

[dorahermes]

I agree with suggestions above and would consider adding instructions to the BIDS-starter-kit, for example in the Tutorials/conversion section, with additional explanations on how to BIDSify sleep study related data with proper annotations (which would be great if they can be HED based).

[alitsaberi]

Hi everyone, thanks a lot for the thoughtful feedback and for keeping the discussion going. This has been very helpful in clarifying both the scope and the most productive next steps for us.

We fully agree that somnography/PSG data itself is already largely covered by BIDS, in particular through EEG-BIDS. Our motivation is not that sleep data cannot be represented in BIDS today, but rather that in practice it is not represented in a standard, uniform way, nor is it yet extensively adopted by sleep datasets, which often rely on custom formats and undocumented conventions.

Our original goal with this BEP was therefore to develop a unified and well-illustrated framework within BIDS that makes existing capabilities easier to use and more consistently adopted by the sleep research community. As mentioned in the proposal, we intended to start by focusing on sleep staging annotations, as a foundational and widely shared component across sleep studies.

We strongly agree with the consensus emerging here that HED provides the right semantic backbone for sleep staging and sleep annotations. We see a very promising path forward in:

• minimally extending HED where needed (e.g., scorer-related metadata),
• representing sleep staging via events.tsv / events.json,
• contributing clear, concrete examples under bids-examples, and
• possibly submitting clarifying PRs to the BIDS website/spec to better illustrate how sleep data and annotations are represented using existing standards.

That said, as we started mapping real sleep datasets to HED/BIDS, a few practical corner cases and open questions came up that we think are worth discussing explicitly:

Sleep stage taxonomies

• Current HED score terms focus on NREM/REM sleep, while Wake is commonly treated as a stage in sleep scoring.
• Beyond AASM, other standards exist (e.g., Rechtschaffen & Kales), and some datasets/algorithms use reduced label sets (e.g., 2-class: Wake/Sleep; 3-class: Wake/NREM/REM; 4-class: Wake/Light/Deep/REM).
• We’d appreciate guidance on best practices for representing these variants consistently within HED.

Multiple annotations for the same/overlapping time segment

• Sleep data often includes multi-scorer annotations, consensus scores, or annotations derived from different channel subsets.
• Algorithm-derived scores may coexist with manual scores.
• How should multiple parallel annotations for the same segment be best represented and distinguished (e.g., scorer ID, method, channel set)?

Confidence and probabilistic outputs

• Automated methods increasingly output confidence values or full hypnodensities.
• These may be provided at high temporal resolution (not just standard 30-second epochs).
• We’d like to clarify how confidence/probability information fits into events.tsv + HED in a scalable way.

Modalities and organization of data

• While PSG is the gold standard (EEG, EOG, EMG, etc.), wearable devices and real-world datasets introduce additional variability.
• Based on our understanding of BIDS, there appear to be (at least) two common organizational patterns:
  • bundling signals into a single EDF (e.g., _acq-psg_eeg.edf, as in https://openneuro.org/datasets/ds005555/versions/1.0.0),
  • separating EEG, EMG, and physio recordings into distinct modality-specific files.
• We’d appreciate guidance on recommended organization patterns and, relatedly, which modality’s events.tsv should host sleep staging annotations.
• Additionally, in cases where sleep staging is derived from non-EEG data (e.g., heart rate), which BIDS modality should be preferred?

Unscored or unknown epochs

• In real datasets, some epochs are intentionally left unscored or marked as unknown.
• We’d like to clarify best practices for representing these cases explicitly.

We really appreciate the guidance so far and are very happy to align our efforts with existing BIDS and HED directions.

[VisLab]

I suggest you join a HED Working Group Meeting to discuss. This is a very interesting application and we would be happy to discuss. Email hed.maintainers@gmail.com so we can get things moving on this. (Anyone else interested -- please email too). We will also open a discussion on the hed-standard/discussions forum.

On individual comments:

Sleep stage taxonomies

• Current HED score terms focus on NREM/REM sleep, while Wake is commonly treated as a stage in sleep scoring.
• Beyond AASM, other standards exist (e.g., Rechtschaffen & Kales), and some datasets/algorithms use reduced label sets (e.g., 2-class: Wake/Sleep; 3-class: Wake/NREM/REM; 4-class: Wake/Light/Deep/REM).
• We’d appreciate guidance on best practices for representing these variants consistently within HED.

This is a vocabulary issue -- one possibility is to develop a dedicated sleep HED library schema that incorporates different standards of annotation. This would involve a community effort, which the HED Working Group would be happy to support.

Multiple annotations for the same/overlapping time segment

• Sleep data often includes multi-scorer annotations, consensus scores, or annotations derived from different channel subsets.
• Algorithm-derived scores may coexist with manual scores.
• How should multiple parallel annotations for the same segment be best represented and distinguished (e.g., scorer ID, method, channel set)?

HED already handles the overlapping segments, multiple annotators, etc. This would be good information to include in a sleep tutorial.

Confidence and probabilistic outputs

• Automated methods increasingly output confidence values or full hypnodensities.
• These may be provided at high temporal resolution (not just standard 30-second epochs).
• We’d like to clarify how confidence/probability information fits into events.tsv + HED in a scalable way.

There are also mechanisms for this already in HED. If something doesn't work well, HED can be extended, but offhand, I think the existing mechanisms would work fine.

Modalities and organization of data

• While PSG is the gold standard (EEG, EOG, EMG, etc.), wearable devices and real-world datasets introduce additional variability.
• Based on our understanding of BIDS, there appear to be (at least) two common organizational patterns:
• bundling signals into a single EDF (e.g., _acq-psg_eeg.edf, as in https://openneuro.org/datasets/ds005555/versions/1.0.0),
• separating EEG, EMG, and physio recordings into distinct modality-specific files.
• We’d appreciate guidance on recommended organization patterns and, relatedly, which modality’s events.tsv should host sleep staging annotations.
• Additionally, in cases where sleep staging is derived from non-EEG data (e.g., heart rate), which BIDS modality should be preferred?

This is something that I don't think has been definitively resolved in the BIDS community. It would require broader discussion in BIDS forums.

Unscored or unknown epochs

• In real datasets, some epochs are intentionally left unscored or marked as unknown.
• We’d like to clarify best practices for representing these cases explicitly.

HED has mechanisms for doing this --- best practices are yet to be specified.

@robertoostenveld @dorahermes @smakeig @neuromechanist @arnodelorme @christinerogers

[yarikoptic]

Listening to a webinar which presented ongoing work to define CDEs using LinkML: https://github.com/linkml/nsrr-sleep-cde-schema/blob/main/src/sleep_cde_schema/schema/sleep_cde_schema.yaml . Worth checking out and potentially aligning to/reusing. edit: The NSRR team can always be reached via sleepdata.org, or at support@sleepdata.org

[viranovskaya]

Hi! I am working on an open Sleep-EDF evaluation that compares expert hypnograms with automated YASA sleep staging. The corner cases discussed above overlap closely with decisions I have had to document in that workflow.

Rather than proposing another extension in the abstract, I would be happy to prepare a small, concrete bids-examples or tutorial contribution demonstrating:

• parallel expert and algorithm hypnograms in events.tsv / events.json;
• explicit R&K-to-AASM mapping, including S3/S4 to N3;
• movement and unscored/unknown epochs;
• scorer/method, channel-set, and confidence metadata, using existing HED mechanisms where possible; and
• a documented edge-Wake analysis policy kept separate from the underlying annotation representation.

I can keep the fixture small and openly redistributable (or synthetic if preferable), include a short README, and cross-check the terminology against the NSRR LinkML sleep CDE schema mentioned above.

Would a first PR to bids-examples be the preferred starting point, or would you recommend joining a HED Working Group meeting before drafting the example?

Related evaluation repository: https://github.com/viranovskaya/sleep-eeg-staging-evaluation

[effigies]

No harm in opening a PR against bids-examples. That way we can see if there are any validation issues and focus discussion.

[viranovskaya]

Sounds good. I’ll open a small PR in bids-examples so we can check the validator output and talk through the structure.

I’ll start simple: sleep-stage annotations with expert vs automated labels, basic scorer/source info, and clear handling of unscored or movement epochs.