Add .nwb as supported format for EEG

[yarikoptic]

NWB is BRAIN Initiative supported, and open data standard https://neurodatawithoutborders.github.io/ .

The main motivation here is to support joint iEEG and EEG datasets for EMBER archive project. Quick google can show that there is a good number of studies which have conjoint EEG and iEEG. Since .nwb is supported for iEEG, it makes only sense to add support for it in EEG modality as well.

There is a good number of .nwb "specific" tools in the ecosystem, with a notable mention of https://neurosift.app which allows "smart" client-side (in browser) access to remote .nwb files for exploration.

It seems that there is also growing number of toolkits in EEG world which support NWB. E.g., EEGLAB already input support for NWB: https://eeglab.org/plugins/NWB-io/ so data would be accessible in this popular toolkit.

Added listing of NWB into EEG formats table + harmonize (change URL also) in iEEG section

Attn @bids-standard/raw-electrophys-eeg and @dorahermes

[effigies]

I support this on the grounds of consistency across modalities and working with other standards.

That said, as someone who wouldn't be tasked with implementing this, I would like to see support from EEG software groups, e.g., MNE-Python (@larsoner), FieldTrip (@robertoostenveld), etc.

[robertoostenveld]

In theory FieldTrip has support for NWB as per fieldtrip/fieldtrip#1419, but in practice it does not work. I just tried it (again, I recall having struggled with it before) and I cannot read any of the NWB test files that I have.

From the FieldTrip pull request:

It is assumed that the user is familiar with the basics of MatNWB (Github MatNWB) and the user's system is in a state that would also allow loading the data using MatNWB (MatNWB must be in the path, generateCore must have been run, the correct schema must be installed, see NWB schemas).

Those assumptions are apparently not met by me and/or my computer.

[robertoostenveld]

Zooming out: I am not yet convinced by the arguments to add NWB to the list of EEG formats. I don't think it is sufficient if someone happens to have data in some format, and if that format is open (whatever loose definition applies for that), that we then extend the BIDS standard with that format. Proliferation of formats has been argued against in many of the BEPs (most recently AFAIK in the EMG BEP).

Did we not have some text somewhere that stated the considerations for extending the list of file formats? ... oh, I found it: it is not under EEG (or elsewhere) but under the iEEG section:

Future versions of BIDS may extend this list of supported file formats. File formats for future consideration MUST have open access documentation, MUST have open source implementation for both reading and writing in at least two programming languages and SHOULD be widely supported in multiple software packages. Other formats that may be considered in the future should have a clear added advantage over the existing formats and should have wide adoption in the BIDS community.

The critical aspects here are "clear added advantage over the existing formats" and "wide adoption" (which I think here refers to the EEG field).

PS I do wish we did have a single standard file format that we could settle on. Perhaps that is something to plan for in BIDS 2.0, where the people that care about a modality get locked up in conclave and are only allowed out when white smoke signals that they have agreed upon a single file format.

[dorahermes]

This discussion is related to guidelines for new formats in issue #2055
Point 5 is important: "The format must meet the technical requirements of neuroscientific workflows". In my interpretation, this means that developers of a format should make sure that common software packages should be able to read and write these data.

On the workflow side for iEEG, data are often collected concurrently with EEG, MEG, ECG etc and everything is collected on one amplifier and stored in one format. IMHO this does not mean that iEEG formats should propagate to such other formats.

[yarikoptic]

It is assumed that the user is familiar with the basics of MatNWB (Github MatNWB) and the user's system is in a state that would also allow loading the data using MatNWB (MatNWB must be in the path, generateCore must have been run, the correct schema must be installed, see NWB schemas).

Those assumptions are apparently not met by me and/or my computer.

That PR is from half a decade ago, or are these observations current? I believe such assumptions were addressed or at least relaxed since then, e.g. see

• [Feature]: avoid demanding users to generateCore and generateExtension manually NeurodataWithoutBorders/matnwb#493

FWIW, as for "one file format to rule them all" insofar I know only two overarching standards of such kind: DICOM and NWB ;-)
My concern here would indeed be NWB's adoption particularly for EEG data (it is the standard for microelectrode neurophysiology data AFAIK, hence would be the main format for BEP032). May be @rly and @oruebel could comment on, point to prominent examples?

[hoechenberger]

I may be a bit naive, but I'm unable to find an official specification of the file format? I can only seem to find implementations, but not an authoritative spec? Can someone help me out here please?

[cbrnr]

I 100% agree with @robertoostenveld. As a maintainer of MNE-Python, I don't think anyone should spend time implementing support unless (1) it is absolutely clear what the advantage of NWB is over other formats, and (2) it can be demonstrated that the format has been widely adopted.

[effigies]

@hoechenberger https://nwb-schema.readthedocs.io/en/latest/format_description.html

[hoechenberger]

Thank you!

[robertoostenveld]

Interesting! I now see that it is based on HDF5 as the container file format and that NWB basically defines the schema that is used within the HDF5 container. As such it is actually quite similar to SNIRF for NIRS.

[yarikoptic]

Correct, it is primarily a schema on top of HDF5 (edit: pynwb and NWB extensions can provide additional convenience interfaces beyond schema driven). Moreover, due to inherent limitations of a single binary file approach, work is in progress to implement NWB on top of Zarr: this is where it is going on - https://github.com/hdmf-dev/hdmf-zarr (HDMF is core library, agnostic of NWB schema).
Related to that, used by @magland's mentioned above neurosift , he implemented https://github.com/NeurodataWithoutBorders/lindi which exposes internal to NWB schema as a json, which is actually a valid zarr archive. See e.g. https://lindi.neurosift.org/dandi/dandisets/000458/assets/6ab37be4-adfe-4bea-a031-eb1a2b0782a8/nwb.lindi.json which is a lindi view of a nwb with EEG and behavioral data which could also be navigated at https://neurosift.app/nwb?url=https://api.dandiarchive.org/api/assets/6ab37be4-adfe-4bea-a031-eb1a2b0782a8/download/&dandisetId=000458&dandisetVersion=0.230317.0039 .

[neuromechanist]

My concern here would indeed be NWB's adoption particularly for EEG data

When looking at the newly merged #2055 as a guideline, it seems all conditions are passing, except for adoption, which, IMO, is a major issue.

Playing with file format selector at NEMAR.org, EEG datasets (currently 303 datasets) are 53% SET files, 27% BVDF, and 20% EDF/BDF.
iEEG datasets (62 datasets) are 56% EDF, 30% BVDF, 11% MEF3, and 0% NWB.

I wonder what is the incentive for BIDS data providers, curators, and software developers to replace their current method and adopt NWB?

To be fair, there is true value in using NWB, but adoption is not there. Using HDF5/NWB is awesome for data management and visualization (which was a motivation to support it in EEGLAB, AFAIK), but isn't that a task for dataset users/services?

[yarikoptic]

Thank you @neuromechanist for the stats! And thanks everyone for chiming in! Indeed adoption for EEG, and apparently iEEG, is lacking :-/ but I would consider it largely to be a chicken&egg issue for this particular case -- this format will not be adopted at this point, unless it is a part of the BIDS which does have EEG supported already.

I wondered, looking forward and not knowing much about those, among those 53% SET files, 27% BVDF, and 20% EDF/BDF. which are open formats developed with community input?

[effigies]

Converting this to draft, as there's no consensus to move forward.

If someone were motivated to add NWB support to BIDS-EEG, what would their strategy be? As @yarikoptic says, it's chicken-and-egg, so NWB dataset adoption will be limited if BIDS does not support it. Would it be to create a reference dataset and verify support in some collection of libraries? Gather some number of practitioners to attest to desire for the format?

[alexrockhill]

https://nwb-schema.readthedocs.io/en/latest/format_description.html

I really like the idea of non-proprietary formats but, as a user, I'm a bit confused how NWB fits in with BIDS; it seems like a specification within a specification that creates more redundancy. Things like metadata and stimulus timing are already in BIDS. How is that going to be handled within validation to check that all the metadata in BIDS matches the metadata in NWB?

From my perspective, BIDS extensions like fUSI-BIDS (considering zarr) and microscopy-BIDS (especially OME-zarr) are very promising ways of moving as much of the metadata into BIDS where it is shared across formats and the BIDS community while having what's essentially a matrix left in the zarr/hdf5 file. Just my two cents, but if an hdf5 is supported, I like the idea of stripping it down to the matrix and what's essential for loading the file in a way that mirrors BIDS and is easy to validate the zarr/hdf5 against the sidecar like OME-zarr does.

That sidebar is pretty a bit tangential to the discussion of how to have NWB in BIDS but how I would approach it is: 1) Consider more if this is a good idea or if there's a middle ground like hdf5 support that could be inter-compatible without being too over-specified, 2) if sticking with the original plan, convert an existing dataset in a different format to NWB as an example and upload to a public storage site, 3) make an issue on several EEG libraries like MNE, Fieldtrip etc. discussing adding it as an example and see if you can get help and get it into at least one library. Maybe if there's also a lot of NWB EEG data laying around that should be published, that might also help and be a nice place to start (by uploading the original draft of a NWB BIDS dataset as accompaniment to a paper for instance)

[yarikoptic]

Thank you @alexrockhill for sharing your perspective. Here is my condensed comment to the questions you posed + my answer to @effigies of what could be next:

BIDS grew from its initial .nii.gz + .json (since .nii.gz lacks metadata) + named folders + subjects.tsv (since we want to know basic demographics) into a standard which covers a WIDE range of domain specific file formats and metadata. We use original file formats so that existing domain specific tools could use the files in BIDS datasets. IMHO it would be non-productive for us to try forcing underlying efforts like OME-Zarr/NWB/DICOM/etc to adopt our BIDSy .json + .tsv files as the requirement for the metadata to accompany their already nicely modularized structured data formats.

As a result, I take BIDS as a "harmonization + organization + summarization layer" on top of domain specific file formats/standards to organize them into datasets across different acquisitions, potentially from different modalities, typically of different file formats. And somewhat more elaborate and visual on the topic of differences among standards -- see SfN 2025 poster we had https://zenodo.org/uploads/18333008

With that in mind, for this PR, I would

1. similarly to what I think was and will be done in virtually any other BEP -- I would look for alignment between metadata in BIDS sidecar .json + .tsv files to what is in NWB files: what metadata in NWB for EEG maps within BIDS EEG already or what is missing in NWB or BIDS? For example, taking ongoing BEP032 which introduces support for .nwb, @CodyCBakerPhD in ongoing satellite to BEP032 effort nwb2bids project established similar mappings: e.g. for microephys _channels.tsv see are table to alignment between BIDS and NWB. Such approach also loosely relates in what we do in schema ATM manually annotating correspondence to original DICOM fields, which might come handy later if we decide to adopt DICOM "file standard" within BIDS (DICOM "file-format" support #1552).

2. do work out a BIDS-aware converter for some less popular file format(s) which were permitted for EEG modality into how it would look like if converted to NWB, and potentially demonstrate advantages (e.g. visualization with neurosift). May be there is in openneuro some dataset with some .bidsignore'd eeg files in custom format? That is where I might look for assistance from @effigies to discover potential use case, and @CodyCBakerPhD , @rly and @oruebel to consult on representation in NWB.

3. potentially even recommend .nwb as the structure for some new data yet to be harmonized to BIDS which is neither of those permitted for EEG file formats.

[alexrockhill]

Yeah I'm mixing a bit my opinion/desire for more nii-like formats (less extra stuff on top of the matrix) in BIDS and my opinion on NWB in BIDS. Always the critical tradeoff: accepting more formats and losing specification specificity and accepting fewer and making it harder to integrate more communities. I like the underlying hdf5 format, I hope the traction builds for more of that in BIDS.

[yarikoptic]

FWIW, @arnodelorme reported at OHBM 2026 brainhack about https://github.com/sccn/nwbio

the slide -- click to expand

so there is support in the flagman eeglab for EEG load(import)/save(export) in NWB