Add metadata to `Annotations`

[PierreGtch]

Describe the new feature or enhancement

The raw.annotations attribute holds information regarding the events happening in a row file.
However, this information is limited: we only have the onset, a string description and the duration of the events. For some tasks, other information could be useful to have; for example: response time, stimulation parameters, etc.

Moreover, the BIDS format supports arbitrary annotations of the events, but only the onset, description and duration can be saved in the mne.Annotations objects.

The mne.Epochs support arbitrary event information through the metadata attribute, but in the standard processing raw->epochs, the user has to handle the metadata on the side, then include it to the epochs.

Describe your proposed implementation

I propose to add a metadata attribute to the mne.Annotations class, with type None | pd.DataFrame.
When creating epochs, this new field should automatically populate the epochs.metadata attribute.

Describe possible alternatives

Possible alternative when doing a standard processing raw->epochs is to:

• load the raw objects
• load the additional event information on the side (metadata)
• create the epochs
• filter and reorder the metadata rows such they match the created epochs
• set the epochs.metadata attribute

Additional context

This issue is related to mne-tools/mne-bids#1389

@drammock what do you think?

CC @matthiasdold

[sappelhoff]

Just FYI, this is somewhat related to

• container for eye tracking related annotation information #12208

[agramfort]

@PierreGtch do you know about epochs metadata ? https://mne.tools/stable/auto_tutorials/epochs/30_epochs_metadata.html

one challenge is to make sure the object can be serialized in fif for saving.

[PierreGtch]

@agramfort Yes that's the whole point!
When you create epochs, you have to somehow align your additional information columns with the epochs and manually set the epochs' metadata.
This could be handled automatically if the annotations had a metadata attribute :)

[agramfort]

if it's a metadata attribute of the object that is a dataframe it means you cannot save it as csv/txt anyone. You would need to rely on .fif to save any annotations you have and you would eventually make pandas a requirement of mne. So different options: - make the internal representation of Annotations a tabular-like structure with any column except start, duration and description that would be required. You would make start, duration, description properties to access the 3 required columns. - make pandas a requirement of mne. It means the Annotations internals can be a pandas dataframe with any column. - make a new subclass of Annotations with "richer" content but not sure how well we will catch errors everywhere without more typing checks in mne. my 2c

[PierreGtch]

Thanks for your input!
Option1 could streamline the code I think
But I started something like option2 in #13213. However, I made the dependency to pandas optional using lazy imports only when the metadata is provided. My implementation is similar to what we have for the epochs' metadata attribute. I also updated the CSV and txt readers and savers to handle the metadata