FIX: Read Nihon Kohden annotation file accurately

[myd7349]

Reference issue (if any)

Fix #11267.

What does this implement/fix?

This PR adds support for reading sub event log blocks in Nihon Kohden EEG annotation files (.LOG).

In certain versions of the .LOG files, in addition to the standard event log blocks, there are sub event log blocks.

• The event log block contains timestamps in HHMMSS format.
• The sub event log block provides additional millisecond and microsecond precision in the form of cccuuu.
• If the event text is too long, it is split into two parts and stored separately in the event log block and the sub event log block.

Additional information

I noticed that @jacobshaw42 also attempted something similar in #11431, but for some reason, the PR was closed.

This PR differs from #11431 in the following ways:

1. Sub event log blocks are not limited to 'EEG-1200A V01.00'

   For example, in MB0400FU.EEG, the device type is EEG-1100C V01.00, yet it does contain sub event log blocks.

   Nihon Kohden does not clearly specify which device types or software versions generate .LOG files that include sub event log blocks. I previously tried to implement a function to determine whether sub event log blocks are present, based on the device type:

   def contains_sub_event_blocks(device_type: str) -> bool:
       device_types_with_sub_events = (
           "EEG-1100A V01.00",
           "EEG-1100A V02.0",
           "EEG-1100B V01.00",
           "EEG-1100C V01.00",
           "EEG-2100  V01.00",
           "EEG-2100  V02.00",
           "EEG-1100A V02.00",
           "EEG-1100B V02.00",
           "EEG-1100C V02.00",
       )
       device_types_without_sub_events = (
           "QI-403A   V01.00",
           "QI-403A   V02.00",
       )
   
       if (
           device_type.startswith("EEG-2110")
           or device_type in device_types_without_sub_events
       ):
           return False
       elif device_type in device_types_with_sub_events:
           return True
   
       raise NotImplementedError(f"Unsupported device type: {device_type}.")

   However, I found this approach overly complicated, so I switched to a more general strategy.

   In Nihon Kohden .LOG files, the control block can define up to 43 event log blocks. When sub event blocks are present:

   • Blocks 1–21 define the offsets for standard event log blocks,
   • Block 22 may be unused,
   • Blocks 23–43 define the offsets for the corresponding sub event log blocks (matching 1–21 one-to-one).

   Therefore, this PR assumes that sub event log blocks are present when the number of log blocks (n_logblocks) parsed from the control block does not exceed 21.

   BTW, in nk2edf, the presence of sub event log blocks is assumed.

   Since the logic for reading event blocks and sub event blocks is largely similar, I refactored the relevant code in _read_nihon_annotations into a helper function _read_event_log_block. Two conditions are used to ensure a sub event block is valid:

   • The block offset in the control block must be greater than zero.
   • The data name inside the block must match the device type from the device block.

2. Decode event description at last

   Because event text can be split across the event log block and the sub event log block, this PR concatenates the byte strings from both blocks before decoding. This affects the following logic:

   • Since the event log block is no longer decoded directly, the strptime method can no longer be used to parse the HHMMSS time(which is a byte string, not a str). Instead, the time is parsed using int for each component.

[myd7349]

The tests failed because:

mne/io/nihon/tests/test_nihon.py:41: in test_nihon_eeg
    assert an1["onset"] == an2["onset"]
E   assert np.float64(1.14) == np.float64(1.0)

I saw a note here:

mne-python/mne/io/nihon/tests/test_nihon.py

Line 39 in dda454d

# EDF has some weird annotations, which are not in the LOG file

There are only two events in the .LOG file, but four annotations in the .EDF file. So I wrote a test script:

# encoding: utf-8

import os.path
import urllib.request

import edfio
from mne.io.nihon import read_raw_nihon


def download_file(url: str, output_path: str):
    try:
        urllib.request.urlretrieve(url, output_path)
        return True
    except Exception as e:
        print(e)
        return False

def test_edf():
    file = r"MB0400FU.EDF"
    if not os.path.exists(file):
        if not download_file("https://raw.githubusercontent.com/mne-tools/mne-testing-data/refs/heads/master/NihonKohden/MB0400FU.EDF",
                             file):
            return

    edf = edfio.read_edf(file)
    for annotation in edf.get_annotations():
        print(annotation)


def test_eeg():
    eeg_file = "MB0400FU.EEG"
    elec_file = "MB0400FU.21E"
    pnt_file = "MB0400FU.PNT"
    log_file = "MB0400FU.LOG"

    if not os.path.exists(eeg_file):
        if not download_file("https://raw.githubusercontent.com/mne-tools/mne-testing-data/refs/heads/master/NihonKohden/MB0400FU.EEG",
                             eeg_file):
            return

    if not os.path.exists(elec_file):
        if not download_file("https://raw.githubusercontent.com/mne-tools/mne-testing-data/refs/heads/master/NihonKohden/MB0400FU.21E",
                             elec_file):
            return

    if not os.path.exists(pnt_file):
        if not download_file("https://raw.githubusercontent.com/mne-tools/mne-testing-data/refs/heads/master/NihonKohden/MB0400FU.PNT",
                             pnt_file):
            return

    if not os.path.exists(log_file):
        if not download_file("https://raw.githubusercontent.com/mne-tools/mne-testing-data/refs/heads/master/NihonKohden/MB0400FU.LOG",
                             log_file):
            return

    raw = read_raw_nihon(eeg_file)

    for onset, duration, description in zip(
        raw.annotations.onset,
        raw.annotations.duration,
        raw.annotations.description,
    ):
        print(onset, description)


if __name__ == "__main__":
    test_edf()
    test_eeg()

Output:

EdfAnnotation(onset=0.0, duration=None, text='+0.000000')
EdfAnnotation(onset=0.0, duration=None, text='Segment: REC START ALLE EEG')
EdfAnnotation(onset=1.0, duration=None, text='+1.140000')
EdfAnnotation(onset=1.0, duration=None, text='A1+A2 OFF')
Loading MB0400FU.EEG
Reading header from D:\edf_demo\MB0400FU.EEG
Found PNT file, reading metadata.
Found LOG file, reading events.
0.0 REC START ALLE EEG
1.0 A1+A2 OFF

[myd7349]

It appears that MB0400FU.EDF is suspicious. Therefore, I used nk2edf to convert MB0400FU.EEG to EDF format: MB0400FU_1-1+.zip.

By the way, the RESET condition shown above is not an actual event or annotation — it’s just a trigger label parsed from the Events/Markers channel:

https://gitlab.com/Teuniz/EDFbrowser/-/blob/master/edf_annotations.cpp?ref_type=heads#L157

When reading MB0400FU_1-1+.zip using edfio or mne.io.read_raw_edf, it returns two annotations.

[myd7349]

I also noticed that Nihon Kohden's software supports a type of annotation called P_COMMENT. When using P_COMMENT, the event text stored in the .LOG file is simply "P_COMMENT", while the actual comment appears to be stored elsewhere.

[myd7349]

Therefore, I believe this test failure should be addressed by updating MB0400FU.EDF in the following way:

1. Convert MB0400FU.EEG to EDF using nk2edf, and replace the current file with the newly generated EDF(MB0400FU_1-1%2B.zip, for example); or
2. Re-export a new EDF using Nihon Kohden's software.Change test code
   Testing revealed that the file at https://github.com/mne-tools/mne-testing-data/blob/master/NihonKohden/MB0400FU.EDF appears to be an EDF file exported using Nihon Kohden's Neuro Workbench software. This export process is actually carried out by invoking a program called BESA EEG Converter. On my machine, after converting MB0400FU.EEG with BESA EEG Converter version 1.0.0.25, the resulting EDF file seems to have the same issue as the one at https://github.com/mne-tools/mne-testing-data/blob/master/NihonKohden/MB0400FU.EDF.