Skip to content

FIX: Read Nihon Kohden annotation file accurately#13251

Merged
larsoner merged 9 commits intomne-tools:mainfrom
myd7349:fix-issue-11267
Oct 22, 2025
Merged

FIX: Read Nihon Kohden annotation file accurately#13251
larsoner merged 9 commits intomne-tools:mainfrom
myd7349:fix-issue-11267

Conversation

@myd7349
Copy link
Copy Markdown
Contributor

@myd7349 myd7349 commented May 17, 2025

Reference issue (if any)

Fix #11267.

What does this implement/fix?

This PR adds support for reading sub event log blocks in Nihon Kohden EEG annotation files (.LOG).

In certain versions of the .LOG files, in addition to the standard event log blocks, there are sub event log blocks.

  • The event log block contains timestamps in HHMMSS format.
  • The sub event log block provides additional millisecond and microsecond precision in the form of cccuuu.
  • If the event text is too long, it is split into two parts and stored separately in the event log block and the sub event log block.

Additional information

I noticed that @jacobshaw42 also attempted something similar in #11431, but for some reason, the PR was closed.

This PR differs from #11431 in the following ways:

  1. Sub event log blocks are not limited to 'EEG-1200A V01.00'

    For example, in MB0400FU.EEG, the device type is EEG-1100C V01.00, yet it does contain sub event log blocks.

    Nihon Kohden does not clearly specify which device types or software versions generate .LOG files that include sub event log blocks. I previously tried to implement a function to determine whether sub event log blocks are present, based on the device type:

    def contains_sub_event_blocks(device_type: str) -> bool:
        device_types_with_sub_events = (
            "EEG-1100A V01.00",
            "EEG-1100A V02.0",
            "EEG-1100B V01.00",
            "EEG-1100C V01.00",
            "EEG-2100  V01.00",
            "EEG-2100  V02.00",
            "EEG-1100A V02.00",
            "EEG-1100B V02.00",
            "EEG-1100C V02.00",
        )
        device_types_without_sub_events = (
            "QI-403A   V01.00",
            "QI-403A   V02.00",
        )
    
        if (
            device_type.startswith("EEG-2110")
            or device_type in device_types_without_sub_events
        ):
            return False
        elif device_type in device_types_with_sub_events:
            return True
    
        raise NotImplementedError(f"Unsupported device type: {device_type}.")

    However, I found this approach overly complicated, so I switched to a more general strategy.

    In Nihon Kohden .LOG files, the control block can define up to 43 event log blocks. When sub event blocks are present:

    • Blocks 1–21 define the offsets for standard event log blocks,
    • Block 22 may be unused,
    • Blocks 23–43 define the offsets for the corresponding sub event log blocks (matching 1–21 one-to-one).

    Therefore, this PR assumes that sub event log blocks are present when the number of log blocks (n_logblocks) parsed from the control block does not exceed 21.

    BTW, in nk2edf, the presence of sub event log blocks is assumed.

    Since the logic for reading event blocks and sub event blocks is largely similar, I refactored the relevant code in _read_nihon_annotations into a helper function _read_event_log_block. Two conditions are used to ensure a sub event block is valid:

    • The block offset in the control block must be greater than zero.
    • The data name inside the block must match the device type from the device block.
  2. Decode event description at last

    Because event text can be split across the event log block and the sub event log block, this PR concatenates the byte strings from both blocks before decoding. This affects the following logic:

    • Since the event log block is no longer decoded directly, the strptime method can no longer be used to parse the HHMMSS time(which is a byte string, not a str). Instead, the time is parsed using int for each component.

@myd7349 myd7349 marked this pull request as ready for review May 17, 2025 10:23
@myd7349
Copy link
Copy Markdown
Contributor Author

myd7349 commented May 17, 2025

The tests failed because:

mne/io/nihon/tests/test_nihon.py:41: in test_nihon_eeg
    assert an1["onset"] == an2["onset"]
E   assert np.float64(1.14) == np.float64(1.0)

I saw a note here:

# EDF has some weird annotations, which are not in the LOG file

There are only two events in the .LOG file, but four annotations in the .EDF file. So I wrote a test script:

# encoding: utf-8

import os.path
import urllib.request

import edfio
from mne.io.nihon import read_raw_nihon


def download_file(url: str, output_path: str):
    try:
        urllib.request.urlretrieve(url, output_path)
        return True
    except Exception as e:
        print(e)
        return False

def test_edf():
    file = r"MB0400FU.EDF"
    if not os.path.exists(file):
        if not download_file("https://raw.githubusercontent.com/mne-tools/mne-testing-data/refs/heads/master/NihonKohden/MB0400FU.EDF",
                             file):
            return

    edf = edfio.read_edf(file)
    for annotation in edf.get_annotations():
        print(annotation)


def test_eeg():
    eeg_file = "MB0400FU.EEG"
    elec_file = "MB0400FU.21E"
    pnt_file = "MB0400FU.PNT"
    log_file = "MB0400FU.LOG"

    if not os.path.exists(eeg_file):
        if not download_file("https://raw.githubusercontent.com/mne-tools/mne-testing-data/refs/heads/master/NihonKohden/MB0400FU.EEG",
                             eeg_file):
            return

    if not os.path.exists(elec_file):
        if not download_file("https://raw.githubusercontent.com/mne-tools/mne-testing-data/refs/heads/master/NihonKohden/MB0400FU.21E",
                             elec_file):
            return

    if not os.path.exists(pnt_file):
        if not download_file("https://raw.githubusercontent.com/mne-tools/mne-testing-data/refs/heads/master/NihonKohden/MB0400FU.PNT",
                             pnt_file):
            return

    if not os.path.exists(log_file):
        if not download_file("https://raw.githubusercontent.com/mne-tools/mne-testing-data/refs/heads/master/NihonKohden/MB0400FU.LOG",
                             log_file):
            return

    raw = read_raw_nihon(eeg_file)

    for onset, duration, description in zip(
        raw.annotations.onset,
        raw.annotations.duration,
        raw.annotations.description,
    ):
        print(onset, description)


if __name__ == "__main__":
    test_edf()
    test_eeg()

Output:

EdfAnnotation(onset=0.0, duration=None, text='+0.000000')
EdfAnnotation(onset=0.0, duration=None, text='Segment: REC START ALLE EEG')
EdfAnnotation(onset=1.0, duration=None, text='+1.140000')
EdfAnnotation(onset=1.0, duration=None, text='A1+A2 OFF')
Loading MB0400FU.EEG
Reading header from D:\edf_demo\MB0400FU.EEG
Found PNT file, reading metadata.
Found LOG file, reading events.
0.0 REC START ALLE EEG
1.0 A1+A2 OFF

@myd7349
Copy link
Copy Markdown
Contributor Author

myd7349 commented May 17, 2025

It appears that MB0400FU.EDF is suspicious. Therefore, I used nk2edf to convert MB0400FU.EEG to EDF format: MB0400FU_1-1+.zip.

By the way, the RESET condition shown above is not an actual event or annotation — it’s just a trigger label parsed from the Events/Markers channel:

https://gitlab.com/Teuniz/EDFbrowser/-/blob/master/edf_annotations.cpp?ref_type=heads#L157

When reading MB0400FU_1-1+.zip using edfio or mne.io.read_raw_edf, it returns two annotations.

@myd7349
Copy link
Copy Markdown
Contributor Author

myd7349 commented May 17, 2025

I also noticed that Nihon Kohden's software supports a type of annotation called P_COMMENT. When using P_COMMENT, the event text stored in the .LOG file is simply "P_COMMENT", while the actual comment appears to be stored elsewhere.

@myd7349
Copy link
Copy Markdown
Contributor Author

myd7349 commented May 17, 2025

Therefore, I believe this test failure should be addressed by updating MB0400FU.EDF in the following way:

  1. Convert MB0400FU.EEG to EDF using nk2edf, and replace the current file with the newly generated EDF(MB0400FU_1-1%2B.zip, for example); or
  2. Re-export a new EDF using Nihon Kohden's software.Change test code
    Testing revealed that the file at https://github.com/mne-tools/mne-testing-data/blob/master/NihonKohden/MB0400FU.EDF appears to be an EDF file exported using Nihon Kohden's Neuro Workbench software. This export process is actually carried out by invoking a program called BESA EEG Converter. On my machine, after converting MB0400FU.EEG with BESA EEG Converter version 1.0.0.25, the resulting EDF file seems to have the same issue as the one at https://github.com/mne-tools/mne-testing-data/blob/master/NihonKohden/MB0400FU.EDF.

@myd7349 myd7349 force-pushed the fix-issue-11267 branch 4 times, most recently from fe60298 to 2265c3d Compare May 23, 2025 13:08
* upstream/main: (46 commits)
  MAINT: Restore edfio git install (mne-tools#13421)
  Support preload=False for the new EEGLAB single .set format (mne-tools#13096)
  [pre-commit.ci] pre-commit autoupdate (mne-tools#13453)
  MAINT: Restore PySide6 6.10.0 testing (mne-tools#13446)
  MAINT: Auth [skip azp] [skip actions]
  MAINT: Deploy [circle deploy] [skip azp] [skip actions]
  Bump github/codeql-action from 3 to 4 in the actions group (mne-tools#13442)
  ENH: Dont constrain fiducial clicks to mesh vertices (mne-tools#13445)
  Use timezone-aware ISO 8601 for website timestamp (mne-tools#13347)
  [pre-commit.ci] pre-commit autoupdate (mne-tools#13443)
  FIX: Update osf.io links to new format (mne-tools#13440)
  MAINT: Ensure full checkout is used (mne-tools#13439)
  Add BDF export (mne-tools#13435)
  [pre-commit.ci] pre-commit autoupdate (mne-tools#13434)
  [pre-commit.ci] pre-commit autoupdate (mne-tools#13431)
  MAINT: Update code credit (mne-tools#13432)
  FIX, TST: Try to get test_export_epochs_eeeglab passing again (mne-tools#13428)
  FIX: Add on_few_samples parameter to core rank estimation (mne-tools#13350)
  MAINT: Reenable mpl nightly (mne-tools#13426)
  [pre-commit.ci] pre-commit autoupdate (mne-tools#13427)
  ...
Copy link
Copy Markdown
Member

@larsoner larsoner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry we missed this one @myd7349 ! In the future if we don't respond in a week or so feel free to ping us again for review, this one just slipped through the cracks

I pushed a couple tiny commits to add your test file from option (1) above, bump testing version, adjust the test appropriately, and adjust a couple tiny style things. Marking for merge-when-green, thanks in advance @myd7349 !

@larsoner larsoner enabled auto-merge (squash) October 22, 2025 14:11
@larsoner larsoner merged commit 181fea1 into mne-tools:main Oct 22, 2025
32 checks passed
@myd7349 myd7349 deleted the fix-issue-11267 branch October 23, 2025 00:21
larsoner added a commit to larsoner/mne-python that referenced this pull request Oct 27, 2025
* upstream/main: (23 commits)
  ENH: Add on_missing for combine_channels (mne-tools#13463)
  Bump the actions group with 2 updates (mne-tools#13464)
  Move development dependencies into a dependency group (no more extra) (mne-tools#13452)
  ENH: add on_missing for rename_channels (mne-tools#13456)
  add advisory board to website (mne-tools#13462)
  ENH: Support Nihon Kohden EEG-1200A V01.00 (mne-tools#13448)
  MAINT: Update dependency specifiers (mne-tools#13459)
  ENH: Add encoding parameter to Nihon Kohden reader (mne-tools#13458)
  [MAINT] Automatic SPEC0 dependency version management (mne-tools#13451)
  FIX: Read Nihon Kohden annotation file accurately (mne-tools#13251)
  MAINT: Restore edfio git install (mne-tools#13421)
  Support preload=False for the new EEGLAB single .set format (mne-tools#13096)
  [pre-commit.ci] pre-commit autoupdate (mne-tools#13453)
  MAINT: Restore PySide6 6.10.0 testing (mne-tools#13446)
  MAINT: Auth [skip azp] [skip actions]
  MAINT: Deploy [circle deploy] [skip azp] [skip actions]
  Bump github/codeql-action from 3 to 4 in the actions group (mne-tools#13442)
  ENH: Dont constrain fiducial clicks to mesh vertices (mne-tools#13445)
  Use timezone-aware ISO 8601 for website timestamp (mne-tools#13347)
  [pre-commit.ci] pre-commit autoupdate (mne-tools#13443)
  ...
sseth pushed a commit to xannnimal/mne-python that referenced this pull request Mar 25, 2026
Co-authored-by: Eric Larson <larson.eric.d@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Nihon Kohden file (.LOG) annotations read incorrectly

2 participants