Skip to content

Feature: Support for file-like objects for EDF/BDF/GDF files #13156

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 67 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 59 commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
b89c5b4
Added support for file-like objects for EDF and BDF files
szz-dvl Mar 12, 2025
0a0ff52
Merge branch 'mne-tools:main' into main
szz-dvl Mar 12, 2025
a0a1e76
test: Added test descriptions
szz-dvl Mar 12, 2025
9e0991d
FEAT: Added support for GDF file-like objects
szz-dvl Mar 12, 2025
d16fc48
Merge branch 'mne-tools:main' into main
szz-dvl Mar 12, 2025
8d9d4f1
DOC: Documentation for EDF/GDF file-like objects
szz-dvl Mar 12, 2025
a9dfb58
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 12, 2025
5816f59
CHORE: Fix pre-commit
szz-dvl Mar 12, 2025
1097569
CHORE: Fix pre-commit
szz-dvl Mar 12, 2025
3c6a053
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 12, 2025
b64a768
CHORE: Fix pre-commit
szz-dvl Mar 12, 2025
90e03fc
FIX: Got rid of file_type extra parameter. RawEDF class duplicated ->…
szz-dvl Mar 13, 2025
a462488
FIX: Moved edf/gdf logic to a new _edf folder
szz-dvl Mar 13, 2025
f80722f
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 13, 2025
a5c9640
Merge branch 'main' into main
szz-dvl Mar 19, 2025
5b1dd29
FIX: Fixed log for GDF files when not file-like
szz-dvl Mar 20, 2025
08c81ec
DOC: Added documentation for file-like objects
szz-dvl Mar 20, 2025
95b5550
FEATURE: Added support for numpy.frombuffer for in memory file objects
szz-dvl Mar 20, 2025
89423c8
FIX: Removed unnecessary preloading from _edf/open.py
szz-dvl Mar 20, 2025
0372f3b
FIX: Support for in memory file-like objects
szz-dvl Mar 20, 2025
2df5210
TEST: Support for in memory file-like objects
szz-dvl Mar 20, 2025
ca2f3ba
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 20, 2025
930c802
CHORE: pre-commit script
szz-dvl Mar 20, 2025
f4b866c
CHORE: pre-commit script
szz-dvl Mar 20, 2025
29aa9c5
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 20, 2025
d7f8fc4
FIX: Fix GDF logs
szz-dvl Mar 21, 2025
9806bd0
TEST: Removed warnings from tests
szz-dvl Mar 21, 2025
ed3a8d0
TEST: Removed empty GDF file
szz-dvl Mar 22, 2025
381bf2f
Merge branch 'main' into main
szz-dvl Mar 22, 2025
f952695
[autofix.ci] apply automated fixes
autofix-ci[bot] Mar 31, 2025
9581dba
Merge branch 'main' into main
szz-dvl Apr 3, 2025
ccb897d
FIX: Moved numpy_fromfile to read_from_file_or_buffer.
szz-dvl Apr 3, 2025
b11b5b8
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 3, 2025
2698f53
TEST: Added @testing.requires_testing_data annotation.
szz-dvl Apr 3, 2025
87107da
TEST: Shortened tests for file-like objects for EDF/BDF files
szz-dvl Apr 3, 2025
1bbedd1
Merge branch 'main' of github.com:szz-dvl/mne-python
szz-dvl Apr 3, 2025
60a73a6
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 3, 2025
579a7d2
Merge branch 'main' into main
szz-dvl Apr 3, 2025
3cf32d5
DOC: Added rst for PR 13156
szz-dvl Apr 3, 2025
154f117
FIX: Added import numpy.typing in fixes.py
szz-dvl Apr 3, 2025
86c0e7d
FIX: Fixed .rst for PR 13156
szz-dvl Apr 3, 2025
29edf89
Added Santi Martínez to names.inc
szz-dvl Apr 4, 2025
9b47224
FIX: Removed extra underscore
szz-dvl Apr 4, 2025
84801a0
FIX: Removed extra declaration of _path_from_fname
szz-dvl Apr 4, 2025
82e83e1
FIX: Dedent test_degenerate
szz-dvl Apr 4, 2025
6ac9b01
Removed unnecessary ignore warning
szz-dvl Apr 4, 2025
63f79e5
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 4, 2025
5700760
FIX: pre-comit.sh
szz-dvl Apr 4, 2025
27d98c9
FIX: pre-comit.sh
szz-dvl Apr 4, 2025
6a766cb
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 4, 2025
42accea
Merge branch 'main' into main
szz-dvl Apr 8, 2025
25138c9
Merge branch 'main' into main
szz-dvl Apr 9, 2025
3655e1c
Merge branch 'main' into main
szz-dvl Apr 9, 2025
fdb2d60
Merge branch 'main' into main
szz-dvl Apr 10, 2025
12a88a4
Merge branch 'main' into main
szz-dvl Apr 11, 2025
d128c89
Merge branch 'main' into main
szz-dvl Apr 11, 2025
3488bc6
Merge branch 'main' into main
szz-dvl Apr 16, 2025
9d273d7
Merge branch 'main' into main
szz-dvl Apr 16, 2025
019ee0e
Merge branch 'main' into main
szz-dvl Apr 18, 2025
1cb64d0
DOC: Added version for file-like object support
szz-dvl Apr 18, 2025
48e6434
FIX: _check_fname to check input file names
szz-dvl Apr 18, 2025
bd64749
FIX: Changed return type annotation for read_raw_bdf
szz-dvl Apr 18, 2025
25d1c49
FIX: Added _check_args function to encapsulate initial argument check…
szz-dvl Apr 18, 2025
bee80e4
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 18, 2025
29ffbe3
Merge branch 'main' into main
szz-dvl Apr 22, 2025
4d37a5c
Merge branch 'main' into main
szz-dvl Apr 23, 2025
0d9bfa8
Merge branch 'main' into main
szz-dvl Apr 24, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/changes/devel/13156.newfeature.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Added support for file like objects in :func:`read_raw_bdf <mne.io.read_raw_bdf>`, :func:`read_raw_edf <mne.io.read_raw_edf>` and :func:`read_raw_gdf <mne.io.read_raw_gdf>`, by `Santi Martínez`_.
1 change: 1 addition & 0 deletions doc/changes/names.inc
Original file line number Diff line number Diff line change
Expand Up @@ -267,6 +267,7 @@
.. _Samuel Louviot: https://github.com/Sam54000
.. _Samuel Powell: https://github.com/samuelpowell
.. _Santeri Ruuskanen: https://github.com/ruuskas
.. _Santi Martínez: https://github.com/szz-dvl
.. _Sara Sommariva: https://github.com/sarasommariva
.. _Sawradip Saha: https://sawradip.github.io/
.. _Scott Huberty: https://orcid.org/0000-0003-2637-031X
Expand Down
23 changes: 23 additions & 0 deletions mne/_edf/open.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Authors: The MNE-Python contributors.
# License: BSD-3-Clause
# Copyright the MNE-Python contributors.

# Maybe we can move this one to utils or something like that.
from pathlib import Path

from mne._fiff.open import _NoCloseRead

from ..utils import _file_like, _validate_type, logger


def _gdf_edf_get_fid(fname, **kwargs):
"""Open a EDF/BDF/GDF file with no additional parsing."""
if _file_like(fname):
logger.debug("Using file-like I/O")
fid = _NoCloseRead(fname)
fid.seek(0)
else:
_validate_type(fname, [Path, str], "fname", extra="or file-like")
logger.debug("Using normal I/O")
fid = open(fname, "rb", **kwargs) # Open in binary mode
return fid
32 changes: 32 additions & 0 deletions mne/fixes.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,14 @@
# because this module is imported many places (but not always used)!

import inspect
import io
import operator as operator_module
import os
import warnings
from math import log

import numpy as np
import numpy.typing
from packaging.version import parse

###############################################################################
Expand Down Expand Up @@ -733,3 +735,33 @@ def sph_harm_y(n, m, theta, phi, *, diff_n=0):
return special.sph_harm_y(n, m, theta, phi, diff_n=diff_n)
else:
return special.sph_harm(m, n, phi, theta)


###############################################################################
# workaround: Numpy won't allow to read from file-like objects with numpy.fromfile,
# we try to use numpy.fromfile, if a blob is used we use numpy.frombuffer to read
# from the file-like object.
def read_from_file_or_buffer(
file: str | bytes | os.PathLike | io.IOBase,
dtype: numpy.typing.DTypeLike = float,
count: int = -1,
):
"""numpy.fromfile() wrapper, handling io.BytesIO file-like streams.

Numpy requires open files to be actual files on disk, i.e., must support
file.fileno(), so it fails with file-like streams such as io.BytesIO().

If numpy.fromfile() fails due to no file.fileno() support, this wrapper
reads the required bytes from file and redirects the call to
numpy.frombuffer().

See https://github.com/numpy/numpy/issues/2230#issuecomment-949795210
"""
try:
return np.fromfile(file, dtype=dtype, count=count)
except io.UnsupportedOperation as e:
if not (e.args and e.args[0] == "fileno" and isinstance(file, io.IOBase)):
raise # Nothing I can do about it
dtype = np.dtype(dtype)
buffer = file.read(dtype.itemsize * count)
return np.frombuffer(buffer, dtype=dtype, count=count)
Loading