[ENH]: New pyqtgraph-backend for 2D-Data-Browser (#9687)

* start abstraction of Browser-Classes

* outline of data-management-class

* wip refactoring MNEBrowserFigure

* refactored Browser-Initialization into MNEDataBrowser

* fix some style-issues and remove remnants

* start with integration of pyqtgraph-prototype

* make pyqtgraph optional (just for development, can be removed again for final PR)

* more refactoring of mpl-methods into BrowserBase

* fixes for failing tests

* add docstrings in _browser.py

* move _update_projector-call into BrowserBase

* add use/set/get for browser

* rename 2D to Browser to avoid confusion

* move inheritance of BrowserBaser from MNEFigure to MNEBrowseFigure

* refactored _annotation_helper from test_raw.py

* fix flake

* move base-classes/funcs from _browser.py back to _figure.py to facilitate review

* update from upstream

* remove set/get/use-browser from __init__.py again to prevent make docstring from failing

* add show_browser and other adjustments for pyqtgraph

* refactoring _close into BrowserBase

* add block for pyqtgraph

* [revert for PR] adjust plot-function to accept kwargs from benchmark

* fix annotation-key still working when plotting epochs

* add docstring to show_browser

* refactor midpoints into BrowserBase for epochs

* add show to show_broser

* add pyqtgraph to browse_backend-fixture

* fix block not supported in Figure.show()

* WIP adapt test_raw.test_plot_raw_traces

* adapt test_plot_raw_traces to make it work for matplotlib and pyqtgraph

* refactor _redraw

* refactor _update_data

* change bad-color from rgb to hex

* pyqtgraph always blocks execution

* reinsert block for pyqtgraph

* make _redraw not abstract anymore (not needed for pyqtgraph)

* add pyqtgraph-backend

* fix butterfly-bug showing channels still in old y-position [ci skip]

* rebase on main [ci skip]

* Set usage of OpenGL to false by default

* organize keyboard-shortcuts [ci skip]

* clarify index-system for traces and set z-values for traces and annotations [ci skip]

* add exception-hook from pytest-qt [ci skip]

* actually raise exceptions from qt [ci skip]

* fix bugs annotations (removing decription/select visible) [ci skip]

* reorganize imports [ci skip]

* fix bug _update_regions_colors [ci skip]

* add '=' to keyboard-shortcuts and make scale-steps smaller [ci skip]

* import pg-backend from separate repo[ci skip]

* remove pg-backend from PR[ci skip]

* update repo-link to mne-tools[ci skip]

* Update mne/viz/_figure.py

Co-authored-by: Alexandre Gramfort <[email protected]>

* remove _pg_figure from _backends[ci skip]

* add browser-backend-functions to documentation[ci skip]

* avoid codespell-failure [ci skip]

* remove pyqtgraph from tests (tests will be run in mne-qt-browser for now)

* update parameters for Raw.plot()

* fix flake

* fix pip install link

* fix table in set_browser_backend

* fix butterfly showing always all channels despite of selection

* adjust test_scale_bar for pyqtgraph [ci skip]

* refactor channel context figs [ci skip]

* change to relative imports [ci skip]

* refactor _new_child_figure [ci skip]

* update some key-presses [ci skip]

* adapt test_plot_raw_ssp_interaction to pyqtgraph [ci skip]

* adapt test_plot_raw_child_figures to pyqtgraph [ci skip]

* change default of event_lines to list [ci skip]

* add drag to _fake_click [ci skip]

* adapt test_annotations to pyqtgraph [ci skip]

* adapt test_clock_xticks to pyqtgraph [ci skip]

* remove install-question [ci skip]

* adapt pyqtgraph-backend to test_plot_raw_selection [ci skip]

* adapt pyqtgraph-backend to test_min_window_size [ci skip]

* adapt pyqtgraph-backend to test_plot_raw_groupby [ci skip]

* fix multiple tests [ci skip]

* adapt annotation-test for pyqtgraph [ci skip]

* fix more test-issues [ci skip]

* add pyqtgraph to test_raw-suite [ci skip]

* fix flake

* update feature-grid

* update from main branch

* fix checkbox-click issue (inconsistent across OS)

* fix test_plot_raw_ssp_interaction for linux

* update from upstream2

* fix flake

* simplify ssp_interaction

* fix test_min_window_size for Windows-CI

* fix flake

* implement review-feedback #1

* fix _proj_click_all for inconsistent fake-click-behaviour

* clarify docs for preload

* fix flake

* [Refactor]: browser_backend-fixture for consistency

* DOC: Add doc comments for block

* add speed test

* rename preload to precompute

* fix unused import

* remove unnecessary parameters

* remove speed-test

* remove automatic installation of mne-qt-browser

* revert removal of automatic installation until mne-qt-browser is uploaded to PyPi

* FIX: Route through call

* add mne-qt-browser to requirements.txt

* update latest.inc

* fix flake

* fix latest.inc

* update test-dependencies

* specify docs regarding block-behaviour

* fix typo in github_actions_dependencies.sh

* fix docstring for plot_raw

* fix docstring for plot_raw again

* fix docstring for plot_raw

* add mne-qt-browser to azure_dependencies.sh

* add mne-qt-browser to environment.yml

* specify latest.inc

* specifiy use_opengl documentation

* Update mne/viz/_figure.py

Co-authored-by: Eric Larson <[email protected]>

* FIX: Fix test [skip azp] [skip circle]

* FIX: Path [skip azp] [skip circle]

* revert color-name-changes

* MAINT: Test only on one run

* FIX: Remove

* MNT: Add to mne sys_info

* DOC: Fix doc build

* FIX: Better

* DOC: Add link to mne-qt-browser issues in docs

* Update mne/utils/docs.py

Co-authored-by: Daniel McCloy <[email protected]>

* Update mne/utils/docs.py

Co-authored-by: Daniel McCloy <[email protected]>

* Update mne/viz/raw.py

Co-authored-by: Daniel McCloy <[email protected]>

* FIX: fix flake

* FIX: Test

* FIX: Correct check

* STY: Flake

* FIX: One more mark

Co-authored-by: Alexandre Gramfort <[email protected]>
Co-authored-by: Eric Larson <[email protected]>
Co-authored-by: Daniel McCloy <[email protected]>

Author: 4 people

.github/workflows/compat_minimal.yml

@@ -55,7 +55,7 @@ jobs:
         run: ./tools/get_testing_version.sh
         name: 'Get testing version'
       - shell: bash -el {0}
-        run: MNE_SKIP_TESTING_DATASET_TESTS=true pytest -m "not ultraslowtest" --tb=short --cov=mne --cov-report xml -vv -rfE mne/
+        run: MNE_SKIP_TESTING_DATASET_TESTS=true pytest -m "not (ultraslowtest or pgtest)" --tb=short --cov=mne --cov-report xml -vv -rfE mne/
         name: Run tests with no testing data
       - uses: actions/cache@v2
         with:

azure-pipelines.yml

@@ -81,7 +81,7 @@ stages:
   variables:
     AZURE_CI: 'true'
   jobs:
-  - job: Ultraslow
+  - job: Ultraslow+PG
     pool:
       vmImage: 'ubuntu-20.04'
     variables:
@@ -111,6 +111,8 @@ stages:
     - bash: |
         set -e
         python -m pip install --progress-bar off --upgrade pip setuptools wheel codecov
+        python -m pip install --progress-bar off mne-qt-browser
+        python -m pip uninstall -yq mne
         python -m pip install --progress-bar off --upgrade -e .[test]
       displayName: 'Install dependencies with pip'
     - script: mne sys_info -pd
@@ -124,8 +126,8 @@ stages:
       displayName: 'Cache testing data'
     - script: python -c "import mne; mne.datasets.testing.data_path(verbose=True)"
       displayName: 'Get test data'
-    - script: pytest -m "ultraslowtest" --tb=short --cov=mne --cov-report=xml --cov-report=html -vv mne
-      displayName: 'Run ultraslow tests'
+    - script: pytest -m "ultraslowtest or pgtest" --tb=short --cov=mne --cov-report=xml --cov-report=html -vv mne
+      displayName: 'Run ultraslow and PyQtGraph mne-qt-browser tests'
     - bash: bash <(curl -s https://codecov.io/bash)
       displayName: 'Codecov'
       condition: succeededOrFailed()
@@ -169,7 +171,7 @@ stages:
       displayName: 'Cache testing data'
     - script: python -c "import mne; mne.datasets.testing.data_path(verbose=True)"
       displayName: 'Get test data'
-    - script: pytest --tb=short --cov=mne --cov-report=xml --cov-report=html -vv mne/viz
+    - script: pytest --tb=short -m "not pgtest" --cov=mne --cov-report=xml --cov-report=html -vv mne/viz
       displayName: 'Run viz tests'
     - bash: bash <(curl -s https://codecov.io/bash)
       displayName: 'Codecov'
@@ -264,7 +266,7 @@ stages:
       displayName: 'Cache testing data'
     - script: python -c "import mne; mne.datasets.testing.data_path(verbose=True)"
       displayName: 'Get test data'
-    - script: pytest -m "not slowtest" --tb=short --cov=mne --cov-report=xml --cov-report=html -vv mne
+    - script: pytest -m "not (slowtest or pgtest)" --tb=short --cov=mne --cov-report=xml --cov-report=html -vv mne
       displayName: 'Run tests'
     - bash: bash <(curl -s https://codecov.io/bash)
       displayName: 'Codecov'

doc/changes/latest.inc

@@ -211,10 +211,14 @@ Enhancements
 
 - Add support for colormap normalization in :meth:`mne.time_frequency.AverageTFR.plot` (:gh:`9851` by `Clemens Brunner`_)
 
-- Add support for BIDS-compatible filenames when splitting big epochs files via the new ``split_naming`` parameter in :meth:`mne.Epochs.save` (:gh:9869 by `Denis Engemann`_)
+- Add support for BIDS-compatible filenames when splitting big epochs files via the new ``split_naming`` parameter in :meth:`mne.Epochs.save` (:gh:`9869` by `Denis Engemann`_)
 
 - Add ``by_event_type`` parameter to :meth:`mne.Epochs.average` to create a list containing an :class:`mne.Evoked` object for each event type (:gh:`9859` by `Marijn van Vliet`_)
 
+- Add pyqtgraph as a new backend for :meth:`mne.io.Raw.plot` (:gh:`9687` by `Martin Schulz`_)
+
+- Add :func:`mne.viz.set_browser_backend`, :func:`mne.viz.use_browser_backend` and :func:`mne.viz.get_browser_backend` to set matplotlib or pyqtgraph as backend for :meth:`mne.io.Raw.plot` (:gh:`9687` by `Martin Schulz`_)
+
 Bugs
 ~~~~
 - Fix bug in :meth:`mne.io.Raw.pick` and related functions when parameter list contains channels which are not in info instance (:gh:`9708` **by new contributor** |Evgeny Goldstein|_)

doc/cited.rst

@@ -3,7 +3,7 @@
 Papers citing MNE-Python
 ========================
 
-Estimates provided by Google Scholar as of 27 January 2021:
+Estimates provided by Google Scholar as of 02 November 2021:
 
-- `MNE (908) <https://scholar.google.com/scholar?cites=12188330066413208874&as_ylo=2014>`_
-- `MNE-Python (771) <https://scholar.google.com/scholar?cites=1521584321377182930&as_ylo=2013>`_
+- `MNE (1100) <https://scholar.google.com/scholar?cites=12188330066413208874&as_ylo=2014>`_
+- `MNE-Python (1060) <https://scholar.google.com/scholar?cites=1521584321377182930&as_ylo=2013>`_

doc/visualization.rst

@@ -79,3 +79,6 @@ Visualization
    close_3d_figure
    close_all_3d_figures
    get_brain_class
+   set_browser_backend
+   get_browser_backend
+   use_browser_backend

environment.yml

@@ -42,3 +42,4 @@ dependencies:
 - pooch
 - pip:
   - ipyvtklink
+  - mne-qt-browser

mne/commands/mne_browse_raw.py

@@ -21,7 +21,6 @@
 
 def run():
     """Run command."""
-    import matplotlib.pyplot as plt
     from mne.commands.utils import get_optparser, _add_verbose_flag
     from mne.viz import _RAW_CLIP_DEF
 
@@ -135,8 +134,8 @@ def run():
     raw.plot(duration=duration, start=start, n_channels=n_channels,
              group_by=group_by, show_options=show_options, events=events,
              highpass=highpass, lowpass=lowpass, filtorder=filtorder,
-             clipping=clipping, proj=not proj_off, verbose=verbose)
-    plt.show(block=True)
+             clipping=clipping, proj=not proj_off, verbose=verbose,
+             show=True, block=True)
 
 
 mne.utils.run_command_if_main()

mne/conftest.py

@@ -25,7 +25,8 @@
 from mne.fixes import has_numba
 from mne.io import read_raw_fif, read_raw_ctf
 from mne.stats import cluster_level
-from mne.utils import _pl, _assert_no_instances, numerics, Bunch
+from mne.utils import (_pl, _assert_no_instances, numerics, Bunch,
+                       _check_pyqt5_version)
 
 # data from sample dataset
 from mne.viz._figure import use_browser_backend
@@ -61,7 +62,7 @@
 def pytest_configure(config):
     """Configure pytest options."""
     # Markers
-    for marker in ('slowtest', 'ultraslowtest'):
+    for marker in ('slowtest', 'ultraslowtest', 'pgtest'):
         config.addinivalue_line('markers', marker)
 
     # Fixtures
@@ -397,11 +398,52 @@ def garbage_collect():
     gc.collect()
 
 
-@pytest.fixture(params=['matplotlib'])
-def browse_backend(request, garbage_collect):
+@pytest.fixture
+def mpl_backend(garbage_collect):
+    """Use for epochs/ica when not implemented with pyqtgraph yet."""
+    with use_browser_backend('matplotlib') as backend:
+        yield backend
+        backend._close_all()
+
+
+def _check_pyqtgraph():
+    try:
+        import PyQt5  # noqa: F401
+    except ModuleNotFoundError:
+        pytest.skip('PyQt5 is not installed but needed for pyqtgraph!')
+    try:
+        assert LooseVersion(_check_pyqt5_version()) >= LooseVersion('5.12')
+    except AssertionError:
+        pytest.skip(f'PyQt5 has version {_check_pyqt5_version()}'
+                    f'but pyqtgraph needs >= 5.12!')
+    try:
+        import mne_qt_browser  # noqa: F401
+    except Exception:
+        pytest.skip('Requires mne_qt_browser')
+
+
+@pytest.mark.pgtest
+@pytest.fixture
+def pg_backend(garbage_collect):
+    """Use for pyqtgraph-specific test-functions."""
+    _check_pyqtgraph()
+    with use_browser_backend('pyqtgraph') as backend:
+        yield backend
+        backend._close_all()
+
+
+@pytest.fixture(params=[
+    'matplotlib',
+    pytest.param('pyqtgraph', marks=pytest.mark.pgtest),
+])
+def browser_backend(request, garbage_collect):
     """Parametrizes the name of the browser backend."""
-    with use_browser_backend(request.param) as backend:
+    backend_name = request.param
+    if backend_name == 'pyqtgraph':
+        _check_pyqtgraph()
+    with use_browser_backend(backend_name) as backend:
         yield backend
+        backend._close_all()
 
 
 @pytest.fixture(params=["mayavi", "pyvistaqt"])
@@ -446,7 +488,7 @@ def renderer_interactive(request):
         if renderer._get_3d_backend() == 'mayavi':
             with warnings.catch_warnings(record=True):
                 try:
-                    from surfer import Brain  # noqa: 401 analysis:ignore
+                    from surfer import Brain  # noqa: F401 analysis:ignore
                 except Exception:
                     pytest.skip('Requires PySurfer')
         yield renderer

mne/io/base.py

@@ -1517,22 +1517,23 @@ def _tmin_tmax_to_start_stop(self, tmin, tmax):
 
     @copy_function_doc_to_method_doc(plot_raw)
     def plot(self, events=None, duration=10.0, start=0.0, n_channels=20,
-             bgcolor='w', color=None, bad_color=(0.8, 0.8, 0.8),
+             bgcolor='w', color=None, bad_color='lightgray',
              event_color='cyan', scalings=None, remove_dc=True, order=None,
              show_options=False, title=None, show=True, block=False,
              highpass=None, lowpass=None, filtorder=4, clipping=_RAW_CLIP_DEF,
              show_first_samp=False, proj=True, group_by='type',
              butterfly=False, decim='auto', noise_cov=None, event_id=None,
              show_scrollbars=True, show_scalebars=True, time_format='float',
-             verbose=None):
+             precompute='auto', use_opengl=True, verbose=None):
         return plot_raw(self, events, duration, start, n_channels, bgcolor,
                         color, bad_color, event_color, scalings, remove_dc,
                         order, show_options, title, show, block, highpass,
                         lowpass, filtorder, clipping, show_first_samp,
                         proj, group_by, butterfly, decim, noise_cov=noise_cov,
                         event_id=event_id, show_scrollbars=show_scrollbars,
-                        show_scalebars=show_scalebars,
-                        time_format=time_format, verbose=verbose)
+                        show_scalebars=show_scalebars, time_format=time_format,
+                        precompute=precompute, use_opengl=use_opengl,
+                        verbose=verbose)
 
     @verbose
     @copy_function_doc_to_method_doc(plot_raw_psd)

mne/utils/config.py

@@ -512,7 +512,7 @@ def sys_info(fid=None, show_paths=False, *, dependencies='user'):
     """  # noqa: E501
     _validate_type(dependencies, str)
     _check_option('dependencies', dependencies, ('user', 'developer'))
-    ljust = 21 if dependencies == 'developer' else 15
+    ljust = 21 if dependencies == 'developer' else 16
     platform_str = platform.platform()
     if platform.system() == 'Darwin' and sys.version_info[:2] < (3, 8):
         # platform.platform() in Python < 3.8 doesn't call
@@ -548,7 +548,7 @@ def sys_info(fid=None, show_paths=False, *, dependencies='user'):
     use_mod_names = ('mne', 'numpy', 'scipy', 'matplotlib', '', 'sklearn',
                      'numba', 'nibabel', 'nilearn', 'dipy', 'cupy', 'pandas',
                      'mayavi', 'pyvista', 'pyvistaqt', 'ipyvtklink', 'vtk',
-                     'PyQt5', 'ipympl')
+                     'PyQt5', 'ipympl', 'mne_qt_browser')
     if dependencies == 'developer':
         use_mod_names += (
             '', 'sphinx', 'sphinx_gallery', 'numpydoc', 'pydata_sphinx_theme',