EEG signals appears flat in the dashboard

[christian-oreilly]

If we run the code of the tutorial https://pylossless.readthedocs.io/en/latest/auto_examples/plot_10_run_pipeline.html and open the generated file with the dashboard, it looks like this:

As can be seen, the EEG signals look flat. This is probably due to the gain use for visualizing these signals not being in the adequate range.

[christian-oreilly]

@iofeii Following up on our exchange: Using

!pip install -q pylossless
!pip install -q openneuro-py

from pathlib import Path
import shutil
import pylossless as ll
import numpy as np
import mne

raw, config, bids_path = ll.datasets.load_openneuro_bids()

data = raw.get_data()
print(data.min(), data.max(), np.abs(data).min(), np.abs(data).max())

outputs

-0.028051713793439494 65536.0 0.0 65536.0

The abnormally large values are due to the fact that you are not getting the data only for EEG channels; they come from a "stimulus" channel which natively encodes UINT16. 65536.0 is the largest value in that range.

print(np.unique(raw.get_channel_types()))

array(['eeg', 'stim'], dtype='<U4')

You can update your analysis as

data = raw.get_data(picks="eeg")
print(data.min(), data.max(), np.abs(data).min(), np.abs(data).max())

and get

-0.028051713793439494 0.024348126884885243 0.00020185900109165913 0.028051713793439494

If you compare this to one of MNE's example, you would rather get:

ssvep_folder = mne.datasets.ssvep.data_path()
ssvep_data_raw_path = (
    ssvep_folder / "sub-02" / "ses-01" / "eeg" / "sub-02_ses-01_task-ssvep_eeg.vhdr"
)
ssvep_raw = mne.io.read_raw_brainvision(ssvep_data_raw_path, verbose=False)

data = ssvep_raw.get_data(picks="eeg")
print(data.min(), data.max(), np.abs(data).min(), np.abs(data).max())

-0.00027979999999999997 0.000405 0.0 0.000405

As you can see, these values are about 1000 times smaller. That means that the openneuro data are in ms, whereas MNE normally expect values in us. I will fix this in a PR using mne_bids.read_raw_bids(..., units={"eeg": "mV"}) in the ll.datasets.load_openneuro_bids function because the unit should be properly set. However, the display should still have adjust automatically since scalings is set to "auto":

pylossless/pylossless/dash/mne_visualizer.py

Line 42 in c985968

scalings="auto",

with

pylossless/pylossless/dash/mne_visualizer.py

Line 107 in c985968

self.scalings_arg = scalings

and

pylossless/pylossless/dash/mne_visualizer.py

Lines 202 to 220 in c985968

	self.scalings = dict(
	mag=1e-12,
	grad=4e-11,
	eeg=20e-6,
	eog=150e-6,
	ecg=5e-4,
	emg=1e-3,
	ref_meg=1e-12,
	misc=1e-3,
	stim=1,
	resp=1,
	chpi=1e-4,
	whitened=1e2,
	)
	if self.scalings_arg == "auto":
	for kind in np.unique(self.inst.get_channel_types()):
	self.scalings[kind] = np.percentile(self.inst.get_data(), 99.5)
	else:
	self.scalings.update(self.scalings_arg)

We should investigate why the scale does not properly adjust automatically as it should. The issue here may also be that the get_data needs to pick the right channel type. I'll check and PR this too if it solves the issue.

[scott-huberty]

mne_bids.read_raw_bids(..., units={"eeg": "ms"})

I'm confused. Does this have to do with the times, or the eeg voltage values? I would expect volts, microvolts, or (maybe) nanovolts, not milliseconds.

Can we convert to Volts upon reading since this is what MNE expects anyways?

e.g.

raw.apply_function(x: x*1e6, picks="eeg")

[christian-oreilly]

Yeah... I am confusing! It should have read "mV" not "ms". If fixed the comment.

By telling MNE what is the unit in the file ("mV") it will read it as "mV" and store it in the raw object as "V", as expected. I think it is cleaner to read it in the right unit than read it in the bad unit and apply a conversion after... but thanks for the info about apply_function. Did not know about this one and it would definitely have been useful at for various things I did in the past!

[scott-huberty]

think it is cleaner to read it in the right unit than read it in the bad unit and apply a conversion after...

Agreed!!

I will fix this in a PR using mne_bids.read_raw_bids(..., units={"eeg": "mV"})

BTW does this work? I don't see units listed as a parameter in the docstring of mne_bids.read_raw_bids nor mne.io.read_raw_branvision (Or maybe they need to document this).

[christian-oreilly]

It could be passed to extra_params for mne.io.read_raw_bids but, actually, it does not seem to be necessary. Now that the problem with the viewer is fixed, the signals display fine

So, I am not totally sure what happens with this dataset... but for now, the original problem was fixed by adding the pick argument to select the specific channel types we are processing

pylossless/pylossless/dash/mne_visualizer.py

Lines 216 to 218 in c985968

	if self.scalings_arg == "auto":
	for kind in np.unique(self.inst.get_channel_types()):
	self.scalings[kind] = np.percentile(self.inst.get_data(), 99.5)

and I cannot see any scaling issue with the openneuro data through the Dash so I won't try to fix what might not be broken.