Skip to content

cross_val_multiscore can crash with too few epochs #1265

Description

@larsoner

I have a dataset with only 14 epochs and it crashes when calling cross_val_multiscore in decoding_time_by_time because there aren't enough epochs of each type:

(Pdb) w
...
> /opt/mne-bids-pipeline/mne_bids_pipeline/steps/sensor/_03_decoding_time_by_time.py(201)run_time_decoding()
-> scores = cross_val_multiscore(
  <decorator-gen-558>(10)cross_val_multiscore()
  /opt/mne-python/mne/decoding/base.py(880)cross_val_multiscore()
-> scores = parallel(
  /opt/mne-python/mne/decoding/base.py(881)<genexpr>()
-> p_func(
  <decorator-gen-559>(10)_fit_and_score()
  /opt/mne-python/mne/decoding/base.py(967)_fit_and_score()
-> test_score = _score(estimator, X_test, y_test, scorer)
  /opt/mne-python/mne/decoding/base.py(992)_score()
-> score = scorer(estimator, X_test, y_test)
  /opt/.../sklearn/metrics/_scorer.py(472)__call__()
-> return estimator.score(*args, **kwargs)
  /opt/mne-python/mne/decoding/search_light.py(333)score()
-> y = _fix_auc(scoring, y)
  /opt/mne-python/mne/decoding/search_light.py(771)_fix_auc()
-> raise ValueError(
(Pdb) p y
array([1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])
  1. Should figure out why this step doesn't fail for the previous decoding step.
  2. Should do something reasonable here -- probably set the scores to nan
  3. Ensure whatever we do in (2) is handled properly by the average task

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions