Skip to content

fix: raise a clear error when ValidSplit gets an IterableDataset - #1151

Merged
BenjaminBossan merged 3 commits into
skorch-dev:masterfrom
aiedwardyi:fix/iterabledataset-train-split
Aug 10, 2026
Merged

fix: raise a clear error when ValidSplit gets an IterableDataset#1151
BenjaminBossan merged 3 commits into
skorch-dev:masterfrom
aiedwardyi:fix/iterabledataset-train-split

Conversation

@aiedwardyi

@aiedwardyi aiedwardyi commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Closes #594.

ValidSplit needs len() and Subset, so an IterableDataset currently dies with an opaque TypeError about a missing length. It now raises a ValueError naming the cause and pointing at train_split=None.

Per the request on the issue that the error only be raised if really needed, the check sits in ValidSplit.__call__ rather than in NeuralNet - a custom train_split that tolerates an IterableDataset keeps working, and there is a test covering that.

Review fixes applied

  • 961a68a: intercept the TypeError from get_len instead of checking isinstance up front, so an IterableDataset subclass that defines __len__ still splits normally; any other error is reraised untouched. Added a test for that case.
  • 729cdcf: chain the original TypeError (from exc instead of from None) so the traceback keeps it; test asserts __cause__.

@BenjaminBossan BenjaminBossan left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR. I made a suggestion, please check.

Comment thread skorch/dataset.py Outdated

@BenjaminBossan BenjaminBossan left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update, just one question.

Comment thread skorch/dataset.py Outdated

@BenjaminBossan BenjaminBossan left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update, LGTM.

@BenjaminBossan
BenjaminBossan merged commit 08680a8 into skorch-dev:master Aug 10, 2026
16 checks passed
@aiedwardyi
aiedwardyi deleted the fix/iterabledataset-train-split branch August 10, 2026 15:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

torch IterableDataset are not fully supported

2 participants