Open
Description
Summary
I would like to remove (almost) all tests in tests/datamodules and replace them with new tests in tests/trainers that actually ensure our datamodules and trainers are compatible.
Rationale
The current tests simply ensure that the data loaders don't crash. They don't actually test that the datamodules are compatible with our trainers.
Implementation
The bulk of these have already been converted in #329. The remaining tests are:
- chesapeake (tests invalid arguments)
- digital typhoon (tests split method)
- fair1m (requires rotated ObjectDetectionTask trainer: Rotated Object Detection #840)
- inria (InriaAerialImageLabelingDataModule: fix predict dimensions #975)
- loveda (Add LoveDADataModule to the trainer tests #966)
- nasa_marine_debris (Remove tests/datamodules/test_nasa_marine_debris.py #979)
- oscd (requires ChangeDetectionTask trainer)
- potsdam (Add crop logic to Potsdam2D datamodule #929)
- usavars (requires multi-label RegressionTask trainer)
- vaihingen (Fix Vaihingen datamodule #853)
- xview2 (requires ChangeDetectionTask trainer)
- utils (coverage for test_pct == None, do we need this?)
Alternatives
We may end up keeping some of these that test invalid arguments. The important thing is not that we don't test datamodules standalone, but that we test them with trainers whenever possible.
Additional information
No response