Make the NNlib tests more robust#152
Conversation
|
Locally I see: (NNlib) pkg> test NNlib
Testing NNlib
Resolving package versions...
Test Summary: | Pass Total
Activation Functions | 257 257
[ Info: Skipping Convolutional fuzzing tests, set NNLIB_TEST_FUZZING=true to run them
∇depthwiseconv_filter_direct/∇depthwiseconv_data_direct: Test Failed at C:\Users\accou\.julia\dev\NNlib\test\conv.jl:540
Expression: ddims(∇conv_filter(x, dy, cdims)) == dw_pad
Evaluated: [NaN, 150.0] == [135.0, 150.0]
Stacktrace:
[1] top-level scope at C:\Users\accou\.julia\dev\NNlib\test\conv.jl:540
[2] top-level scope at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Test\src\Test.jl:1107
[3] top-level scope at C:\Users\accou\.julia\dev\NNlib\test\conv.jl:513
[4] top-level scope at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Test\src\Test.jl:1107
[5] top-level scope at C:\Users\accou\.julia\dev\NNlib\test\conv.jl:449
[6] top-level scope at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Test\src\Test.jl:1107
[7] top-level scope at C:\Users\accou\.julia\dev\NNlib\test\conv.jl:446
[ Info: Skipping Depthwise Convolutional fuzzing tests, set NNLIB_TEST_FUZZING=true to run them
58.176720 seconds (143.73 M allocations: 7.287 GiB, 3.04% gc time)
Testing NNlib tests passedThe testsets in the conv part seem to be nested in such a way that when they fail they don't trigger a failure? |
|
The current test failure traces back to: https://travis-ci.org/FluxML/NNlib.jl/jobs/619051416#L203-L238 meaning that #142 is the culprit. Before that, the library had other hidden test failures: https://travis-ci.org/FluxML/NNlib.jl/jobs/612356942#L207-L259 So I think the current issues may trace back to that PR? @staticfloat |
|
The downstream tests reproduce the expected failures: https://travis-ci.org/FluxML/NNlib.jl/jobs/628608690#L488-L517 |
|
Also, this sets Travis to |
|
I think testing should happen on multiple Julia versions regardless. Adding testing on 1.1 forces us to not break code on earlier versions too |
|
Added tests for older versions |
The NNlib tests were found to not be catching many issues because they committed a few sins:
This should now correctly give red tests.