Sprint: Pytest conversion

[effigies]

Tasks

• Doctest modules
  • Enable --doctest-modules in pytest
    • @effigies
• Failing modules
  • nibabel.streamlines.tests.test_array_sequence
    • @robbisg: TEST: pytest conversion #864 #865 #870
  • nibabel.streamlines.tests.test_tck
  • nibabel.streamlines.tests.test_tractogram
    • @robbisg: TEST: pytest conversion #864 #865 #870
  • nibabel.streamlines.tests.test_trk
• Skipped modules (@djarecka: "I'll work on all of these")
  • nibabel.tests.test_scripts
  • nibabel.tests.test_spaces
  • nibabel.tests.test_spatialimages
  • nibabel.tests.test_spm2analyze
  • nibabel.tests.test_spm99analyze
  • nibabel.tests.test_testing
  • nibabel.tests.test_tmpdirs
  • nibabel.tests.test_trackvis
  • nibabel.tests.test_tripwire
  • nibabel.tests.test_viewers
  • nibabel.tests.test_volumeutils
• Uncollected tests
  • TEST: Refactor NetCDF tests to be more pytest friendly #879 @effigies
    • nibabel/externals/tests/test_netcdf.py:48: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_read_write_files will be ignored
    • nibabel/externals/tests/test_netcdf.py:89: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_read_write_sio will be ignored
    • nibabel/externals/tests/test_netcdf.py:140: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_write_invalid_dtype will be ignored
  • nibabel/tests/test_image_api.py:715: PytestCollectionWarning: cannot collect test class 'TestMinc2API' because it has a __init__ constructor
  • nibabel/tests/test_scripts.py:104: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_nib_ls will be ignored
  • nibabel/tests/test_scripts.py:187: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_nib_diff will be ignored
    • @djarecka
• Untouched submodules
  • nibabel.benchmarks
  • nibabel.cifti2.tests
    • @robbisg
  • nibabel.cmdline.tests
  • nibabel.externals.tests
  • nibabel.freesurfer.tests
    • @orduek
  • nibabel.gifti.tests
    • @anibalsolon
  • nibabel.nicom.tests
    • TEST: Convert nibabel.nicom to unittest/pytest #884 @effigies
• Refactors
  • Skip base test cases like _TestWrapStructBase.
    • Possible idea: use setUp() to test whether the class name starts with _, and pytest.skip().
    • @effigies: TEST: Add BaseTestCase class to skip TestCases starting with _ #871
  • Use pytest.deprecated_call() and pytest.warns() instead of clear_and_catch_warning, when sensible. See TEST: Drop nose for pytest #865 (comment) and TEST: Drop nose for pytest #865 (comment).
    • Related: assert_warn_len_equal is kind of a mess, and made redundant with going Python 3+.

Some principles:

• Try to avoid numpy test methods and decorators when a vanilla unittest/pytest construct exists. NOTABLE EXCEPTIONS: Tools for comparing arrays and floating points, such as assert_almost_equal.
• Clear is better than brief.
• ...

Original post follows:

---

Summary

nose is finally going to start failing on Python 3.9 due to some standard library API changes.

Python 3.9.0 is due to be released in October, but we might as well get out ahead of it. We don't need to rush, so I'm going to target the nibabel 3.1.0 release.

I put out a call on Twitter to see if anybody wanted to sprint on this, and got responses from @arokem, @chrisgorgo, @orduek, @robbisg and @anibalsolon (thanks to all for having the same handle on GitHub and Twitter).

This thread exists to coordinate the sprint.

Status

@djarecka and @chrisgorgo started on conversion in November:

• Ci/pytest nibabeltests #835
• adding testing_pytest that removed nose (it will be used temp. for te… #838
• converted test_volumeutils to pytest #839
• [wip] converting tests from nibabel/tests/ #840

There have been some changes in the tests since #840 was started, so it requires merging/rebasing.

Plans

To facilitate the transition, I've started the pytest branch, which is #840's commits, rebased on master.

The first thing to do is to identify all remaining tasks. Off the top of my head we need to:

1. Eliminate the use of nose.tools.
2. Move any nose test case classes to vanilla unittest features, which pytest should be able to handle transparently.
3. Verify coverage doesn't change/increases.
4. Run doctests in the documentation with pytest.

Logistics

Let's set at least one definite window that people want to be actively working on this and where I can commit to being more responsive to keep things from stalling out. We don't all need to align, but it would probably be best for me if there isn't a separate window for each participant.

Would it be useful to attempt to do this (semi-)synchronously, and use a chat room? Or would people rather go through the usual process of creating issues and pull requests?

[djarecka]

I can work on this one of the day of the week Feb 1st-7th, preferably Mon-Wed. I believe I converted all tests from nibabel/tests, now we have to check each subdirectory of the nibabel and change the tests inside

[anibalsolon]

@effigies thank you for organizing this!

Tuesdays and Thursdays work for me in general. The chat room idea is nice for quick checks, but the issue/PRs process might be useful to keep tracking.

[robbisg]

In general I'm quite available during the week!!

[effigies]

So would Tuesday, February 4 work for all three of you? Is there anybody that it wouldn't work for?

Also, does anybody have a Windows machine for debugging any issues there?

[orduek]

Tuesday would work for me, part from two hours (10am-12am EST). No windows machine here...

…

On Sun, Jan 26, 2020 at 11:35 AM Chris Markiewicz ***@***.***> wrote: So would Tuesday, February 4 work for all three of you? Is there anybody that it wouldn't work for? Also, does anybody have a Windows machine for debugging any issues there? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#864?email_source=notifications&email_token=AAPF3WJJU5ME34DY3RLVXHTQ7W3VZA5CNFSM4KLSCEIKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJ5YAYQ#issuecomment-578519138>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAPF3WIRGOUMV5A4UCAIBDDQ7W3VZANCNFSM4KLSCEIA> .

-- Or A. Duek, PhD Postdoctoral Associate Harpaz-Rotem Stress and PTSD Lab Yale University School of Medicine, Dept of Psychiatry https://medicine.yale.edu/lab/harpazrotem/ https://www.ptsdstresslab.org/ +1 (203) 305 7172 *Please consider the environment before printing this e-mail *

[anibalsolon]

Works for me, apart from the same two hours (unrelated tho)

I have a Windows at home, will setup the proper environment to test it.

[robbisg]

Works for me!! I am on Linux and avaliable considering GMT+2 ;)

[effigies]

Okay. I'll be around 9am-5pm EST (UTC-5), except a couple of calls from 12:45pm-2pm EST. I'll see if I can push them around, but we should have plenty of synchronous time, even if I can't.

How would we like to do live chatting? I've gone ahead and set up a Riot.im channel (#nibabel:matrix.org). If people prefer, the Brainhack Mattermost is another option. I'd rather not set up gitter...

[anibalsolon]

Riot.im seems nice! I am fine with it.

[yarikoptic]

FWIW we use riot and jitsi for DataLad. This I +1 on riot ;-)

[effigies]

Hi all, I just want to draw your attention to #865, which shows the diff of the pytest branch against master. You may want to have a look through to see what's already been done, and I've left comments about idioms we might want to try to work with, for consistency. Please feel free to comment with additional thoughts.

[djarecka]

and I will prepare description of my "workflow" for changing the tests, so people can use it if they don't have better ideas.