Upgrade requirements and generate test files workflow (#1550)

Co-authored-by: Oliver Ruebel <[email protected]>

Author: rly

.github/ISSUE_TEMPLATE/bug_report.yml

@@ -60,10 +60,10 @@ body:
     attributes:
       label: Python Version
       options:
-        - 3.7
-        - 3.8
-        - 3.9
-        - 3.10
+        - "3.7"
+        - "3.8"
+        - "3.9"
+        - "3.10"
     validations:
       required: true
   - type: textarea

.github/workflows/generate_test_files.yml

@@ -39,4 +39,4 @@ jobs:
         with:
           name: test-files-${{ matrix.name }}
           path: |
-            tests/back_compat/*.nwb
+            src/pynwb/testing/*.nwb

.gitignore

@@ -69,5 +69,8 @@ tests/coverage/htmlcov
 # vscode
 .vscode/
 
-#mypy
+# mypy
 .mypy_cache/
+
+# macos
+.DS_Store

CHANGELOG.md

@@ -1,27 +1,39 @@
 # PyNWB Changelog
 
-## PyNWB 2.1.1 (Upcoming)
+## PyNWB 2.1.1 (September 1, 2022)
 
 ### Documentation and tutorial enhancements:
-- Support explicit ordering of sphinx gallery tutorials in the docs. @oruebel (#1504), @bdichter (#1495)
-- Add developer guide on how to create a new tutorial. @oruebel (#1504)
-- Add images tutorial. @weiglszonja (#1470)
-- Add example code for s3fs in the streaming tutorial. @bdichter (#1499)
+- Added support for explicit ordering of sphinx gallery tutorials in the docs. @oruebel 
+  [#1504](https://github.com/NeurodataWithoutBorders/pynwb/pull/1504), @bdichter
+  [#1495](https://github.com/NeurodataWithoutBorders/pynwb/pull/1495)
+- Added developer guide on how to create a new tutorial. @oruebel
+  [#1504](https://github.com/NeurodataWithoutBorders/pynwb/pull/1504)
+- Added images tutorial. @weiglszonja
+  [#1470](https://github.com/NeurodataWithoutBorders/pynwb/pull/1470)
+- Added example code for fsspec in the streaming tutorial. @bdichter
+  [#1499](https://github.com/NeurodataWithoutBorders/pynwb/pull/1499)
 
 ### Enhancements and minor changes
-- Update coverage workflow, report separate unit vs integration coverage. @rly (#1509)
-- Delete test files generated from running sphinx gallery examples. @rly (#1517)
-- Enable passing an S3File created through s3fs, which provides a method for reading an NWB file directly
+- Updated coverage workflow, report separate unit vs integration coverage. @rly
+  [#1509](https://github.com/NeurodataWithoutBorders/pynwb/pull/1509)
+- Deleted test files generated from running sphinx gallery examples. @rly
+  [#1517](https://github.com/NeurodataWithoutBorders/pynwb/pull/1517)
+- Enabled passing an S3File created through s3fs, which provides a method for reading an NWB file directly
   from s3 that is an alternative to ros3. This required relaxing of `NWBHDF5IO` input validation. The `path`
   arg is not needed if `file` is provided. `mode` now has a default value of "r".
-  @bendichter (#1499)
+  @bendichter
+  [#1499](https://github.com/NeurodataWithoutBorders/pynwb/pull/1499)
 - Added a method to `NWBMixin` that only raises an error when a check is violated on instance creation,
   otherwise throws a warning when reading from a file. The new checks in `ImageSeries` when `external_file`
   is provided is used with this method to ensure that that files with invalid data can be read, but prohibits
-  the user from creating new instances when these checks are violated. @weiglszonja (#1516)
+  the user from creating new instances when these checks are violated. @weiglszonja
+  [#1516](https://github.com/NeurodataWithoutBorders/pynwb/pull/1516)
 - Created a GitHub Actions workflow to generate test files for testing backward compatibility. @rly
   [#1548](https://github.com/NeurodataWithoutBorders/pynwb/pull/1548)
-- Enhanced docs for ``LabMetaData`` to clarify its usage @oruebel [#1546](https://github.com/NeurodataWithoutBorders/pynwb/pull/1546)
+- Updated requirements, including allowing numpy 1.23. @rly
+  [#1550](https://github.com/NeurodataWithoutBorders/pynwb/pull/1550)
+- Enhanced docs for ``LabMetaData`` to clarify its usage. @oruebel 
+  [#1546](https://github.com/NeurodataWithoutBorders/pynwb/pull/1546)
 
 ## PyNWB 2.1.0 (July 6, 2022)
 

requirements-dev.txt

@@ -1,11 +1,11 @@
 # pinned dependencies to reproduce an entire development environment to use PyNWB, run PyNWB tests, check code style,
 # compute coverage, and create test environments
 codecov==2.1.12
-coverage==6.3.2
-flake8==4.0.1
+coverage==6.4.2
+flake8==5.0.4
 flake8-debugger==4.1.2
 flake8-print==5.0.0
 importlib-metadata==4.2.0
 pytest==7.1.2
 pytest-cov==3.0.0
-tox==3.25.0
+tox==3.25.1

requirements.txt

@@ -1,7 +1,7 @@
 # pinned dependencies to reproduce an entire development environment to use PyNWB
-h5py==3.6.0
-hdmf==3.4.0
+h5py==3.7.0
+hdmf==3.4.2
 numpy==1.21.5  # note that numpy 1.22 dropped python 3.7 support
 pandas==1.3.5  # note that pandas 1.4 dropped python 3.7 support
 python-dateutil==2.8.2
-setuptools==62.2.0
+setuptools==63.4.1

setup.cfg

@@ -25,16 +25,16 @@ exclude =
   src/pynwb/_version.py
   src/pynwb/_due.py
 per-file-ignores =
-  docs/gallery/*:E402,T001
-  docs/source/tutorials/*:E402,T001
+  docs/gallery/*:E402,E501,T201
+  docs/source/tutorials/*:E402,T201
   src/pynwb/io/__init__.py:F401
   src/pynwb/legacy/io/__init__.py:F401
   tests/integration/__init__.py:F401
   src/pynwb/testing/__init__.py:F401
-  src/pynwb/validate.py:T001
-  setup.py:T001
-  test.py:T001
-  scripts/*:T001
+  src/pynwb/validate.py:T201
+  setup.py:T201
+  test.py:T201
+  scripts/*:T201
 extend-ignore = E203
 
 [metadata]

setup.py

@@ -20,8 +20,8 @@
 
 reqs = [
     'h5py>=2.10,<4',
-    'hdmf>=3.3.2,<4',
-    'numpy>=1.16,<1.23',
+    'hdmf>=3.4.2,<4',
+    'numpy>=1.16,<1.24',
     'pandas>=1.0.5,<2',
     'python-dateutil>=2.7,<3',
     'setuptools'

src/pynwb/testing/make_test_files.py

@@ -1,4 +1,5 @@
 import numpy as np
+from pathlib import Path
 
 from datetime import datetime
 from pynwb import NWBFile, NWBHDF5IO, __version__, TimeSeries
@@ -11,8 +12,7 @@
 
 
 def _write(test_name, nwbfile):
-    filename = 'tests/back_compat/%s_%s.nwb' % (__version__, test_name)
-
+    filename = str("%s/%s_%s.nwb") % (Path(__file__).parent, __version__, test_name)
     with NWBHDF5IO(filename, 'w') as io:
         io.write(nwbfile)
 
@@ -83,7 +83,7 @@ def _make_imageseries_no_data():
     image_series = ImageSeries(
         name='test_imageseries',
         external_file=['external_file'],
-        starting_frame=[1, 2, 3],
+        starting_frame=[1],
         format='external',
         timestamps=[1., 2., 3.]
     )
@@ -152,6 +152,8 @@ def _make_imageseries_nonmatch_starting_frame():
 if __name__ == '__main__':
     # install these versions of PyNWB and run this script to generate new files
     # python src/pynwb/testing/make_test_files.py
+    # files will be made in src/pynwb/testing/
+    # files should be moved to tests/back_compat/
 
     if __version__ == '1.1.2':
         _make_empty()