Merge pull request #1611 from NeurodataWithoutBorders/schema_2.6.0

Author: rly

.github/workflows/run_coverage.yml

@@ -71,7 +71,7 @@ jobs:
 
       - name: Run integration tests and generate coverage report
         run: |
-          python -m coverage run -p test.py --integration
+          python -m coverage run -p test.py --integration --backwards
           # validation CLI tests generate separate .coverage files that need to be merged
           python -m coverage combine
           python -m coverage xml  # codecov uploader requires xml format

CHANGELOG.md

@@ -1,9 +1,12 @@
 # PyNWB Changelog
 
-## Upcoming
+## PyNWB 2.3.0 (February 23, 2023)
 
 ### Enhancements and minor changes
-- `Subject.age` can be input as a `timedelta`. @bendichter [#1590](https://github.com/NeurodataWithoutBorders/pynwb/pull/1590)
+- Added support for NWB Schema 2.6.0. @mavaylon1 [#1636](https://github.com/NeurodataWithoutBorders/pynwb/pull/1636)
+- Added a class and tests for the `OnePhotonSeries` new in NWB v2.6.0. @CodyCBakerPhD [#1593](https://github.com/NeurodataWithoutBorders/pynwb/pull/1593)(see also NWB Schema [#523](https://github.com/NeurodataWithoutBorders/nwb-schema/pull/523)
+- `Subject.age` can be input as a `timedelta` type. @bendichter [#1590](https://github.com/NeurodataWithoutBorders/pynwb/pull/1590)
+- Added `Subject.age__reference` field. @bendichter ([#1540](https://github.com/NeurodataWithoutBorders/pynwb/pull/1540))
 - `IntracellularRecordingsTable.add_recording`: the `electrode` arg is now optional, and is automatically populated from the stimulus or response.
   [#1597](https://github.com/NeurodataWithoutBorders/pynwb/pull/1597)
 - Added module `pynwb.testing.mock.icephys` and corresponding tests. @bendichter
@@ -13,6 +16,7 @@
 - Added ``NWBHDF5IO.nwb_version`` property to get the NWB version from an NWB HDF5 file @oruebel [#1612](https://github.com/NeurodataWithoutBorders/pynwb/pull/1612)
 - Updated ``NWBHDF5IO.read`` to check NWB version before read and raise more informative error if an unsupported version is found @oruebel [#1612](https://github.com/NeurodataWithoutBorders/pynwb/pull/1612)
 - Added the `driver` keyword argument to the `pynwb.validate` function as well as the corresponding namespace caching. @CodyCBakerPhD [#1588](https://github.com/NeurodataWithoutBorders/pynwb/pull/1588)
+- Updated HDMF requirement to version 3.5.1. [#1611](https://github.com/NeurodataWithoutBorders/pynwb/pull/1611)
 - Increased the stacklevel of the warning from `_add_missing_timezone` in `pynwb.file` to make identification of which datetime field is missing a timezone easier. @CodyCBakerPhD [#1641](https://github.com/NeurodataWithoutBorders/pynwb/pull/1641)
 
 ### Documentation and tutorial enhancements:

Legal.txt

@@ -1,4 +1,4 @@
-“pynwb” Copyright (c) 2017-2022, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy).  All rights reserved.
+“pynwb” Copyright (c) 2017-2023, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy).  All rights reserved.
 
 If you have questions about your rights to use or distribute this software, please contact Berkeley Lab's Innovation & Partnerships Office at [email protected].
 

README.rst

@@ -112,7 +112,7 @@ Citing NWB
 LICENSE
 =======
 
-"pynwb" Copyright (c) 2017-2022, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy).  All rights reserved.
+"pynwb" Copyright (c) 2017-2023, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy).  All rights reserved.
 Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
 
 (1) Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
@@ -128,7 +128,7 @@ You are under no obligation whatsoever to provide any bug fixes, patches, or upg
 COPYRIGHT
 =========
 
-"pynwb" Copyright (c) 2017-2022, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy).  All rights reserved.
+"pynwb" Copyright (c) 2017-2023, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy).  All rights reserved.
 If you have questions about your rights to use or distribute this software, please contact Berkeley Lab's Innovation & Partnerships Office at [email protected].
 
 NOTICE.  This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so.

docs/source/conf.py

@@ -146,13 +146,13 @@ def __call__(self, filename):
     'fsspec': ("https://filesystem-spec.readthedocs.io/en/latest/", None),
 }
 
-extlinks = {'incf_lesson': ('https://training.incf.org/lesson/%s', ''),
-            'incf_collection': ('https://training.incf.org/collection/%s', ''),
-            'nwb_extension': ('https://github.com/nwb-extensions/%s', ''),
-            'pynwb': ('https://github.com/NeurodataWithoutBorders/pynwb/%s', ''),
-            'nwb_overview': ('https://nwb-overview.readthedocs.io/en/latest/%s', ''),
-            'hdmf-docs': ('https://hdmf.readthedocs.io/en/stable/%s', ''),
-            'dandi': ('https://www.dandiarchive.org/%s', '')}
+extlinks = {'incf_lesson': ('https://training.incf.org/lesson/%s', '%s'),
+            'incf_collection': ('https://training.incf.org/collection/%s', '%s'),
+            'nwb_extension': ('https://github.com/nwb-extensions/%s', '%s'),
+            'pynwb': ('https://github.com/NeurodataWithoutBorders/pynwb/%s', '%s'),
+            'nwb_overview': ('https://nwb-overview.readthedocs.io/en/latest/%s', '%s'),
+            'hdmf-docs': ('https://hdmf.readthedocs.io/en/stable/%s', '%s'),
+            'dandi': ('https://www.dandiarchive.org/%s', '%s')}
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
@@ -169,7 +169,7 @@ def __call__(self, filename):
 
 # General information about the project.
 project = u'PyNWB'
-copyright = u'2017-2022, Neurodata Without Borders'
+copyright = u'2017-2023, Neurodata Without Borders'
 
 # The version info for the project you're documenting, acts as replacement for
 # |version| and |release|, also used in various other places throughout the

license.txt

@@ -1,4 +1,4 @@
-“pynwb” Copyright (c) 2017-2022, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy).  All rights reserved.
+“pynwb” Copyright (c) 2017-2023, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy).  All rights reserved.
 
 Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
 

requirements-min.txt

@@ -1,6 +1,6 @@
 # minimum versions of package dependencies for installing PyNWB
 h5py==2.10  # support for selection of datasets with list of indices added in 2.10
-hdmf==3.4.0
+hdmf==3.5.1
 numpy==1.16
 pandas==1.1.5
 python-dateutil==2.7.3

requirements.txt

@@ -1,6 +1,6 @@
 # pinned dependencies to reproduce an entire development environment to use PyNWB
 h5py==3.7.0
-hdmf==3.4.6
+hdmf==3.5.1
 numpy==1.23.3;python_version>='3.8'
 numpy==1.21.5;python_version<'3.8'  # note that numpy 1.22 dropped python 3.7 support
 pandas==1.5.0;python_version>='3.8'

setup.py

@@ -20,7 +20,7 @@
 
 reqs = [
     'h5py>=2.10,<4',
-    'hdmf>=3.4.2,<4',
+    'hdmf>=3.5.1,<4',
     'numpy>=1.16,<1.24',
     'pandas>=1.1.5,<2',
     'python-dateutil>=2.7.3,<3',

src/pynwb/file.py

@@ -55,6 +55,7 @@ class Subject(NWBContainer):
 
     __nwbfields__ = (
         'age',
+        "age__reference",
         'description',
         'genotype',
         'sex',
@@ -73,8 +74,20 @@ class Subject(NWBContainer):
                    'A timedelta will automatically be converted to The ISO 8601 Duration format.',
             "default": None,
         },
-        {'name': 'description', 'type': str,
-         'doc': 'A description of the subject, e.g., "mouse A10".', 'default': None},
+        {
+            "name": "age__reference",
+            "type": str,
+            "doc": "Age is with reference to this event. Can be 'birth' or 'gestational'. If reference is omitted, "
+                   "then 'birth' is implied. Value can be None when read from an NWB file with schema version "
+                   "2.0 to 2.5 where age__reference is missing.",
+            "default": "birth",
+        },
+        {
+            "name": "description",
+            "type": str,
+            "doc": 'A description of the subject, e.g., "mouse A10".',
+            "default": None,
+        },
         {'name': 'genotype', 'type': str,
          'doc': 'The genotype of the subject, e.g., "Sst-IRES-Cre/wt;Ai32(RCL-ChR2(H134R)_EYFP)/wt".',
          'default': None},
@@ -95,18 +108,30 @@ class Subject(NWBContainer):
         {'name': 'strain', 'type': str, 'doc': 'The strain of the subject, e.g., "C57BL/6J"', 'default': None},
     )
     def __init__(self, **kwargs):
-        keys_to_set = ("age",
-                       "description",
-                       "genotype",
-                       "sex",
-                       "species",
-                       "subject_id",
-                       "weight",
-                       "date_of_birth",
-                       "strain")
+        keys_to_set = (
+            "age",
+            "age__reference",
+            "description",
+            "genotype",
+            "sex",
+            "species",
+            "subject_id",
+            "weight",
+            "date_of_birth",
+            "strain",
+        )
         args_to_set = popargs_to_dict(keys_to_set, kwargs)
-        kwargs['name'] = 'subject'
-        super().__init__(**kwargs)
+        super().__init__(name="subject", **kwargs)
+
+        # NOTE when the Subject I/O mapper (see pynwb.io.file.py) reads an age__reference value of None from an
+        # NWB 2.0-2.5 file, it sets the value to "unspecified" so that when Subject.__init__ is called, the incoming
+        # age__reference value is NOT replaced by the default value ("birth") specified in the docval.
+        # then we replace "unspecified" with None here. the user will never see the value "unspecified".
+        # the ONLY way that age__reference can now be None is if it is read as None from an NWB 2.0-2.5 file.
+        if self._in_construct_mode and args_to_set["age__reference"] == "unspecified":
+            args_to_set["age__reference"] = None
+        elif args_to_set["age__reference"] not in ("birth", "gestational"):
+            raise ValueError("age__reference, if supplied, must be 'birth' or 'gestational'.")
 
         weight = args_to_set['weight']
         if isinstance(weight, float):