Use ndx-optogenetics and bump to 0.2.0 (#6)

* Use ndx-optogenetics and bump to 0.2.0

* Update inner cached spec

* Remove "experimenter" from FrankLabOptogeneticEpochsTable

* Remove serial_number from CameraDevice

* Remove resolution_in_pixels from CameraDevice

* Fix dims of spatial filter columns

* Update upper bounds on pynwb and hdmf

* Update changelog

* Update docs and tests

* Remove extra spec files

* Add ndx-optogenetics to pinned testing reqs

* Update requirement versions

* Use latest pynwb and hdmf

* Update CHANGELOG.md

* Update CHANGELOG.md

* Update Loren's email address

Author: rly

CHANGELOG.md

@@ -1,5 +1,12 @@
 # ndx-franklab-novela Changelog
 
+## 0.2.0 (February 14, 2025)
+
+- Added `FrankLabOptogeneticEpochsTable` to store optogenetic stimulation metadata.
+- Added extension dependency on `ndx-optogenetics`.
+- Added optional `frame_rate` field to `CameraDevice`.
+- Updated versions of dependencies.
+
 ## 0.1.0 (December 9, 2021)
 
 - Removed `NdxImageSeries` data type. The core NWB `ImageSeries` type should be used instead.

README.md

@@ -5,23 +5,26 @@ ndx-franklab-novela is a python package containing NWB custom extensions for Lor
 
 # How to install
 
-Add ndx-franklab-novela to your conda environment
+Add ndx-franklab-novela to your conda environment:
+```
+pip install ndx-franklab-novela
+```
 
-`pip install git+git://github.com/LorenFrankLab/ndx-franklab-novela`
+Or install the latest version from the repository:
+```
+pip install git+git://github.com/LorenFrankLab/ndx-franklab-novela
+```
 
 The original published extension maintained by NovelaNeuro can be installed using:
+```
+conda install -c conda-forge -c novelakrk ndx-franklab-novela
+```
 
-`conda install -c conda-forge -c novelakrk ndx-franklab-novela`
-
-
-# How to install
-
-Add ndx-franklab-novela to your conda environment<br>
-```pip install git+git://github.com/LorenFrankLab/ndx-franklab-novela```
-
-The original published extension maintained by NovelaNeuro can be installed using:
-```conda install -c conda-forge -c novelakrk ndx-franklab-novela```
+# Dependencies
 
+This extension uses the [ndx-optogenetics](https://github.com/rly/ndx-optogenetics) extension.
+Installing ndx-franklab-novela will install the latest version of ndx-optogenetics from PyPI.
+Loading `ndx-franklab-novela` by importing `ndx_franklab_novela` will also load `ndx_optogenetics`.
 
 # Extensions
 
@@ -102,6 +105,31 @@ Representation of a camera device in NWB.
 - **model**  `string`: model of this camera device
 - **lens**  `string`: info about lens in this camera
 - **camera_name**  `string`: name of this camera
+- **frame_rate**  `float`: frame rate of this camera (optional)
+
+## FrankLabOptogeneticEpochsTable
+An extension of the `OptogeneticEpochsTable` from [ndx-optogenetics](https://github.com/rly/ndx-optogenetics) with the following columns:
+
+**Columns:**
+- **epoch_name**  `string`: name of this epoch
+- **epoch_number**  `int`: 1-indexed number of this epoch
+- **convenience_code**  `string`: convenience code of this epoch
+- **epoch_type**  `string`: type of this epoch
+- **theta_filter_on**  `bool`: whether the theta filter was on (optional)
+- **theta_filter_lockout_period_in_samples**  `int`: lockout period in samples for theta filter (optional)
+- **theta_filter_phase_in_deg**  `float`: phase in degrees for theta filter (optional)
+- **theta_filter_reference_ntrode**  `int`: reference ntrode for theta filter (optional)
+- **spatial_filter_on**  `bool`: whether the spatial filter was on (optional)
+- **spatial_filter_lockout_period_in_samples**  `int`: lockout period in samples for spatial filter (optional)
+- **spatial_filter_bottom_left_coord_in_pixels**  `float`, shape `(2, )`: bottom left coordinate in pixels for spatial filter (optional)
+- **spatial_filter_top_right_coord_in_pixels**  `float`, shape `(2, )`: top right coordinate in pixels for spatial filter (optional)
+- **spatial_filter_cameras_index**  `int`: index column for spatial filter cameras (optional)
+- **spatial_filter_cameras** : references to `CameraDevice` objects used for spatial filter (optional)
+- **spatial_filter_cameras_cm_per_pixel**  `float`: cm per pixel for spatial filter cameras (optional)
+- **ripple_filter_on**  `bool`: whether the ripple filter was on (optional)
+- **ripple_filter_lockout_period_in_samples**  `int`: lockout period in samples for ripple filter (optional)
+- **ripple_filter_threshold_sd**  `float`: threshold in standard deviations for ripple filter (optional)
+- **ripple_filter_num_above_threshold**  `int`: number of tetrodes above threshold for ripple filter (optional)
 
 ---
 This extension was created using [ndx-template](https://github.com/nwb-extensions/ndx-template).

requirements-dev.txt

@@ -1,3 +1,3 @@
 # pinned dependencies to reproduce an entire development environment to run tests and check code style
-flake8==4.0.1
-pytest==6.2.5
+flake8==7.1.1
+pytest==8.3.4

requirements.txt

@@ -1,4 +1,5 @@
 # pinned dependencies to reproduce a working development environment
-hdmf_docutils==0.4.4
-hdmf==3.1.1
-pynwb==2.0.0
+hdmf_docutils==0.4.8
+hdmf==4.0.0
+pynwb==2.8.3
+ndx-optogenetics==0.2.0

setup.py

@@ -4,7 +4,7 @@
 
 from setuptools import find_packages, setup
 
-version = "0.1.0"
+version = "0.2.0"
 print(version)
 
 # load README.md/README.rst file
@@ -33,8 +33,9 @@
     'url': '',
     'license': 'BSD 3-Clause',
     'install_requires': [
-        'hdmf>=3.1.1,<4',
-        'pynwb>=2.0.0,<3'
+        'hdmf>=4.0.0,<5',
+        'pynwb>=2.8.3,<4',
+        "ndx-optogenetics>=0.2.0",
     ],
     'packages': find_packages('src/pynwb'),
     'package_dir': {'': 'src/pynwb'},

spec/ndx-franklab-novela.extensions.yaml

@@ -83,6 +83,10 @@ groups:
   - name: lens
     dtype: text
     doc: lens info
+  - name: frame_rate
+    dtype: float
+    doc: Frame rate of the camera, in frames per second.
+    required: false
 - neurodata_type_def: HeaderDevice
   neurodata_type_inc: Device
   doc: metadata from global configuration from header
@@ -179,3 +183,155 @@ groups:
   - name: units
     dtype: text
     doc: 'units of fields, acceptable values: um or mm'
+- neurodata_type_def: FrankLabOptogeneticEpochsTable
+  neurodata_type_inc: OptogeneticEpochsTable
+  doc: General metadata about the optogenetic stimulation that may change per epoch,
+    with fields specific to Loren Frank Lab experiments. If the spatial filter is
+    ON, then the experimenter can stimulate in either open (frequency-based) or closed
+    loop (theta-based), only when animal is in a particular position. If the spatial
+    filter is OFF, then ignore the position (this is not common / doesn't happen).
+    If the spatial filter is ON and the experimeter is stimulating in open loop mode
+    and the animal enters the spatial filter rectangle, then immediately apply one
+    and only one stimulation bout. If stimulating in closed loop mode and the animal
+    enters the rectangle, then every time the particular theta phase is detected,
+    immediately apply one stimulation bout (accounting for the lockout period).
+  datasets:
+  - name: epoch_name
+    neurodata_type_inc: VectorData
+    dtype: text
+    doc: Name of the epoch.
+  - name: epoch_number
+    neurodata_type_inc: VectorData
+    dtype: int
+    doc: 1-indexed number of the epoch.
+  - name: convenience_code
+    neurodata_type_inc: VectorData
+    dtype: text
+    doc: Convenience code of the epoch.
+  - name: epoch_type
+    neurodata_type_inc: VectorData
+    dtype: text
+    doc: Type of the epoch.
+  - name: theta_filter_on
+    neurodata_type_inc: VectorData
+    dtype: bool
+    doc: Whether the theta filter was on. A theta filter is closed-loop stimulation
+      - read one tetrode and calculate the phase. Depending on the phase of theta,
+      apply stimulation immediately. If this column is not present, then the theta
+      filter was not used.
+    quantity: '?'
+  - name: theta_filter_lockout_period_in_samples
+    neurodata_type_inc: VectorData
+    dtype: int
+    doc: If the theta filter was used, lockout period in the number of samples (based
+      on the clock of the SpikeGadgets hardware) needed between stimulations, start
+      to start. Use -1 if the theta filter was not used.
+    quantity: '?'
+  - name: theta_filter_phase_in_deg
+    neurodata_type_inc: VectorData
+    dtype: float
+    doc: 'If the theta filter was used, phase in degrees during closed-loop theta
+      phase-specific stimulation experiments. 0 is defined as the trough. 90 is ascending
+      phase. Options are: 0, 90, 180, 270, 360, NaN. Use NaN if the theta filter was
+      not used.'
+    quantity: '?'
+  - name: theta_filter_reference_ntrode
+    neurodata_type_inc: VectorData
+    dtype: int
+    doc: If the theta filter was used, reference electrode that used used for theta
+      phase-specific stimulation. ntrode is related to SpikeGadgets. ntrodes are specified
+      in the electrode groups. (note that ntrodes are 1-indexed.) mapping from ntrode
+      to electrode ID is in the electrode metadata files. Use -1 if the theta filter
+      was not used.
+    quantity: '?'
+  - name: spatial_filter_on
+    neurodata_type_inc: VectorData
+    dtype: bool
+    doc: Whether the spatial filter was on. Closed-loop stimulation based on whether
+      the position of the animal is within a specified rectangular region of the video.
+      If this column is not present, then the spatial filter was not used.
+    quantity: '?'
+  - name: spatial_filter_lockout_period_in_samples
+    neurodata_type_inc: VectorData
+    dtype: int
+    doc: If the spatial filter was used, lockout period in the number of samples.
+      Uses trodes time (samplecount). Use -1 if the spatial filter was not used.
+    quantity: '?'
+  - name: spatial_filter_bottom_left_coord_in_pixels
+    neurodata_type_inc: VectorData
+    dtype: int
+    dims:
+    - n_epochs
+    - x y
+    shape:
+    - null
+    - 2
+    doc: If the spatial filter was used, the (x, y) coordinate of the bottom-left
+      corner pixel of the rectangular region of the video that was used for space-specific
+      stimulation. (0,0) is the bottom-left corner of the video. Use (-1, -1) if the
+      spatial filter was not used.
+    quantity: '?'
+  - name: spatial_filter_top_right_coord_in_pixels
+    neurodata_type_inc: VectorData
+    dtype: int
+    dims:
+    - n_epochs
+    - x y
+    shape:
+    - null
+    - 2
+    doc: If the spatial filter was used, the (x, y) coordinate of the top-right corner
+      pixel of the rectangular region of the video that was used for space-specific
+      stimulation. (0,0) is the bottom-left corner of the video. Use (-1, -1) if the
+      spatial filter was not used.
+    quantity: '?'
+  - name: spatial_filter_cameras_index
+    neurodata_type_inc: VectorIndex
+    doc: Index column for `spatial_filter_cameras` so that each epoch can have multiple
+      cameras.
+    quantity: '?'
+  - name: spatial_filter_cameras
+    neurodata_type_inc: VectorData
+    dtype:
+      target_type: CameraDevice
+      reftype: object
+    doc: References to camera objects used for the spatial filter.
+    quantity: '?'
+  - name: spatial_filter_cameras_cm_per_pixel_index
+    neurodata_type_inc: VectorIndex
+    doc: Index column for `spatial_filter_cameras_cm_per_pixel` so that each epoch
+      can have multiple cameras.
+    quantity: '?'
+  - name: spatial_filter_cameras_cm_per_pixel
+    neurodata_type_inc: VectorData
+    dtype: float
+    doc: The cm/pixel values for each spatial filter camera used in this epoch, in
+      the same order as `spatial_filter_cameras`. Use this if the cm/pixel values
+      change per epoch. Otherwise, use the `meters_per_pixel` attribute of `CameraDevice`.
+    quantity: '?'
+  - name: ripple_filter_on
+    neurodata_type_inc: VectorData
+    dtype: bool
+    doc: Whether the ripple filter was on. Closed-loop stimulation based on whether
+      a ripple was detected - whether N tetrodes have their signal cross the standard
+      deviation threshold. If this column is not present, then the ripple filter was
+      not used.
+    quantity: '?'
+  - name: ripple_filter_lockout_period_in_samples
+    neurodata_type_inc: VectorData
+    dtype: int
+    doc: If the ripple filter was used, lockout period in the number of samples. Uses
+      trodes time (samplecount).
+    quantity: '?'
+  - name: ripple_filter_threshold_sd
+    neurodata_type_inc: VectorData
+    dtype: float
+    doc: If the ripple filter was used, the threshold for detecting a ripple, in number
+      of standard deviations.
+    quantity: '?'
+  - name: ripple_filter_num_above_threshold
+    neurodata_type_inc: VectorData
+    dtype: int
+    doc: If the ripple filter was used, the number of tetrodes that have their signal
+      cross the standard deviation threshold.
+    quantity: '?'

spec/ndx-franklab-novela.namespace.yaml

@@ -14,9 +14,6 @@ namespaces:
   name: ndx-franklab-novela
   schema:
   - namespace: core
-    neurodata_types:
-    - ElectrodeGroup
-    - Device
-    - NWBDataInterface
+  - namespace: ndx-optogenetics
   - source: ndx-franklab-novela.extensions.yaml
-  version: 0.1.0
+  version: 0.2.0

src/pynwb/ndx_franklab_novela/__init__.py

@@ -1,6 +1,9 @@
 import os
 
 from pynwb import load_namespaces, get_class
+import ndx_optogenetics  # noqa: F401
+# the above import is needed because the definition of FrankLabOptogeneticsEpochsTable
+# depends on the definition of OptogeneticEpochsTable in ndx_optogenetics
 
 # Set path of the namespace.yaml file to the expected install location
 
@@ -26,6 +29,7 @@
 AssociatedFiles = get_class('AssociatedFiles', 'ndx-franklab-novela')
 CameraDevice = get_class('CameraDevice', 'ndx-franklab-novela')
 DataAcqDevice = get_class('DataAcqDevice', 'ndx-franklab-novela')
+FrankLabOptogeneticEpochsTable = get_class('FrankLabOptogeneticEpochsTable', 'ndx-franklab-novela')
 HeaderDevice = get_class('HeaderDevice', 'ndx-franklab-novela')
 NwbElectrodeGroup = get_class('NwbElectrodeGroup', 'ndx-franklab-novela')
 Probe = get_class('Probe', 'ndx-franklab-novela')

src/pynwb/tests/test_cameraDevice.py

@@ -9,15 +9,20 @@ def test_cameraDevice_created_successfully(self):
 
         cam_1 = CameraDevice(
             name='1',
+            description="Camera used for tracking running",
             meters_per_pixel=0.20,
             camera_name='test name',
             model='ndx2000',
             lens='500dpt',
-            manufacturer='sony'
+            manufacturer='sony',
+            frame_rate=30.0,
         )
 
         self.assertEqual(cam_1.name, '1')
+        self.assertEqual(cam_1.description, 'Camera used for tracking running')
+        self.assertEqual(cam_1.meters_per_pixel, 0.20)
         self.assertEqual(cam_1.camera_name, 'test name')
         self.assertEqual(cam_1.model, 'ndx2000')
         self.assertEqual(cam_1.lens, '500dpt')
         self.assertEqual(cam_1.manufacturer, 'sony')
+        self.assertEqual(cam_1.frame_rate, 30.0)

src/pynwb/tests/test_nwbElectrodeGroup.py

@@ -1,3 +1,4 @@
+import numpy as np
 import unittest
 from unittest.mock import Mock
 
@@ -38,7 +39,10 @@ def test_nwb_electrode_group_successful_created(self):
         self.assertEqual(nwb_electrode_group.description, description)
         self.assertEqual(nwb_electrode_group.location, location)
         self.assertEqual(nwb_electrode_group.device, device)
-        self.assertEqual(nwb_electrode_group.position, position)
+        np.testing.assert_array_equal(
+            nwb_electrode_group.position,
+            np.array((1., 2., 3.), dtype=[('x', '<f8'), ('y', '<f8'), ('z', '<f8')])
+        )
         self.assertEqual(nwb_electrode_group.targeted_location, targeted_location)
         self.assertEqual(nwb_electrode_group.targeted_x, targeted_x)
         self.assertEqual(nwb_electrode_group.targeted_y, targeted_y)