Refactor to store complete set of photometry metadata (calderast#152)

* Refactor to store more photometry metadata
* Update optic fiber and indicator names to be unique, only add recorded indicators to table
* Add virus info and photometry mappings
* Standardize references to gACh4h (its the same as GRAB-ACh3.8)
* Update tests to reflect metadata changes
* Update example metadata

---------

Co-authored-by: Stephanie Crater <[email protected]>
Co-authored-by: Stephanie Crater <[email protected]>

Author: rly

metadata_example.yaml

@@ -52,44 +52,36 @@ photometry:
 
   # see Methods > Photometry section in https://pmc.ncbi.nlm.nih.gov/articles/PMC6555489/#S6 and
   # see Star Methods > Fiber Photometry section in https://pmc.ncbi.nlm.nih.gov/articles/PMC10841332/#S11
+  # NOTE: this setup assumes that only one fiber was recorded in the session
+  # NOTE: the mapping of recorded signals to identity of the signal is hardcoded in the conversion code
   excitation_sources:  # reference existing excitation sources from resources/photometry_devices.yaml
     - Thorlabs Purple LED
     - Thorlabs Blue LED
-  optic_fibers:  # reference existing optic fibers from resources/photometry_devices.yaml
-    - Doric 0.66mm Flat 40mm Optic Fiber
-
-  # Note "optic_fiber_implant_sites", "viruses", and "virus_injections" 
-  # are not currently parsed and will be ignored.
-  # This is waiting on changes to be pushed to ndx-fiber-photometry.
-  # For now, specify all virus/fiber info as plain text in the "surgery" and "virus" fields (see below).
-  optic_fiber_implant_sites:
-    - reference: bregma at the cortical surface
-      hemisphere: right
+
+  photodetector: Doric iFMC7-G2 (7 ports Fluorescence Mini Cube - Three Fluorophores)  # reference existing photodetectors from resources/photometry_devices.yaml
+
+  optic_fiber_implant_sites:  # for bilateral implants, specify two entries with the same virus_name and change the ml_in_mm
+    - optic_fiber: Doric 0.66mm Flat 40mm Optic Fiber  # reference existing optic fibers from resources/photometry_devices.yaml
+      targeted_location: NAcc
       ap_in_mm: 1.7
       ml_in_mm: 1.7
       dv_in_mm: -6.0  # 6.0 mm deep for males, 5.8 mm deep for females
-      optic_fiber: Doric 0.66mm Flat 40mm Optic Fiber  # name of an optic fiber in the 'optic_fibers' field
-      excitation_sources:  # names of excitation sources in the 'excitation_sources' field
-        - Thorlabs Purple LED  
-        - Thorlabs Blue LED
-  viruses:
-    - name: dLight
-      construct_name: AAVDJ-CAG-dLight1.3b
-      description: AAV virus with dLight1.3b sensor
-      manufacturer: Vigene
-      titer_in_vg_per_ml: 2e12
-  virus_injections:
-    - description: Injection of dLight AAV virus into nucleus accumbens core
-      location: NAcc
-      hemisphere: right
-      reference: bregma at the cortical surface
+      recording: true  # ONLY ONE of the optic fibers should have recording set to true
+    - optic_fiber: Doric 0.66mm Flat 40mm Optic Fiber  # reference existing optic fibers from resources/photometry_devices.yaml
+      targeted_location: NAcc
+      ap_in_mm: 1.7
+      ml_in_mm: -1.7
+      dv_in_mm: -6.0  # 6.0 mm deep for males, 5.8 mm deep for females
+
+  virus_injections:  # for bilateral injections, specify two entries with the same virus_name and change the ml_in_mm
+    - virus_name: dLight1.3b  # reference existing indicator or opsin name in the virus_info.yaml
+      targeted_location: NAcc
       ap_in_mm: 1.7
       ml_in_mm: 1.7
       dv_in_mm: -6.2  # 6.2 mm deep for males, 6.0 mm deep for females
       volume_in_uL: 1.0
-      virus: dLight  # name of a virus in the 'viruses' field
-  photodetectors:  # reference existing photodetectors from resources/photometry_devices.yaml
-    - Doric iFMC7-G2 (7 ports Fluorescence Mini Cube - Three Fluorophores)
+      titer_in_vg_per_mL: 2e12  # after dilution
+
 
 behavior:
   arduino_text_file_path: "tests/test_data/behavior/arduinoraw0.txt"
@@ -107,14 +99,14 @@ surgery:
   Specify surgery type (virus injection or probe implantation), date, and
   any probe/virus details not already found in 'virus' or 'device' fields
 
-virus: 
-  Description of relevant virus information as text. 
+virus:
+  Description of relevant virus information as text.
   Specify injection date, manufacturer, titer, location/coordinates, etc.
 
 notes:
-  General notes about this session, if any. 
-  For example - Reward used was 15uL sucrose solution (10% sucrose, 0.1% NaCl) 
-  at each reward port. The rat was water deprived before the experiment. 
+  General notes about this session, if any.
+  For example - Reward used was 15uL sucrose solution (10% sucrose, 0.1% NaCl)
+  at each reward port. The rat was water deprived before the experiment.
   Behavior was ok, but this rat likes to climb over the barriers.
 
 keywords:

resources/README.md

@@ -51,5 +51,16 @@ Extra 8 channels for 6mm probe:
 2880,1
 
 # Photometry resources
-The `photometry_devices.yaml` file contains a running list of excitation sources (i.e. LEDs and their metadata/wavelengths), optic fibers, and photodetectors used by the Berke Lab for the hex maze task. Note that this list is currently in-progress and some values are placeholders. The ndx-fiber-photometry extension for nwb is still in development, so how we specify these values is subject to change.
+The `photometry_devices.yaml` file contains a running list of excitation sources (i.e. LEDs and their metadata/wavelengths), optic fibers, and photodetectors used by the Berke Lab for the hex maze task. 
+
+`virus_info.yaml` contains a running list of all indicators used for fiber photometry, as well as any excitatory/inhibitory opsins used for experiments.
+
+`photometry_mappings.yaml` maps each indicator to a list of possible excitation sources.
+Our current photometry processing code makes a lot of assumptions about which channel is which signal based on known experimental choices (e.g. we always record dLight in the old maze room, and always do dual recordings of rDA3m and gACh4h in the new maze room). Currently the old maze room has blue (470 nm) and purple (405 nm) LEDs, and the new maze room has blue (470 nm), purple (405 nm), and green (565 nm) LEDs.
+
+See Github issue #28 (https://github.com/calderast/jdb_to_nwb/issues/28) for a complete overview of the current setup and assumptions.
+
+As we move towards adding more photometry metadata and flexibility to our photometry processing, we will allow the user to specify which wavelength was recorded by each channel. Knowing the indicators present and the wavelengths of each channel allows us to flexibly process signals for various recording setups. Each time we use a new indicator, it should be added to both `virus_info.yaml` and `photometry_mappings.yaml`
+
+Note that these lists are currently in-progress. The ndx-fiber-photometry extension for nwb is still in development, so how we specify these values is subject to change.
 

resources/photometry_mappings.yaml

@@ -0,0 +1,32 @@
+# Map each indicator to a list of possible excitation sources.
+# Each indicator should match a named virus in `virus_info.yaml`
+
+dLight1.3b:
+  # Old maze room (Thorlabs LEDs)
+  - Thorlabs Blue LED               # 470 nm (signal wavelength)
+  - Thorlabs Purple LED             # 405 nm (isosbestic wavelength)
+  # New maze room (Doric LEDs)
+  - Doric Blue LED                  # 470 nm (signal wavelength)
+  - Doric Purple LED                # 405 nm (isosbestic wavelength)
+
+dLight3.8:
+  # Old maze room (Thorlabs LEDs)
+  - Thorlabs Blue LED               # 470 nm (signal wavelength)
+  - Thorlabs Purple LED             # 405 nm (isosbestic wavelength)
+  # New maze room (Doric LEDs)
+  - Doric Blue LED                  # 470 nm (signal wavelength)
+  - Doric Purple LED                # 405 nm (isosbestic wavelength)
+
+rDA3m (AAV9):
+  - Doric Green LED                 # 565 nm (signal wavelength)
+
+rDA3m (rAAV):
+  - Doric Green LED                 # 565 nm (signal wavelength)
+
+gACh4h:
+  # Old maze room (Thorlabs LEDs)
+  - Thorlabs Blue LED               # 470 nm (peak signal wavelength)
+  - Thorlabs Purple LED             # 405 nm (for ratiometric index)
+  # New maze room (Doric LEDs)
+  - Doric Blue LED                  # 470 nm (peak signal wavelength)
+  - Doric Purple LED                # 405 nm (for ratiometric index)

resources/virus_info.yaml

@@ -0,0 +1,28 @@
+indicators:
+  - name: dLight1.3b
+    construct_name: AAVDJ-CAG-dLight1.3b
+    description: AAV-DJ virus expressing the dopamine sensor dLight1.3b under the CAG promoter
+    manufacturer: Vigene
+
+  - name: dLight3.8
+    construct_name: AAV-DJ-CAG-dLight3.8
+    description: AAV-DJ virus expressing the dopamine sensor dLight3.8 under the CAG promoter
+    manufacturer: UNC
+
+  - name: rDA3m (AAV9)
+    construct_name: AAV9-hsyn-rDA3m
+    description: AAV9 virus expressing the red-shifted dopamine sensor GRAB rDA3m under the hSyn promoter
+    manufacturer: BrainVTA
+
+  - name: rDA3m (rAAV)
+    construct_name: rAAV-hsyn-rDA3m
+    description: Recombinant AAV expressing the red-shifted dopamine sensor GRAB rDA3m under the hSyn promoter
+    manufacturer: BrainVTA
+
+  - name: gACh4h
+    construct_name: AAV-hSyn-ACh3.8
+    description: >
+      AAV virus expressing the acetylcholine sensor GRAB-ACh3.8 under the hSyn promoter. Unpublished, Yulong Li Lab
+    manufacturer: BrainVTA
+
+opsins: