Feature Request: Support for rel_x, rel_y, rel_z in ElectrodesTable

[pauladkisson]

Is your feature request related to a problem? Please describe.

The NWB standard supports both relative electrode coordinates (rel_x, rel_y, rel_z - coordinates in electrode group) and absolute coordinates (x, y, z - coordinates in the brain), both of which are optional. This flexibility allows users to specify electrode positions in whichever reference frame they have available.

Spyglass currently stores only absolute brain coordinates (x, y, z), which aligns with the NWB standard definition. When NWB files contain only relative coordinates (common when labs record probe position but rely on manufacturer specs for individual electrode positions on the probe), users must manually calculate absolute coordinates before creating NWB files. If they don't, electrode coordinates are stored as NaN in Spyglass.

NWB Standard Context:

According to the NWB ElectrodesTable implementation:

• x, y, z: "coordinate of the channel location in the brain" (optional)
• rel_x, rel_y, rel_z: "coordinate in electrode group" (optional)

Spyglass Electrode table documentation shows:

• Spyglass maps to nwbf.electrodes.[index].x/y/z
• No support for rel_x, rel_y, rel_z fields

This is a reasonable design choice, as users can calculate absolute coordinates when needed: xyz_absolute = probe_position + rel_xyz.

Current Behavior:

When an NWB file contains electrodes with only rel_x, rel_y, rel_z:

• Spyglass insertion succeeds without error
• The x, y, z fields in the Spyglass Electrode table are set to NaN
• The relative coordinate information is not transferred (Spyglass has no rel_xyz fields)
• Users who need spatial analyses must manually calculate absolute coordinates before creating NWB files

Describe the enhancement you'd like

Optional enhancements to improve NWB standard support and reduce manual conversion steps:

1. Store relative coordinates (optional): Add rel_x, rel_y, rel_z fields to the Spyglass Electrode table to preserve this NWB standard metadata when available. This would maintain full fidelity with NWB files that include relative coordinates.

2. Automatically calculate absolute coordinates (optional): When probe position is known (e.g., from NwbElectrodeGroup.targeted_x/y/z), automatically calculate absolute electrode coordinates: xyz_absolute = probe_position + rel_xyz. This would reduce manual preprocessing steps for users.

3. Improve documentation (recommended): Document the current workflow:

   • Spyglass x, y, z fields store absolute brain coordinates per NWB standard
   • Users with only relative coordinates should calculate absolute coordinates before creating NWB files
   • Example calculation: xyz_absolute = probe_position + rel_xyz

Describe alternatives you've considered

The current workflow (users manually calculating absolute coordinates) is functional and reasonable:

• Users can compute xyz_absolute = probe_position + rel_xyz in their data conversion pipelines
• This gives users control over the coordinate transformation
• The calculation is straightforward when probe position is known

However, improved documentation would help users understand:

• That Spyglass x, y, z represent brain coordinates
• How to perform this calculation when needed
• Why NaN values appear when only relative coordinates are provided

Additional context

The NWB standard includes a relative coordinate system (rel_xyz) to support cases where electrode positions are known relative to a probe, but absolute positions may not be available or may vary across experiments. Supporting this feature in Spyglass would enhance interoperability with the NWB standard and reduce manual preprocessing for some users, though the current approach of requiring absolute coordinates is a reasonable design choice.

Test case:

The following script demonstrates the current behavior. When run, it shows that Spyglass stores NaN for x, y, z coordinates when only relative coordinates are provided:

from pynwb.testing.mock.file import mock_NWBFile
from pynwb.testing.mock.ecephys import mock_ElectricalSeries
from ndx_franklab_novela import DataAcqDevice, Probe, Shank, ShanksElectrode, NwbElectrodeGroup
from pynwb import NWBHDF5IO
import numpy as np
from pathlib import Path


def main():
    nwbfile = mock_NWBFile(
        identifier="electrode_coordinates_test_demo",
        session_description="Test NWB file to verify behavior with only relative electrode coordinates"
    )

    data_acq_device = DataAcqDevice(
        name="my_data_acq",
        system="my_system",
        amplifier="my_amplifier",
        adc_circuit="my_adc_circuit"
    )
    nwbfile.add_device(data_acq_device)

    electrodes = [
        ShanksElectrode(name="1", rel_x=0.0, rel_y=0.0, rel_z=0.0),
        ShanksElectrode(name="2", rel_x=0.0, rel_y=20.0, rel_z=0.0),
    ]
    
    probe = Probe(
        name="probe1",
        id=1,
        probe_type="tetrode",
        units="um",
        probe_description="test probe",
        contact_side_numbering=False,
        contact_size=10.0,
        shanks=[Shank(name="1", shanks_electrodes=electrodes)],
    )
    nwbfile.add_device(probe)

    electrode_group = NwbElectrodeGroup(
        name="electrode_group1",
        description="electrode group for probe1",
        location="CA1",
        device=probe,
        targeted_location="CA1",
        targeted_x=0.0,
        targeted_y=0.0,
        targeted_z=0.0,
        units="um",
    )
    nwbfile.add_electrode_group(electrode_group)

    extra_cols = ["probe_shank", "probe_electrode", "bad_channel", "ref_elect_id"]
    for col in extra_cols:
        nwbfile.add_electrode_column(name=col, description=f"description for {col}")

    for electrode_id in [1, 2]:
        nwbfile.add_electrode(
            location="CA1",
            group=electrode_group,
            probe_shank=1,
            probe_electrode=electrode_id,
            bad_channel=False,
            ref_elect_id=0,
        )

    electrodes_region = nwbfile.electrodes.create_region(
        name="electrodes",
        region=[0, 1],
        description="electrodes"
    )
    mock_ElectricalSeries(electrodes=electrodes_region, nwbfile=nwbfile, data=np.ones((10, 2)))

    nwbfile.create_processing_module(name="behavior", description="dummy behavior module")

    nwbfile_path = Path("/Volumes/T7/CatalystNeuro/Spyglass/raw/mock_electrode_coordinates_test.nwb")
    if nwbfile_path.exists():
        nwbfile_path.unlink()
    nwbfile_path.parent.mkdir(parents=True, exist_ok=True)
    
    with NWBHDF5IO(nwbfile_path, "w") as io:
        io.write(nwbfile)

    import datajoint as dj
    dj_local_conf_path = "/Users/pauladkisson/Documents/CatalystNeuro/Spyglass/spyglass/dj_local_conf.json"
    dj.config.load(dj_local_conf_path)
    import spyglass.common as sgc
    import spyglass.data_import as sgi
    from spyglass.utils.nwb_helper_fn import get_nwb_copy_filename
    
    nwb_copy_file_name = get_nwb_copy_filename(nwbfile_path.name)
    (sgc.Nwbfile & {"nwb_file_name": nwb_copy_file_name}).delete()
    (sgc.ProbeType & {"probe_type": "tetrode"}).delete()
    
    sgi.insert_sessions(str(nwbfile_path), rollback_on_fail=True, raise_err=True)
    
    print("\nInsertion successful!")
    print("Checking what coordinates were stored in Spyglass Electrode table:")
    print(sgc.Electrode & {"nwb_file_name": nwb_copy_file_name})


if __name__ == "__main__":
    main()

Actual output:

Insertion successful!
Checking what coordinates were stored in Spyglass Electrode table:
*nwb_file_name *electrode_gro *electrode_id  probe_id     probe_shank    probe_electrod region_id     name     original_refer x       y       z       filtering      impedance     bad_channel    x_warped     y_warped     z_warped     contacts    
+------------+ +------------+ +------------+ +----------+ +------------+ +------------+ +-----------+ +------+ +------------+ +-----+ +-----+ +-----+ +------------+ +-----------+ +------------+ +----------+ +----------+ +----------+ +----------+
mock_electrode electrode_grou 0              tetrode      1              1              2             0        0              nan     nan     nan     unfiltered     nan           False          0.0          0.0          0.0                      
mock_electrode electrode_grou 1              tetrode      1              2              2             1        0              nan     nan     nan     unfiltered     nan           False          0.0          0.0          0.0                      
(Total: 2)

The relative coordinate information (rel_y=20.0 for electrode 2) is lost, and all spatial information (x, y, z) is stored as nan.