ImportedCompassDirection #1439
Description
Is your feature request related to a problem? Please describe.
Spyglass supports imported Position but not imported CompassDirection, even though the computed-table IntervalPositionInfo generates components for both position and orientation.
Describe the solution you'd like
It would be great if alongside sgc.RawPosition, there were a table for imported CompassDirection objects like sgc.RawCompassDirection.
Describe alternatives you've considered
Currently, as a workaround, I am including the head direction data as an additional spatial series in the Position object, which then gets inserted into sgc.RawPosition. But this definitely feels like sub-optimal data representation.
Additional context
Here is a minimal reproduction of the problem.
from pynwb.testing.mock.file import mock_NWBFile
from pynwb import NWBHDF5IO
from pynwb.behavior import SpatialSeries, Position, CompassDirection
from pynwb.core import DynamicTable
from pathlib import Path
import numpy as np
def add_compass_direction(nwbfile):
nwbfile.add_epoch(start_time=0.0, stop_time=100.0, tags=["01"])
# Create behavior module
behavior_module = nwbfile.create_processing_module(
name="behavior",
description="Behavioral data including position and compass direction"
)
# Create mock position data (circular trajectory)
n_samples = 1000
timestamps = np.linspace(0, 100, n_samples) # 100 seconds
theta = np.linspace(0, 4 * np.pi, n_samples) # 2 full circles
radius = 50.0 # cm
x_pos = radius * np.cos(theta)
y_pos = radius * np.sin(theta)
position_data = np.column_stack([x_pos, y_pos])
# Create Position object with SpatialSeries
position_spatial_series = SpatialSeries(
name="position",
description="(x,y) position in the environment",
data=position_data,
timestamps=timestamps,
reference_frame="arena coordinates",
unit="centimeters"
)
position_obj = Position(spatial_series=position_spatial_series)
behavior_module.add(position_obj)
# Create mock head direction data (aligned with movement direction)
# Head direction follows the tangent of the circular path
head_direction = theta % (2 * np.pi) # Keep in [0, 2π] range
head_direction_data = head_direction[:, np.newaxis] # Spyglass requires 2D array
# Create CompassDirection object with SpatialSeries
direction_spatial_series = SpatialSeries(
name="head_direction",
description="Horizontal angle of the head (yaw) in radians",
data=head_direction_data,
timestamps=timestamps,
reference_frame="arena coordinates",
unit="radians"
)
compass_direction_obj = CompassDirection(spatial_series=direction_spatial_series)
behavior_module.add(compass_direction_obj)
def insert_session(nwbfile_path: Path):
"""Insert the NWB file into Spyglass database."""
import datajoint as dj
dj_local_conf_path = "/Users/pauladkisson/Documents/CatalystNeuro/Spyglass/spyglass/dj_local_conf.json"
dj.config.load(dj_local_conf_path) # load config for database connection info
# General Spyglass Imports
import spyglass.common as sgc
import spyglass.data_import as sgi
from spyglass.utils.nwb_helper_fn import get_nwb_copy_filename
nwb_copy_file_name = get_nwb_copy_filename(nwbfile_path.name)
# Delete existing entries
(sgc.Nwbfile & {"nwb_file_name": nwb_copy_file_name}).delete()
# Insert the session
sgi.insert_sessions(str(nwbfile_path), rollback_on_fail=True, raise_err=True)
print(sgc.PositionSource.SpatialSeries & {"nwb_file_name": "mock_compass_direction_.nwb"})
print((sgc.RawPosition & {"nwb_file_name": "mock_compass_direction_.nwb"}).fetch1_dataframe())
def main():
nwbfile = mock_NWBFile(
identifier="compass_direction_bug_demo",
session_description="Mock NWB file demonstrating Spyglass CompassDirection import bug"
)
add_compass_direction(nwbfile)
# Save the mock NWB file
nwbfile_path = Path("/Volumes/T7/CatalystNeuro/Spyglass/raw/mock_compass_direction.nwb")
if nwbfile_path.exists():
nwbfile_path.unlink()
# Create directory if it doesn't exist
nwbfile_path.parent.mkdir(parents=True, exist_ok=True)
with NWBHDF5IO(nwbfile_path, "w") as io:
io.write(nwbfile)
insert_session(nwbfile_path=nwbfile_path)
if __name__ == "__main__":
main()And the resulting output from running the script.
/opt/anaconda3/envs/spyglass/lib/python3.10/site-packages/datajoint/plugin.py:4: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
import pkg_resources
[2025-10-22 14:07:08,884][INFO]: DataJoint is configured from /Users/pauladkisson/Documents/CatalystNeuro/Spyglass/spyglass/dj_local_conf.json
[2025-10-22 14:07:09,999][INFO]: DataJoint 0.14.6 connected to root@localhost:3306
[2025-10-22 14:07:11,825][INFO]: Deleting 1 rows from `common_behav`.`_raw_position__pos_object`
[2025-10-22 14:07:11,837][INFO]: Deleting 1 rows from `common_behav`.`_raw_position`
[2025-10-22 14:07:11,858][INFO]: Deleting 1 rows from `common_behav`.`position_source__spatial_series`
[2025-10-22 14:07:11,872][INFO]: Deleting 1 rows from `common_behav`.`position_source`
[2025-10-22 14:07:11,891][INFO]: Deleting 2 rows from `common_interval`.`interval_list`
[2025-10-22 14:07:11,903][INFO]: Deleting 1 rows from `common_session`.`_session`
[2025-10-22 14:07:11,917][INFO]: Deleting 1 rows from `common_nwbfile`.`nwbfile`
Commit deletes? [yes, No]: yes
[2025-10-22 14:07:14,522][INFO]: Delete committed.
[14:07:14][INFO] Spyglass: Creating a copy of NWB file mock_compass_direction.nwb with link to raw ephys data: mock_compass_direction_.nwb
[14:07:22][INFO] Spyglass: Populating Session...
[14:07:23][INFO] Spyglass: Session: No config found at raw/mock_compass_direction_.nwb
[14:07:23][INFO] Spyglass: Session populates Institution...
[14:07:23][INFO] Spyglass: No institution metadata found.
[14:07:23][INFO] Spyglass: Session populates Lab...
[14:07:23][INFO] Spyglass: No lab metadata found.
[14:07:23][INFO] Spyglass: Session populates LabMember...
[14:07:23][INFO] Spyglass: No experimenter metadata found.
[14:07:23][INFO] Spyglass: Session populates Subject...
[14:07:23][WARNING] Spyglass: No subject metadata found.
[14:07:23][INFO] Spyglass: Session populates Populate DataAcquisitionDevice...
[14:07:23][WARNING] Spyglass: No conforming data acquisition device metadata found.
[14:07:23][INFO] Spyglass: Session populates Populate CameraDevice...
[14:07:23][WARNING] Spyglass: No conforming camera device metadata found.
[14:07:23][INFO] Spyglass: Session populates Populate Probe...
[14:07:23][WARNING] Spyglass: No conforming probe metadata found.
[14:07:23][INFO] Spyglass: Skipping Apparatus for now...
[14:07:23][INFO] Spyglass: Session populates IntervalList...
[14:07:23][INFO] Spyglass: Populating ElectrodeGroup...
[14:07:23][INFO] Spyglass: Populating Raw...
/Users/pauladkisson/Documents/CatalystNeuro/Spyglass/catalystneuro_fork/spyglass/src/spyglass/common/common_ephys.py:302: UserWarning: Unable to get acquisition object in: /Volumes/T7/CatalystNeuro/Spyglass/raw/mock_compass_direction_.nwb
Skipping entry in `common_ephys`.`_raw`
warnings.warn(
[14:07:23][INFO] Spyglass: Populating SampleCount...
[14:07:23][INFO] Spyglass: Unable to import SampleCount: no data interface named "sample_count" found in mock_compass_direction_.nwb.
[14:07:23][INFO] Spyglass: Populating DIOEvents...
[14:07:23][WARNING] Spyglass: No conforming behavioral events data interface found in mock_compass_direction_.nwb
[14:07:23][INFO] Spyglass: Populating TaskEpoch...
[14:07:23][WARNING] Spyglass: No tasks processing module found in root pynwb.file.NWBFile at 0x5375169616
Fields:
epochs: epochs <class 'pynwb.epoch.TimeIntervals'>
file_create_date: [datetime.datetime(2025, 10, 22, 14, 7, 7, 173086, tzinfo=tzoffset(None, -25200))]
identifier: compass_direction_bug_demo
intervals: {
epochs <class 'pynwb.epoch.TimeIntervals'>
}
processing: {
behavior <class 'pynwb.base.ProcessingModule'>
}
session_description: Mock NWB file demonstrating Spyglass CompassDirection import bug
session_start_time: 1970-01-01 00:00:00-08:00
timestamps_reference_time: 1970-01-01 00:00:00-08:00
or config
[14:07:23][INFO] Spyglass: Populating ImportedSpikeSorting...
[14:07:24][WARNING] Spyglass: No units found in NWB file
[14:07:24][INFO] Spyglass: Populating SensorData...
[14:07:24][INFO] Spyglass: No conforming sensor data found in mock_compass_direction_.nwb
[14:07:24][INFO] Spyglass: Populating Electrode...
[14:07:24][INFO] Spyglass: Populating PositionSource...
[14:07:24][INFO] Spyglass: Estimated sampling rate for None: 10.0 Hz
[14:07:24][INFO] Spyglass: Populating VideoFile...
[14:07:24][INFO] Spyglass: Populating StateScriptFile...
[14:07:24][INFO] Spyglass: Populating ImportedPose...
[14:07:24][INFO] Spyglass: Populating ImportedLFP...
[14:07:24][WARNING] Spyglass: No LFP objects found in mock_compass_direction_.nwb. Skipping.
[14:07:24][INFO] Spyglass: Populating VirusInjection...
[14:07:25][INFO] Spyglass: Populating OpticalFiberImplant...
[14:07:25][INFO] Spyglass: Populating OptogeneticProtocol...
[14:07:25][INFO] Spyglass: Populating RawPosition...
*nwb_file_name *interval_list *id name
+------------+ +------------+ +----+ +----------+
mock_compass_d pos 0 valid ti 0 position
(Total: 1)
x y
time
0.0000 50.000000 0.000000e+00
0.1001 49.996044 6.289309e-01
0.2002 49.984178 1.257762e+00
0.3003 49.964402 1.886395e+00
0.4004 49.936721 2.514729e+00
... ... ...
99.5996 49.936721 -2.514729e+00
99.6997 49.964402 -1.886395e+00
99.7998 49.984178 -1.257762e+00
99.8999 49.996044 -6.289309e-01
100.0000 50.000000 -2.449294e-14
[1000 rows x 2 columns]