Skip to content

Commit 2b71fb3

Browse files
samuelbray32Samuel BraySamuel BrayCBroz1
authored
Moseq Pipeline (#1056)
* Add organization tables for keypoint pose data * Add model training tables * apply fit moseq model to data * add ability to initialize model training from pre-existing model * make model names unique * add initial tutorial * add moseq pipeline diagram * get rid of PoseOutput and fetch pose data from PositionOutput * remove PoseOutput and corresponding references * allow passing of null entries from DLCPoseEstimation to PositionOutput for Pose-dependent analysis * Apply suggestions from code review Co-authored-by: Chris Broz <[email protected]> * lint * implement changes from review * rename fetch_video_name to fetch_video_path * cleanup outdated usages of PoseOutput * Cleanup errors from removal of PoseOutput * add get_position_interval_epoch * Add method to get training results pdf * Add description and example of hyperparameter sweep * move moseq dir definition to spyglass config * move moseq config function to method * Update changelog * update pipeline diagram * Apply suggestions from code review Co-authored-by: Chris Broz <[email protected]> * move moseq into v1 folder * add docstrings * make video symlink more robust * cleanup setup_project call * cleanup config method * Implement suggestions from code review * cleanup readability of DLCPosV1 make conditions * allow key argument when fetching video path * add moseq dependencies * Add moseq install instructions to tutorials * accept key in all get video path functions * fix spelling * fix spelling * make model project dir unique * add default empty restriction * add moseq to omitted tests * tutorial error cleanup * fix for codespell update * typo fix * remove jax version pin * add docstrings * make restriction in fetch_pose_dataframe compatible with export * add moseq dirs to config example * Update src/spyglass/position/v1/position_dlc_pose_estimation.py Co-authored-by: Chris Broz <[email protected]> * cleanup long settings if statements * change to and register the behavior schema prefix * Update src/spyglass/position/position_merge.py Co-authored-by: Chris Broz <[email protected]> * lint --------- Co-authored-by: Samuel Bray <[email protected]> Co-authored-by: Samuel Bray <[email protected]> Co-authored-by: Chris Broz <[email protected]>
1 parent 4f39dfa commit 2b71fb3

26 files changed

+4245
-57
lines changed

‎CHANGELOG.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66

77
- Ensure merge tables are declared during file insertion #1205
88
- Update URL for DANDI Docs #1210
9+
- Add common method `get_position_interval_epoch` #1056
910
- Improve cron job documentation and script #1226, #1241
1011
- Update export process to include `~external` tables #1239
1112
- Only add merge parts to `source_class_dict` if present in codebase #1237
@@ -22,6 +23,10 @@
2223
- Fix compatibility bug between v1 pipeline and `SortedSpikesGroup` unit
2324
filtering #1238
2425

26+
- Behavior
27+
- Implement pipeline for keypoint-moseq extraction of behavior syllables #1056
28+
29+
2530
## [0.5.4] (December 20, 2024)
2631

2732
### Infrastructure

‎dj_local_conf_example.json‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,11 @@
5151
"video": "/your/base/path/deeplabcut/video",
5252
"output": "/your/base/path/deeplabcut/output"
5353
},
54+
"moseq_dirs": {
55+
"base": "/your/base/path/moseq",
56+
"project": "/your/base/path/moseq/projects",
57+
"video": "/your/base/path/moseq/video",
58+
},
5459
"kachery_zone": "franklab.default"
5560
}
5661
}

‎environment_moseq_cpu.yml‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# 1. Install a conda distribution.
2+
# https://mamba.readthedocs.io/en/latest/installation/mamba-installation.html
3+
# 2. Run: `mamba env create -f environment.yml`
4+
# 3. Activate: `conda activate spyglass`
5+
#
6+
# (lines intentionally left blank)
7+
#
8+
#
9+
name: spyglass-moseq-cpu
10+
channels:
11+
- conda-forge
12+
# - defaults # deprecated
13+
- franklab
14+
- edeno
15+
# - pytorch # dlc-only
16+
# - anaconda # dlc-only, for cudatoolkit
17+
dependencies:
18+
- bottleneck
19+
# - cudatoolkit=11.3 # dlc-only
20+
# - ffmpeg # dlc-only
21+
- ipympl
22+
- jupyterlab>=3.*
23+
# - libgcc # dlc-only
24+
- matplotlib
25+
- non_local_detector
26+
- numpy
27+
- pip
28+
- position_tools
29+
- pybind11 # req by mountainsort4 -> isosplit5
30+
- pydotplus
31+
- pyfftw<=0.12.0 # ghostipy req. install from conda-forge for Mac ARM
32+
- python>=3.9,<3.13
33+
- pytorch<1.12.0
34+
- ripple_detection
35+
- seaborn
36+
# - torchaudio # dlc-only
37+
# - torchvision # dlc-only
38+
- track_linearization>=2.3
39+
- pip:
40+
- ghostipy # for common_filter
41+
- mountainsort4
42+
- .[moseq-cpu]

‎environment_moseq_gpu.yml‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# 1. Install a conda distribution.
2+
# https://mamba.readthedocs.io/en/latest/installation/mamba-installation.html
3+
# 2. Run: `mamba env create -f environment.yml`
4+
# 3. Activate: `conda activate spyglass`
5+
#
6+
# (lines intentionally left blank)
7+
#
8+
#
9+
name: spyglass-moseq-gpu
10+
channels:
11+
- conda-forge
12+
# - defaults # deprecated
13+
- franklab
14+
- edeno
15+
# - pytorch # dlc-only
16+
# - anaconda # dlc-only, for cudatoolkit
17+
dependencies:
18+
- bottleneck
19+
# - cudatoolkit=11.3 # dlc-only
20+
# - ffmpeg # dlc-only
21+
- ipympl
22+
- jupyterlab>=3.*
23+
# - libgcc # dlc-only
24+
- matplotlib
25+
- non_local_detector
26+
- numpy
27+
- pip
28+
- position_tools
29+
- pybind11 # req by mountainsort4 -> isosplit5
30+
- pydotplus
31+
- pyfftw<=0.12.0 # ghostipy req. install from conda-forge for Mac ARM
32+
- python>=3.9,<3.13
33+
- pytorch<1.12.0
34+
- ripple_detection
35+
- seaborn
36+
# - torchaudio # dlc-only
37+
# - torchvision # dlc-only
38+
- track_linearization>=2.3
39+
- pip:
40+
- ghostipy # for common_filter
41+
- mountainsort4
42+
- .[moseq-gpu]

‎notebook-images/moseq_outline.png‎

59.7 KB
Loading

‎notebooks/00_Setup.ipynb‎

Lines changed: 43 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -285,7 +285,47 @@
285285
"The Decoding pipeline relies on `jax` to process data with GPUs. Please see\n",
286286
"their conda installation steps\n",
287287
"[here](https://jax.readthedocs.io/en/latest/installation.html#conda-installation).\n",
288-
"\n"
288+
"\n",
289+
"#### Deep Lab Cut (DLC)\n",
290+
"\n",
291+
"Spyglass provides an environment build for using the DLC pipeline. To create an \n",
292+
"environment with these features, please:\n",
293+
"1. navigate to your cloned spyglass repo.\n",
294+
"2. build the environment from the dlc version\n",
295+
"3. activate the environment to use\n",
296+
"\n",
297+
"```bash\n",
298+
"cd /path/to/spyglass # 1\n",
299+
"mamba env create -f environment_dlc.yml # 2\n",
300+
"mamba activate spyglass-dlc # 3\n",
301+
"```\n",
302+
"\n",
303+
"Alternatively, you can pip install using\n",
304+
"```bash\n",
305+
"pip install spyglass[dlc]\n",
306+
"```\n",
307+
"\n",
308+
"#### Keypoint-Moseq\n",
309+
"\n",
310+
"Spyglass provides an environment build for using the Moseq pipeline. To create an \n",
311+
"environment with these features, please:\n",
312+
"1. navigate to your cloned spyglass repo.\n",
313+
"2. build the environment from one of the moseq versions\n",
314+
"3. activate the environment to use\n",
315+
"\n",
316+
"```bash\n",
317+
"cd /path/to/spyglass # 1\n",
318+
"mamba env create -f environment_moseq_cpu.yml # 2\n",
319+
"mamba activate spyglass-moseq-cpu # 3\n",
320+
"```\n",
321+
"\n",
322+
"Alternatively, you can pip install using\n",
323+
"```bash\n",
324+
"pip install spyglass[moseq-cpu]\n",
325+
"```\n",
326+
"\n",
327+
"To use a GPU enabled version of the package, replace `cpu` with `gpu` in the above \n",
328+
"commands\n"
289329
]
290330
},
291331
{
@@ -593,7 +633,7 @@
593633
],
594634
"metadata": {
595635
"kernelspec": {
596-
"display_name": "Python 3 (ipykernel)",
636+
"display_name": "spyglass",
597637
"language": "python",
598638
"name": "python3"
599639
},
@@ -607,7 +647,7 @@
607647
"name": "python",
608648
"nbconvert_exporter": "python",
609649
"pygments_lexer": "ipython3",
610-
"version": "3.9.19"
650+
"version": "3.9.18"
611651
}
612652
},
613653
"nbformat": 4,

0 commit comments

Comments
 (0)