Merge pull request #17 from MilagrosMarin/feat_outliers_removal

Major Changes! Modernize, add new features, adapt to the latest kpms version, pipeline redesign, new `report` schema

Author: ttngu207

.devcontainer/Dockerfile

@@ -1,10 +1,13 @@
-FROM python:3.9-slim@sha256:5f0192a4f58a6ce99f732fe05e3b3d00f12ae62e183886bca3ebe3d202686c7f
+ARG PY_VER=3.11
+ARG DISTRO=bullseye
+FROM mcr.microsoft.com/devcontainers/python:${PY_VER}-${DISTRO}
 
-ENV PATH /usr/local/bin:$PATH
-ENV PYTHON_VERSION 3.9.17
+# Avoid warnings by switching to noninteractive
+ENV DEBIAN_FRONTEND=noninteractive
+
+USER root
 
 RUN \
-    adduser --system --disabled-password --shell /bin/bash vscode && \
     # install docker
     apt-get update && \
     apt-get install ca-certificates curl gnupg lsb-release -y && \
@@ -31,27 +34,19 @@ COPY ./ /tmp/element-moseq/
 
 RUN \
     # pipeline dependencies
-    apt-get update && \
-    apt-get install -y gcc ffmpeg graphviz && \
-    pip install ipywidgets && \
-    pip install --no-cache-dir -e /tmp/element-moseq[kpms,elements,tests] && \
+    apt-get install gcc g++ ffmpeg libsm6 libxext6 -y && \
+    pip install --no-cache-dir -e /tmp/element-moseq[elements,tests] && \
     # clean up
-    rm -rf /tmp/element-moseq/ && \
+    rm -rf /tmp/element-moseq && \
     apt-get clean
 
-# Install CPU version for KPMS
-RUN pip install "jax[cpu]==0.3.22" -f https://storage.googleapis.com/jax-releases/jax_releases.html
-
 ENV DJ_HOST fakeservices.datajoint.io
 ENV DJ_USER root
 ENV DJ_PASS simple
 
-ENV DATA_MOUNTPOINT /workspaces/element-moseq/example_data
-ENV KPMS_ROOT_DATA_DIR $DATA_MOUNTPOINT/inbox
-ENV KPMS_PROCESSED_DATA_DIR $DATA_MOUNTPOINT/outbox
+ENV KPMS_ROOT_DATA_DIR /workspaces/element-moseq/example_data
 ENV DATABASE_PREFIX neuro_
 
 USER vscode
-CMD bash -c "sudo rm /var/run/docker.pid; sudo dockerd"
 
-ENV LD_LIBRARY_PATH="/lib:/opt/conda/lib"
+CMD bash -c "sudo rm /var/run/docker.pid; sudo dockerd"

CHANGELOG.md

@@ -3,6 +3,39 @@
 Observes [Semantic Versioning](https://semver.org/spec/v2.0.0.html) standard and
 [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) convention.
 
+## [1.0.0] - 2025-09-10
+
+> **BREAKING CHANGES** - This version contains breaking changes due to keypoint-moseq upgrade and API refactoring. Please review the changes below and update your code accordingly.
+
+### Breaking Changes
++ **BREAKING**: Upgrade keypoint-moseq from pinned 0.4.8 version to the latest version from source with breaking changes adding new features that are not compatible with the previous kpms versions
++ **BREAKING**: Rename `kpms_reader` functions to generate, load, and update kpms config files
++ **BREAKING**: Rename `PCAPrep` to `PreProcessing` table and add new attributes
++ **BREAKING**: Add feature to remove outlier keypoints in `PreProcessing` table with new `outlier_scale_factor` attribute in `PCATask`, only available in latest version of kpms
++ **BREAKING**: Add `sigmasq_loc` feature in `PreFit` and `FullFit` to automatically estimate sigmasq_loc (prior controlling the centroid movement across frames), only available in latest version of kpms
+
+### New Features and Fixes
++ Feat - Add support to load from both DLC `config.yml` and `config.yaml` file extensions
++ Feat - Add new `mosesq_report` schema with comprehensive reporting capabilities
++ Feat - Refactor `PreProcessing` table to use 3-part make function, add a new `Video` part table, and add new attributes `video_duration`, `frame_rate` and `average_frame_rate` to store these new computations
++ Feat - Move and refactor `viz_utils` into new `plotting` module
++ Feat - Update `model_name` varchar and folder naming
++ Feat - Migrate from `setup.py` to `pyproject.toml` and `conda_env.yml` for modern Python packaging standards
++ Fix - Update devcontainer to use Python 3.11 and upgrade dependencies
++ Fix - Remove JAX dependencies from `pyproject.toml`
++ Fix - Update pre-commit hooks for improved linting and consistency
++ Fix - Correct generation of `kpms_dj_config.yml` and refactor `moseq_train` and `moseq_infer` to use the renamed functions
++ Fix - Refactor `moseq_infer` and `moseq_train`, and implement a three-part make function in the most resource-intensive functions
++ Fix - Update folder naming logic to use string of combined primary attributes instead of datetime in `PreFit` and `FullFit`
++ Fix - Improve path and directory handling using `Path` objects and robust existence checks
++ Fix - Remove redundancy of variables in `PreProcessing` table
++ Fix - Update deprecated datetime usage
++ Fix - Fix filename generation in `Inference` table
++ Fix - Bugfix in `Model` imported foreign key
++ Fix - Update tutorial_pipeline
++ Add - Update docstrings across all modules
++ Add - Update pipeline images to reflect new architecture
+
 ## [0.3.2] - 2025-08-25
 + Feat - modernize packaging and environment management migrating from `setup.py` to `pyproject.toml`and `env.yml`
 + Fix - JAX compatibility issues