Feature/issue 14/combined temperature acceleration nonwear detect (#141)

* Update metrics.py

Initial implementation of CTA non wear detect

* Update metrics.py

switch to float as input

* Update metrics.py

Handling uneven time sampling between temp and acceleration
Adding moving average as low pass filter

* Added initial unit tests + rearrange function

Added helper function and re-arranged +simplified main algorithm
Added first unit tests

* Update metrics.py

Fixed acceleration SD criteria to be for at least 2 out of 3 axis to agree with paper publication

* Update test_metrics.py

Added coverage for second criteria when accel criteria not met

* Update metrics.py

Switch to if:elif logic for clarity

* Integrate scikit support

Adding skdh DETACH algorithm
Addressed initial review comments for CTA

* Update test_metrics.py

Fix _pre_process_temperature naming

* Modify tests for macos, added unit test for DETACH

Added homebrew install for libomp for skdh support
Added initial unit test for DETACH (not good)

* Update metrics.py

Remove == comparison since mean_temp is float
Fixed scipy.interpolate import

#Note directly importing skdh.preprocessing.DETACH (skdh did not properly set dependencies?)

* Update test_metrics.py

Fix for unit tests with non-decreasing temperature

* Update metrics.py

Fix imports

* Update test_metrics.py

Additional test for the final if (for decreasing temps under temp_threshold)

* Adding majority vote function + resample function

Initial version of majority vote function and the resample function to be used to assess nonwear detection performance

* Modifications to enable cross comparison of nonwear

Changed temporal resolution of DETACH to one minute
Removed monotonically increasing requirement for resample

* Update test_computations.py

Test diff start time

* Update test_computations.py

Add coverage for ValueError because why not

* Update computations.py

Change default temporal resolution to one minute
Rounding for nonwear measurements after resampling

* Update test_computations.py

Test needs to have different temporal resolution otherwise resample doesn't get called

* Update poetry.lock

* Update metrics.py

* Update poetry.lock

resolve merge conflicts

* Update metrics.py

* Added combined ggir_detach ensenmble

Ensemble nonwear for ggir + detach
Added tests and fixed some variable naming

* Update poetry.lock

* Update test_calibration.py

* Adding in logic support for nonwear type into orchestrator

First pass at logic support for the nonwear alogirthms. Defaults to GGIR for all cases (temperature sensor or not)

* Test nonwear options

Unit test failing due to data size of sample bin file

* Add cli support for nonwear algo

Adding in cli support for nonwear
Adding test support
Using argparse, will investigate typer soon to replace this

* Add longer sample data for DETACH

Tests for orchestrator, new example data + move nonwear regrouping into the nonwear definition

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix dependency addition

Remove extras from pyproject.toml and poetry lock file

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Moving into nonwear_utils

Moving some function into nonwear_utils module

* Update orchestrator.py

* Addressing PR comments

* Initial changes for PR

* Adding in support for nonwear dispatcher

* Final edits to cli to support multiple nw algos

* Fixes after chaning var name in get_nonwear_measurements()

* Update metrics.py

Forgot to change year of reference.

* Refactoring nonwear implementation

* Update orchestrator.py

* First pass removing None epoch_length

* Update nonwear_utils.py

* Update nonwear_utils.py

* Epoch length ValueError check

* Removing support for epoch_length = None in enmo and anglez

* Update orchestrator.py

ValueError is raised in internal function.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

Author: Asanto32

.github/workflows/test.yaml

@@ -23,6 +23,12 @@ jobs:
       with:
         python-version: ${{ matrix.python_version }}
         cache: pip
+
+    - name: Install dependencies on macOS
+      if: runner.os == 'macOS'
+      run: |
+        brew install libomp
+
     - name: Install dependencies
       run: |
         poetry env use python

poetry.lock

@@ -73,13 +73,24 @@ def parse_arguments(args: Optional[List[str]] = None) -> argparse.Namespace:
         "Exactly three values must be given in ascending order, and comma seperated.",
     )
 
+    parser.add_argument(
+        "-nw",
+        "--nonwear_algorithm",
+        type=parse_nonwear_algorithms,
+        default="ggir",
+        help="Specify the non-wear detection algorithm(s) to use."
+        "Specify one or more of 'ggir', 'cta', 'detach' as a comma-separated list"
+        "(e.g. 'ggir,detach')."
+        "When multiple algorithms are specified, majority voting will be applied.",
+    )
+
     parser.add_argument(
         "-e",
         "--epoch_length",
         default=5,
         type=int,
-        help="Specify the sampling rate in seconds for all metrics. To skip resampling,"
-        " enter 0.",
+        help="Specify the sampling rate in seconds for all metrics."
+        "Must be greater than 0.",
     )
 
     parser.add_argument(
@@ -141,7 +152,8 @@ def main(
         thresholds=None
         if arguments.thresholds is None
         else cast(Tuple[float, float, float], tuple(arguments.thresholds)),
-        epoch_length=None if arguments.epoch_length == 0 else arguments.epoch_length,
+        epoch_length=arguments.epoch_length,
+        nonwear_algorithm=arguments.nonwear_algorithm,
         verbosity=log_level,
         output_filetype=arguments.output_filetype,
     )
@@ -180,3 +192,24 @@ def _none_or_float_list(value: str) -> Optional[List[float]]:
         raise argparse.ArgumentTypeError(
             f"Invalid value: {value}. Must be a comma-separated list or 'None'."
         )
+
+
+def parse_nonwear_algorithms(algorithm_name: str) -> List[str]:
+    """Parse comma-separated non-wear algorithm names.
+
+    Args:
+        algorithm_name: Command line input string, comma-separated algorithm names.
+
+    Returns:
+        The List of algorithm names.
+    """
+    valid_algorithm_names = ["ggir", "cta", "detach"]
+    algorithms = [algo.strip().lower() for algo in algorithm_name.split(",")]
+
+    for algo in algorithms:
+        if algo not in valid_algorithm_names:
+            raise argparse.ArgumentTypeError(
+                f"Invalid algorithm: '{algo}'. Must be one of: "
+                f"{', '.join(valid_algorithm_names)}."
+            )
+    return algorithms

src/wristpy/core/cli.py

@@ -1,13 +1,11 @@
 """Python based runner."""
 
-import datetime
 import itertools
 import logging
 import pathlib
-from typing import Dict, List, Literal, Optional, Tuple, Union
+from typing import Dict, List, Literal, Optional, Sequence, Tuple, Union
 
 import numpy as np
-import polars as pl
 
 from wristpy.core import config, exceptions, models
 from wristpy.io.readers import readers
@@ -16,6 +14,7 @@
     calibration,
     idle_sleep_mode_imputation,
     metrics,
+    nonwear_utils,
 )
 
 logger = config.get_logger()
@@ -48,53 +47,6 @@ def format_sleep_data(
     return sleep_array
 
 
-def format_nonwear_data(
-    nonwear_data: models.Measurement,
-    reference_measure: models.Measurement,
-    original_temporal_resolution: float,
-) -> np.ndarray:
-    """Formats nonwear data to match the temporal resolution of the other measures.
-
-    The detect_nonweaer algorithm outputs non-wear values in 15-minute windows, where
-    each timestamp represents the beginning of the window. This structure does not align
-    well with the polars upsample function, which treats the last timestamp as the end
-    of the time series. As a result, using the upsample function would cut off the
-    final window. To avoid this, we manually map the non-wear data to the reference
-    measure's resolution.
-
-    Args:
-        nonwear_data: The nonwear array to be upsampled.
-        reference_measure: The measurement we match the non_wear data's temporal
-            resolution to.
-        original_temporal_resolution: The original temporal resolution of the
-            nonwear_data.
-
-    Returns:
-        1-D np.ndarray with 1 indicating a nonwear timepoint. Will match the
-            length of the reference measure.
-    """
-    logger.debug("Upsampling nonwear data.")
-    nonwear_df = pl.DataFrame(
-        {
-            "nonwear": nonwear_data.measurements.astype(np.int64),
-            "time": nonwear_data.time,
-        }
-    ).set_sorted("time")
-
-    nonwear_array = np.zeros(len(reference_measure.time), dtype=bool)
-
-    for row in nonwear_df.iter_rows(named=True):
-        nonwear_value = row["nonwear"]
-        time = row["time"]
-        nonwear_mask = (reference_measure.time >= time) & (
-            reference_measure.time
-            <= time + datetime.timedelta(seconds=original_temporal_resolution)
-        )
-        nonwear_array[nonwear_mask] = nonwear_value
-
-    return nonwear_array
-
-
 def run(
     input: Union[pathlib.Path, str],
     output: Optional[Union[pathlib.Path, str]] = None,
@@ -103,8 +55,9 @@ def run(
         None,
         Literal["ggir", "gradient"],
     ] = "gradient",
-    epoch_length: Union[float, None] = 5,
+    epoch_length: float = 5,
     activity_metric: Literal["enmo", "mad", "ag_count"] = "enmo",
+    nonwear_algorithm: Sequence[Literal["ggir", "cta", "detach"]] = ["ggir"],
     verbosity: int = logging.WARNING,
     output_filetype: Optional[Literal[".csv", ".parquet"]] = None,
 ) -> Union[models.OrchestratorResults, Dict[str, models.OrchestratorResults]]:
@@ -128,10 +81,9 @@ def run(
             Default values are optimized for subjects ages 7-11 [1].
         calibrator: The calibrator to be used on the input data.
         epoch_length: The temporal resolution in seconds, the data will be down sampled
-            to. If None is given, and `enmo` is the chosen physical activity metric,
-            no down sampling is preformed. Otherwise, for `mad` and `ag_count`, a
-            ValueError will be raised.
+            to. It must be > 0.0.
         activity_metric: The metric to be used for physical activity categorization.
+        nonwear_algorithm: The algorithm to be used for nonwear detection.
         verbosity: The logging level for the logger.
         output_filetype: Specifies the data format for the save files. Must be None when
             processing files, must be a valid file type when processing directories.
@@ -188,6 +140,7 @@ def run(
             epoch_length=epoch_length,
             activity_metric=activity_metric,
             verbosity=verbosity,
+            nonwear_algorithm=nonwear_algorithm,
         )
 
     return _run_directory(
@@ -198,6 +151,7 @@ def run(
         epoch_length=epoch_length,
         verbosity=verbosity,
         output_filetype=output_filetype,
+        nonwear_algorithm=nonwear_algorithm,
     )
 
 
@@ -209,7 +163,8 @@ def _run_directory(
         None,
         Literal["ggir", "gradient"],
     ] = "gradient",
-    epoch_length: Union[float, None] = 5,
+    epoch_length: float = 5,
+    nonwear_algorithm: Sequence[Literal["ggir", "cta", "detach"]] = ["ggir"],
     verbosity: int = logging.WARNING,
     output_filetype: Optional[Literal[".csv", ".parquet"]] = None,
 ) -> Dict[str, models.OrchestratorResults]:
@@ -230,9 +185,8 @@ def _run_directory(
             Default values are optimized for subjects ages 7-11 [1].
         calibrator: The calibrator to be used on the input data.
         epoch_length: The temporal resolution in seconds, the data will be down sampled
-            to. If None is given, and `enmo` is the chosen physical activity metric,
-            no down sampling is preformed. Otherwise, for `mad` and `ag_count`, a
-            ValueError will be raised.
+            to. It must be > 0.0.
+        nonwear_algorithm: The algorithm to be used for nonwear detection.
         verbosity: The logging level for the logger.
         output_filetype: Specifies the data format for the save files.
 
@@ -291,6 +245,7 @@ def _run_directory(
                 calibrator=calibrator,
                 epoch_length=epoch_length,
                 verbosity=verbosity,
+                nonwear_algorithm=nonwear_algorithm,
             )
         except Exception as e:
             logger.error("Did not run file: %s, Error: %s", file, e)
@@ -306,8 +261,9 @@ def _run_file(
         None,
         Literal["ggir", "gradient"],
     ] = "gradient",
-    epoch_length: Union[float, None] = 5,
+    epoch_length: float = 5.0,
     activity_metric: Literal["enmo", "mad", "ag_count"] = "enmo",
+    nonwear_algorithm: Sequence[Literal["ggir", "cta", "detach"]] = ["ggir"],
     verbosity: int = logging.WARNING,
 ) -> models.OrchestratorResults:
     """Runs main processing steps for wristpy and returns data for analysis.
@@ -328,15 +284,19 @@ def _run_file(
             Default values are optimized for subjects ages 7-11 [1].
         calibrator: The calibrator to be used on the input data.
         epoch_length: The temporal resolution in seconds, the data will be down sampled
-            to. If None is given, and `enmo` is the chosen physical activity metric,
-            no down sampling is preformed. Otherwise, for `mad` and `ag_count`, a
-            ValueError will be raised.
+            to. It must be > 0.0.
         activity_metric: The metric to be used for physical activity categorization.
+        nonwear_algorithm: The algorithm to be used for nonwear detection. A sequence of
+            algorithms can be provided. If so, a majority vote will be taken.
         verbosity: The logging level for the logger.
 
     Returns:
         All calculated data in a save ready format as a OrchestratorResults object.
 
+    Raises:
+        ValueError: If an invalid Calibrator is chosen.
+        ValueError: If epoch_length is not greater than 0.
+
     References:
         [1] Hildebrand, M., et al. (2014). Age group comparability of raw accelerometer
         output from wrist- and hip-worn monitors. Medicine and Science in Sports and
@@ -358,6 +318,11 @@ def _run_file(
         logger.error(msg)
         raise ValueError(msg)
 
+    if epoch_length <= 0:
+        msg = "Epoch_length must be greater than 0."
+        logger.error(msg)
+        raise ValueError(msg)
+
     watch_data = readers.read_watch_data(input)
 
     if calibrator is None:
@@ -386,10 +351,18 @@ def _run_file(
     sleep_detector = analytics.GgirSleepDetection(anglez)
     sleep_windows = sleep_detector.run_sleep_detection()
 
-    non_wear_array = metrics.detect_nonwear(calibrated_acceleration)
+    nonwear_array = nonwear_utils.get_nonwear_measurements(
+        calibrated_acceleration=calibrated_acceleration,
+        temperature=watch_data.temperature,
+        non_wear_algorithms=nonwear_algorithm,
+    )
+    nonwear_epoch = nonwear_utils.nonwear_array_cleanup(
+        nonwear_array=nonwear_array,
+        reference_measurement=activity_measurement,
+        epoch_length=epoch_length,
+    )
     physical_activity_levels = analytics.compute_physical_activty_categories(
-        activity_measurement,
-        thresholds,
+        activity_measurement, thresholds
     )
 
     sleep_array = models.Measurement(
@@ -398,14 +371,6 @@ def _run_file(
         ),
         time=activity_measurement.time,
     )
-    nonwear_epoch = models.Measurement(
-        measurements=format_nonwear_data(
-            nonwear_data=non_wear_array,
-            reference_measure=activity_measurement,
-            original_temporal_resolution=900,
-        ),
-        time=activity_measurement.time,
-    )
 
     results = models.OrchestratorResults(
         physical_activity_metric=activity_measurement,
@@ -437,7 +402,7 @@ def _run_file(
 def _compute_activity(
     acceleration: models.Measurement,
     activity_metric: Literal["ag_count", "mad", "enmo"],
-    epoch_length: Union[float, None],
+    epoch_length: float,
 ) -> models.Measurement:
     """This is a helper function to organize the computation of the activity metric.
 
@@ -452,19 +417,12 @@ def _compute_activity(
 
     Returns:
         A Measurement object with the computed physical activity metric.
-
-    Raises:
-        ValueError: If the activity_metric is 'ag_count' or 'mad' and epoch_length is
-            None.
     """
-    if activity_metric in ("ag_count", "mad") and epoch_length is None:
-        raise ValueError("If using 'ag_count' or 'mad', epoch_length must be provided.")
-
     if activity_metric == "ag_count":
         return metrics.actigraph_activity_counts(
             acceleration,
-            epoch_length=epoch_length,  # type: ignore[arg-type] # Guarded by the ValueError statement above.
+            epoch_length=epoch_length,
         )
     elif activity_metric == "mad":
-        return metrics.mean_amplitude_deviation(acceleration, epoch_length=epoch_length)  # type: ignore[arg-type] # Guarded by the ValueError statement above.
+        return metrics.mean_amplitude_deviation(acceleration, epoch_length=epoch_length)
     return metrics.euclidean_norm_minus_one(acceleration, epoch_length=epoch_length)