feature/issue-28/implement-batch-processing

README.md

@@ -13,6 +13,131 @@ A Python toolkit for analysis of graphomotor data collected via Curious.
 
 Welcome to `graphomotor`, a specialized Python library for analyzing graphomotor data collected via [Curious](https://www.gettingcurious.com/). This toolkit provides comprehensive tools for processing, analyzing, and visualizing data from various graphomotor assessment tasks including spiral drawing, trails making, alphabetic writing, digit symbol substitution, and the Rey-Osterrieth Complex Figure Test.
 
+## Feature Extraction Capabilities
+
+The toolkit extracts clinically relevant metrics from digitized drawing data. Currently implemented features include:
+
+- **Temporal Features**: Task completion duration.
+- **Velocity Features**: Velocity analysis including linear, radial, and angular velocity components with statistical measures (sum, median, variation, skewness, kurtosis).
+- **Distance Features**: Spatial accuracy measurements using Hausdorff distance metrics with temporal normalizations and segment-specific analysis.
+- **Drawing Error Features**: Area under the curve (AUC) calculations between drawn paths and ideal reference trajectories to quantify spatial accuracy.
+
+## Installation
+
+Install the graphomotor package from PyPI:
+
+```sh
+pip install graphomotor
+```
+
+Or install the latest development version directly from GitHub:
+
+```sh
+pip install git+https://github.com/childmindresearch/graphomotor
+```
+
+## Quick Start
+
+> **⚠️ This implementation requires data to adhere to a specific format matching the standard output from [Curious drawing responses](https://mindlogger.atlassian.net/servicedesk/customer/portal/3/article/859242501).**
+
+Currently, `graphomotor` is available as an importable Python library. CLI functionality is planned for future releases.
+
+### Extracting Features from Spiral Drawing Data
+
+#### Single File Processing
+
+```python
+from graphomotor.core import orchestrator
+
+# Path to your spiral drawing data file
+input_file = "path/to/your/spiral_data.csv"
+
+# Option 1: Process file without saving any CSV file
+# Only return the DataFrame with extracted features
+features_df = orchestrator.run_pipeline(
+    input_path=input_file
+)
+
+# Features are returned as a pandas DataFrame with source file as index
+print(f"Extracted features: {list(features_df.columns)}")
+
+# Access the single file's data (features_df has one row)
+file_path = features_df.index[0]
+print(f"File: {file_path}")
+print(f"Participant: {features_df.loc[file_path, 'participant_id']}")
+print(f"Task: {features_df.loc[file_path, 'task']}")
+print(f"Hand: {features_df.loc[file_path, 'hand']}")
+print(f"Duration: {features_df.loc[file_path, 'duration']}")
+```
+
+```python
+# Option 2: Save to a directory with auto-generated filename
+# Creates a CSV file with auto-generated name in the specified directory
+# Format: {participant_id}_{task}_{hand}_features_{YYYYMMDD_HHMM}.csv
+features_df = orchestrator.run_pipeline(
+    input_path=input_file,
+    output_path="path/to/output/directory"
+)
+```
+
+```python
+# Option 3: Save to a specific CSV file
+# Features will be saved to the specified file path
+features_df = orchestrator.run_pipeline(
+    input_path=input_file,
+    output_path="path/to/features.csv"
+)
+```
+
+#### Batch Processing
+
+```python
+from graphomotor.core import orchestrator
+
+# Path to directory containing multiple spiral drawing data files
+input_dir = "path/to/your/spiral_data_directory"
+
+# Option 1: Process files without saving any CSV files
+# Only return the DataFrame with extracted features
+features_df = orchestrator.run_pipeline(
+    input_path=input_dir,
+)
+
+# Features are returned as a pandas DataFrame with source files as index
+# Columns include: participant_id, task, hand, start_time, and calculated features
+print(f"Successfully processed {len(features_df)} files")
+
+# Access metadata and features for a specific file
+for file_path in features_df.index:
+    print(f"File: {file_path}")
+    print(f"Participant: {features_df.loc[file_path, 'participant_id']}")
+    print(f"Task: {features_df.loc[file_path, 'task']}")
+    print(f"Hand: {features_df.loc[file_path, 'hand']}")
+    print(f"Duration: {features_df.loc[file_path, 'duration']}")
+
+```
+
+```python
+# Option 2: Save to a directory with auto-generated filename
+# Creates a single consolidated CSV file with auto-generated name
+# Format: batch_features_{YYYYMMDD_HHMM}.csv
+features_df = orchestrator.run_pipeline(
+    input_path=input_dir,
+    output_path="path/to/output/directory"
+)
+```
+
+```python
+# Option 3: Save to a specific CSV file (single consolidated file)
+# All features will be written to one specified file
+features_df = orchestrator.run_pipeline(
+    input_path=input_dir,
+    output_path="path/to/consolidated_features.csv"
+)
+```
+
+For detailed configuration options and additional parameters, refer to the [`run_pipeline` documentation](https://childmindresearch.github.io/graphomotor/graphomotor/core/orchestrator.html#run_pipeline).
+
 ## Development Progress
 
 ⚠️ **This package is under active development.** Currently, the focus is on the Spiral task. After finalizing feature extraction, the next steps will involve implementing both preprocessing and visualization for this task. Once these parts are in place, we plan to extend support to other tasks.
@@ -27,8 +152,6 @@ Welcome to `graphomotor`, a specialized Python library for analyzing graphomotor
 
 ## Data Format Requirements
 
-⚠️ **This implementation requires data to adhere to a specific format matching the standard output from [Curious drawing responses](https://mindlogger.atlassian.net/servicedesk/customer/portal/3/article/859242501).**
-
 When exporting drawing data from Curious, you typically receive the following files:
 
 - **report.csv**: Contains the participants' actual responses.
@@ -67,58 +190,6 @@ line_number, x, y, UTC_Timestamp, seconds, epoch_time_in_seconds_start
 
 This format represents the standard output from [Curious drawing responses data dictionary](https://mindlogger.atlassian.net/servicedesk/customer/portal/3/article/596082739).
 
-## Feature Extraction Capabilities
-
-The toolkit extracts clinically relevant metrics from digitized drawing data. Currently implemented features include:
-
-- **Temporal Features**: Task completion duration.
-- **Velocity Features**: Velocity analysis including linear, radial, and angular velocity components with statistical measures (sum, median, variation, skewness, kurtosis).
-- **Distance Features**: Spatial accuracy measurements using Hausdorff distance metrics with temporal normalizations and segment-specific analysis.
-- **Drawing Error Features**: Area under the curve (AUC) calculations between drawn paths and ideal reference trajectories to quantify spatial accuracy.
-
-## Installation
-
-Install the graphomotor package from PyPI:
-
-```sh
-pip install graphomotor
-```
-
-Or install the latest development version directly from GitHub:
-
-```sh
-pip install git+https://github.com/childmindresearch/graphomotor
-```
-
-## Quick Start
-
-Currently, `graphomotor` is available as an importable Python library. CLI functionality is planned for future releases.
-
-### Extracting Features from Spiral Drawing Data
-
-```python
-from graphomotor.core import orchestrator
-
-# Path to your spiral drawing data file
-input_file = "path/to/your/spiral_data.csv"
-
-# Directory where extracted features will be saved
-output_dir = "path/to/output/directory"
-
-# Run the analysis pipeline
-features = orchestrator.run_pipeline(
-    input_path=input_file,
-    output_path=output_dir
-)
-
-# Features are returned as a dictionary and saved as CSV
-print(f"Successfully extracted {len(features)} feature categories")
-```
-
-For detailed configuration options and additional parameters, refer to the [`run_pipeline` documentation](https://childmindresearch.github.io/graphomotor/graphomotor/core/orchestrator.html#run_pipeline).
-
-> **Note:** Currently, only single file processing is supported, with batch processing planned for future releases.
-
 ## Future Directions
 
 The Graphomotor Study Toolkit is under active development. For more detailed information about upcoming features and development plans, please refer to our [GitHub Issues](https://github.com/childmindresearch/graphomotor/issues) page.

pyproject.toml

@@ -13,7 +13,8 @@ dependencies = [
   "pandas>=2.2.3",
   "pydantic>=2.11.1",
   "scipy>=1.15.2",
-  "shapely>=2.1.0"
+  "shapely>=2.1.0",
+  "tqdm>=4.66.0"
 ]
 
 [dependency-groups]

src/graphomotor/core/models.py

@@ -18,7 +18,8 @@ class Spiral(pydantic.BaseModel):
             - id: Unique identifier for the participant,
             - hand: Hand used ('Dom' for dominant, 'NonDom' for non-dominant),
             - task: Task name,
-            - start_time: Start time of drawing.
+            - start_time: Start time of drawing,
+            - source_path: Path to the source CSV file.
     """
 
     model_config = pydantic.ConfigDict(arbitrary_types_allowed=True)