Open-rppg

Open-rppg is a comprehensive Python toolbox designed for Remote Photoplethysmography (rPPG) inference. It provides a unified interface for state-of-the-art deep learning models, enabling physiological signal measurement (such as heart rate and heart rate variability) from facial videos. The toolkit supports both offline video processing and low-latency real-time inference using JAX.

API document https://kegangwangccnu.github.io/open-rppg/

Installation

This package requires Python 3.9 through 3.13.

To install the standard version:

pip install open-rppg

To enable GPU acceleration (Linux/CUDA), install the CUDA-supported version of JAX:

pip install jax[cuda]

Quick Start

The core interface is managed through the Model class. By default, it initializes a robust, general-purpose model (FacePhys.rlap).

import rppg

# Initialize the model
model = rppg.Model()

# Process a video file
results = model.process_video("path/to/video.mkv")

# Display the heart rate
print(f"Estimated Heart Rate: {results['hr']} BPM")

Usage Guide

1. Offline Video Processing

To analyze a pre-recorded video file, use the process_video method. This method handles frame extraction, face detection, and signal inference automatically.

results = model.process_video("subject_test.mkv")

Output Structure: The returned dictionary contains the following keys:

• hr: Heart Rate estimated via the frequency domain (FFT).
• SQI: Signal Quality Index (0.0 to 1.0), indicating the reliability of the measurement.
• latency: Inference latency (primarily relevant for real-time streams).
• hrv: A dictionary of Heart Rate Variability metrics calculated in the time domain:
  • bpm: Heart rate derived from peak detection.
  • ibi: Inter-Beat Interval (milliseconds).
  • sdnn: Standard deviation of NN intervals.
  • rmssd: Root mean square of successive differences.
  • pnn50: Proportion of NN50 > 50ms.
  • LF/HF: Ratio of Low Frequency to High Frequency power.
  • breathingrate: Estimated respiration rate.

2. Real-Time Inference

Open-rppg includes a threaded pipeline optimized for real-time webcam inference. Use the video_capture context manager to handle the video stream safely.

import rppg
import time
import cv2

model = rppg.Model()
# Open the default camera (index 0)
with model.video_capture(0):
    last_process_time = 0
    current_hr = None
    
    # Iterate through the preview generator (this is the main loop)
    for frame, box in model.preview:
        frame = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)
        
        # 1. Calculate HR every 1 second to avoid lag
        now = time.time()
        if now - last_process_time > 1.0:
            result = model.hr(start=-10)
            if result and result['hr']:
                current_hr = result['hr']
                print(f"Real-time HR: {current_hr:.1f} BPM")
            last_process_time = now
            
        # 2. Visualization
        if box is not None:
            # box format: [[row_min, row_max], [col_min, col_max]]
            y1, y2 = box[0]
            x1, x2 = box[1]
            cv2.rectangle(frame, (x1, y1), (x2, y2), (0, 255, 0), 2)
            
            # Display HR on the frame if available
            if current_hr is not None:
                cv2.putText(frame, f"HR: {current_hr:.1f}", (x1, y1 - 10), 
                           cv2.FONT_HERSHEY_SIMPLEX, 0.9, (0, 255, 0), 2)
        
        cv2.imshow("rPPG Monitor", frame)
        
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break

Advanced API

Retrieving Raw Signals

You can extract the underlying Blood Volume Pulse (BVP) waveform for further analysis or plotting.

# Retrieve the full BVP signal and corresponding timestamps
bvp, timestamps = model.bvp()

# Retrieve the raw, unfiltered BVP signal
raw_bvp, timestamps = model.bvp(raw=True)

Time Slicing

The toolbox allows specific time windows to be analyzed within the buffer.

# Get signal from t=10s to t=20s
bvp_slice, ts_slice = model.bvp(start=10, end=20)

# Get metrics for the last 15 seconds
metrics = model.hr(start=-15)

Tensor Inputs

For integration into existing pipelines where frames are already loaded as memory arrays, use the tensor processing methods.

• Input Format: uint8 array with shape (Frames, Height, Width, 3).

import numpy as np

# tensor shape: (T, H, W, 3)
video_tensor = np.zeros((300, 480, 640, 3), dtype='uint8') # 480p video

result = model.process_video_tensor(video_tensor, fps=30.0)

faces_tensor = np.zeros((300, 128, 128, 3), dtype='uint8') # face array

result = model.process_faces_tensor(faces_tensor, fps=30.0)

Model Selection

You can specify different architectures during initialization. The models are categorized by architecture and training configuration (rlap or pure).

# Example: Initialize the PhysMamba model
model = rppg.Model('PhysMamba.pure')

Model Zoo

The following architectures are supported.

Model Name	Description	Reference
ME-chunk	State-space model rPPG (chunk inference)	arXiv 2025
ME-flow	State-space model rPPG (low-latency flow)	arXiv 2025
PhysMamba	Dual-branch Mamba architecture	CCBR 2024
RhythmMamba	Frequency-domain constrained Mamba	AAAI 2025
PhysFormer	Temporal Difference Transformer	CVPR 2022
TSCAN	Temporal Shift Convolutional Attention Network	NeurIPS 2020
EfficientPhys	Self-attention variant of TSCAN	WACV 2023
PhysNet	3D Convolutional Encoder-Decoder	BMVC 2019
FacePhys	Optimized state-space model	-

Note: Suffixes .rlap and .pure indicate different training protocols/weights.

Licensing

The source code and tools in this repository are released under the MIT License.

Important: Pretrained models and model configurations provided in this repository are derived from academic research. They are the intellectual property of their respective authors and are subject to the license terms specified in their original publications. Please refer to the citations below for details.

Citation

If you use this toolkit or the included models in your research, please cite the relevant papers:

@article{yu2019remote,
  title={Remote photoplethysmograph signal measurement from facial videos using spatio-temporal networks},
  author={Yu, Zitong and Li, Xiaobai and Zhao, Guoying},
  journal={arXiv preprint arXiv:1905.02419},
  year={2019}
}

@article{liu2020multi,
  title={Multi-task temporal shift attention networks for on-device contactless vitals measurement},
  author={Liu, Xin and Fromm, Josh and Patel, Shwetak and McDuff, Daniel},
  journal={Advances in Neural Information Processing Systems},
  volume={33},
  pages={19400--19411},
  year={2020}
}

@inproceedings{liu2023efficientphys,
  title={Efficientphys: Enabling simple, fast and accurate camera-based cardiac measurement},
  author={Liu, Xin and Hill, Brian and Jiang, Ziheng and Patel, Shwetak and McDuff, Daniel},
  booktitle={Proceedings of the IEEE/CVF winter conference on applications of computer vision},
  pages={5008--5017},
  year={2023}
}

@inproceedings{yu2022physformer,
  title={Physformer: Facial video-based physiological measurement with temporal difference transformer},
  author={Yu, Zitong and Shen, Yuming and Shi, Jingang and Zhao, Hengshuang and Torr, Philip HS and Zhao, Guoying},
  booktitle={Proceedings of the IEEE/CVF conference on computer vision and pattern recognition},
  pages={4186--4196},
  year={2022}
}

@inproceedings{luo2024physmamba,
  title={PhysMamba: Efficient Remote Physiological Measurement with SlowFast Temporal Difference Mamba},
  author={Luo, Chaoqi and Xie, Yiping and Yu, Zitong},
  booktitle={Chinese Conference on Biometric Recognition},
  pages={248--259},
  year={2024},
  organization={Springer}
}

@inproceedings{zou2025rhythmmamba,
  title={RhythmMamba: Fast, Lightweight, and Accurate Remote Physiological Measurement},
  author={Zou, Bochao and Guo, Zizheng and Hu, Xiaocheng and Ma, Huimin},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
  volume={39},
  number={10},
  pages={11077--11085},
  year={2025}
}

@article{wang2025memory,
  title={Memory-efficient Low-latency Remote Photoplethysmography through Temporal-Spatial State Space Duality},
  author={Wang, Kegang and Tang, Jiankai and Fan, Yuxuan and Ji, Jiatong and Shi, Yuanchun and Wang, Yuntao},
  journal={arXiv preprint arXiv:2504.01774},
  year={2025}
}