Enhanced remote photoplethysmography using hybrid RGB and near-infrared cameras

Abstract

This study investigates the enhancement of remote photoplethysmography (rPPG) using a hybrid system that integrates RGB and near-infrared (NIR) cameras. rPPG, a non-contact method for heart rate (HR) measurement, relies on analyzing subtle color changes in facial videos. We compare traditional signal processing methods, such as independent component analysis and principal component analysis, with advanced deep learning models such as DeepPhys and PhysFormer. Traditional methods often struggle with lighting variations and motion artifacts, whereas deep learning approaches demonstrate improved robustness and accuracy. Our hybrid approach leverages the strengths of both RGB and NIR cameras, addressing their individual limitations. We propose two methods, the first is a four-channel method, which overlays RGB and NIR frames to create a four-channel image, and the second is an ensemble method, which applies weights to the tokens extracted from each video and then sums them. These hybrid approaches significantly enhance performance, achieving higher percentage of time with error within 6 BPM (PTE6) scores across various scenarios, including still conditions and motion environments. PTE6 represents the proportion of the entire timeline where the predicted HR is within six beats per minute (BPM) of the actual HR and is a commonly used metric in HR prediction studies. For example, the four-channel method achieved a PTE6 score of 72.89% in still conditions and 56.73% in motion scenarios, outperforming the single-camera setups. Specifically, the RGB camera achieved 70.97% and the NIR camera 67.89% in still conditions, whereas in motion scenarios, the RGB camera scored 53.24% and the NIR camera 37.99%. These findings highlight the potential of combining RGB and NIR cameras to improve the accuracy and reliability of rPPG. This approach opens new avenues for advanced telemedicine applications, providing a robust solution for continuous health monitoring in diverse environments. (c) 2024 SPIE and IS&T