Remote Photoplethysmography (rPPG): A State-of-the-Art Review

Pireh Pirzada, Adriana Wilde, Gayle H Doherty, David Harris-Birtill
doi: https://doi.org/10.1101/2023.10.12.23296882

Abstract

Peripheral oxygen saturation (SpO2) and heart rate (HR) are critical physiological measures that clinicians need to observe to decide on an emergency intervention. These measures are typically determined using a contact-based pulse oximeter. This approach may pose difficulties in many cases, such as with young children, patients with burnt or sensitive skin, cognitive impairments, and those undergoing certain medical procedures or severe illnesses. Remote Photoplethysmography (rPPG) allows for unobtrusive sensing of these vital signs in a variety of settings for health monitoring systems. Several research studies have been conducted to use rPPG for this purpose; however, there is still not a commercially available, clinically validated system that overcomes the concerns highlighted in this paper. We present a state-of-the-art review of rPPG-related research conducted including related processes and techniques, such as regions of interest (ROI) selection, extracting the raw signal, pre-processing data, applying noise reduction algorithms, Fast Fourier transforms (FFT), filtering and extracting these vital signs. Further, we present a detailed, critical evaluation of available rPPG systems. Limitations and future directions have also been identified to aid rPPG researchers in further advancing this field.

I. Introduction

Heart Rate (HR) and blood oxygen saturation (SpO2) are physiological signs that are used as critical indicators of human health [1]. Also called vital signs, these typically become the first clinical alerts to a significant change in a person’s health. This is due to the existence of many complex relationships between these signs and the components of underlying circulatory and neuro-hormonal systems [1].

The first of these signs, HR, is a measure of cardiac activity, critically important in the assessment of emergent ill-health [2]. For example, a decrease in HR may be the result of an increase in intracranial pressure [3], whereas an increase may reflect hypovolemic shock [4], both medical emergencies [3], [4]. Normal values for HR in resting state lie between 60-100 BPM, with lower values being referred to as ‘bradycardia’, and higher ones, ‘tachycardia’ [5], [2].

The second of these signs, SpO2, is a broad measure of the levels of blood oxygenation, which can be defined as the percentage of Haemoglobin (Hb) in the blood carrying oxygen. When a person breathes in, oxygen enters the lungs and then transfers across the alveolar membrane and into the bloodstream [6]. The function of the circulatory system is to transfer this oxygen to all of the organs and cells of the body [7]. Once the oxygen has been off-loaded, the Hb becomes deoxygenated haemoglobin (also known as deoxy-Hb) [7], eventually returning to the heart which then sends it on to the lungs to acquire more oxygen, and so the process continues [2]. An abnormal change in SpO2 can have a serious effect on a person’s organs and is often associated with disease, typically of the lungs and/or of the heart [7]. These include pneumonia [8], [9], asthma [10], Chronic obstructive pulmonary disease (COPD) [11], [12], drug overdose [13], hypoxia [13], respiratory [14] or heart diseases [15]. A SpO2 value below 94% is considered to be abnormal for most people and indicates a significant respiratory illness, whilst some individuals with stable chronic lung disease may have a reading of 88-94% [16]. Monitoring changes in SpO2 can help identify complications including respiratory failure [14].

A. Measuring HR and SpO2

Pulse oximetry is a widely used method for measuring HR and SpO2 [17], [18], [19]. It is based on the photoplethys-mography (PPG) principles first introduced by Hertzman in 1937 [20]. PPG measures blood flow by calculating changes in the dispersion of light [21], [22]. This technique uses a Red and Infrared (IR) light-emitting diode (LED) and a photodetector, with a person’s finger in between. This setup allows two wavelengths of light to travel through the finger to a photodetector, which detects light not absorbed by the finger. The light absorption is governed by Beer’s law [23], whereby light is absorbed proportionately to the concentration of the light-absorbing material, such as Hb and deoxy-Hb, which means that the higher their concentration, the more light is absorbed. Lambert’s law also applies, whereby light absorbed is proportionate to the distance of the light path. This means that more light is absorbed when it travels through a longer path. As light-absorbing materials, Hb and deoxy-Hb differ: Hb absorbs more IR light than deoxy-Hb while deoxy-Hb absorbs more red light, as shown in Fig. 1. Pulse oximeter LEDs emit light at 660 nm (red) and 940 nm (Infrared) wavelengths [24]. Fig. 1 shows that at 650 nm deoxy-Hb absorbs more light than Hb, while at 940 nm the opposite is true. These physical properties are used by a pulse oximeter to measure the varying absorbance at each of the wavelengths [23].

Understanding these vitals can give important information about a person’s health and its result and performance impact clinical decisions [26]. Another form of measuring HR is via an electrocardiogram (ECG), used to characterise the cardiac rhythm in detail. However, this requires being fixed on location, typically using gel patches or chest straps [27], which might require shaving body hair. The extra adherent tape is applied to keep the electrode pad in place. This can irritate the skin and cause allergic reactions [28], [29], especially in long-term monitoring [29]. Whereas, SpO2 is currently assessed either by a simple method such as a pulse oximeter finger probe placed around the tip of the finger or by more invasive procedures such as Arterial Blood Gas (ABG) measurement where a needle is used to draw blood from an artery in the wrist [30]. Using the ABG method of getting true SpO2 can be painful and carries significant risks of infection [30], [16]. In the case of neonatal where regular sampling is required, it has a high risk of blood loss [31]. In addition to that, this process is also time-consuming [32].

Fig. 1.
Fig. 1.

The optical extinction spectra of oxy-Hb and deoxy-Hb within the blood. Figure generated using data by Prahl [25].

A pulse oximeter device uses a clip that is mounted on a person’s fingertip1 or uses a strap form for example where it is wrapped on a baby’s foot2. Monitoring HR and SpO2 is also possible in other forms such as a wrist band3 [33], [34] or a smartwatch4, a clip that can be attached to an ear5 [35], or stick to the forehead6 [36]. Generally, a pulse oximeter clip which is mounted on a finger is the most common, accessible and cheaper form of measuring the vitals (clinical or commercial) [31], [37], [18], [19]. It is a simple and common device used in a medical setting, with relatively low intrusiveness [38], [37], [18], [19]. Recently, physicians worldwide have suggested using portable pulse oximeters for self-evaluating health vitals [39]. It plays a vital role in indicating shortness of breath related to COVID-19 [39]. It is not only widely used in clinics and hospitals [40] such as ICU but also for sports [41], [42] and within homes [43], [44], [45] due to the wide availability of commercial pulse oximeter devices. Commercial pulse oximeter devices even come with facilities that allow Bluetooth connection and their applications to enable pairing to smartphones, record vitals and store the vital history of a person [46], [47], [48], [49].

B. Limitations of contact-based pulse oximetry

Although effective, pulse oximetry poses several problems as a monitoring tool including being open to significant error [30], [16]. The pulse oximeter clip has to be physically mounted on the finger of a person, which can be difficult to manage for adults or for patients with any form of incapacity [50] for example burnt skin [51], cognitive challenges [50], [51], during surgery [51], young children [31], sensitive skin [28] and in those with severe illness of any cause [28]. It can also be challenging to work with children or adults with physical incapacity [50]. In addition to that, it may be cumbersome to wear for long periods of time [52], restricting movement if wire-based pulse oximeter devices are used [36] and providing a constant physical reminder that their health is being monitored [28], [48].It also carries an increased risk of cross-infection [53].

Moreover, this presents a potential hazard with a high risk of choking among infants when using devices with wires [54]. Along with that, pulse oximeters have also been reported to have an increase in errors when using nail varnish [7], [50], pigmentation of skin [7], [50], access movement[7], [50], in cases where abnormal Hb and Carboxyhaemoglobin (COHb) is present and intravascular dyes; it also reduces accuracy for SpO2 that is below 83% [7]. Previous research shows that different skin pigmentation especially those with darker skin such as black or brown (i.e. with higher values in the Fitzpatrick scale7) had an increased error rate [57], [58]. For example, one study found a substantial bias in pulse oximeter devices used with people with darker skin pigmentation (IV-VI type from Fitzpatrick scale) had a higher error rate (5.1% ±9.2%) in comparison to white skin pigmentation (I-III type from Fitzpatrick scale) (1.9% ±10.2%) for SpO2 [59], [60]. Another study reported an increased error rate of 3.56 ±2.45% among darker skin pigmentation participants compared to white skin pigmentation participants with an error rate of 0.37 ±3.20% SpO2 [61]. Also, in a research study with anaemic participants during hypoxemia, it was observed that the error rate for the anaemic participants was 15% compared to 6.4% among non-anaemic participants [59], [40]. Different studies have been conducted where participants applied different colours of nail paint. Past studies reported conflicting results that is, some studies discovered that nail paint reduces SpO2 values [62], [63], whereas others found no effect of applying nail paint [64], [65]. One study stated that black, blue, and green nail paint decreased SpO2 reading by 3 to 6%, whereas blue nail paint decreased it from 97% to 87% [62].

C. Remote Photoplethysmography (rPPG)

rPPG is the measurement of the flow of blood by optical means, typically involving measurement of changes in the transmission or scattering of light created by blood flow in a part of the body [54]. These changes are also reflected on the face via subtle changes on the skin where the pulse flashes lighter and darker over time. This phenomenon is not visible to the naked eye; however, it can be detected by measuring the reflectance of light over a period of time from skin pixel intensity extracted from Region of Interests (ROIs) using a camera-based system [51].

rPPG system measures the HR by analysing the skin pixel intensity of the heartbeats over time; the skin flashes darker and lighter as more and less blood flows through the region. It measures SpO2 by using the optical absorption differences across the visible and near-infrared wavelength regions between Hb and deoxy-Hb [23], [66]. Various techniques from different research systems have been evaluated, critiqued with their gaps, summarised and presented in this research paper.

II. Research Motivation

Current contact-based methods have limitations which can be prohibitive for people with certain incapacities, severe illnesses, or burns as mentioned in section I-B which can be mitigated by developing and optimising rPPG systems. rPPG is an alternative method to measuring vital sign data such as HR and SpO2 over contact-based methods such as pulse oximetry. rPPG helps overcome the limitations of pulse oximetry which are stated in section I-B. It has the potential to revolutionise vital sign measurement by providing a more convenient and non-invasive approach to capturing physiological information from the human body without the need for physical contact. rPPG research field is expanding rapidly, but it predominantly concentrates on monitoring HR, neglecting the crucial aspect of simultaneously tracking HR and SpO2. An essential dimension of advancing these systems involves comprehending their performance in various settings, including laboratories, clinics, and homes. Unfortunately, there is currently no comprehensive review paper available that elucidates the rPPG (remote photoplethysmography) process for both HR and SpO2 and offers an in-depth analysis of the existing gaps within rPPG systems. This absence of information underscores the importance of identifying limitations and outlining future directions for researchers interested in further advancing rPPG research.

III. Rppg Process

Previous research studies have used a variety of different equipment and set-up to capture data from participants to obtain vital signs. Thermal camera [67], [68], [69], [70], Charge-Coupled Device (CCD) camera [71], [72], other affordable web cameras or those built-in laptops [73], [74], [75], [76], [77], [78], [79], [80], [81], MS Kinect V2 [82], [83], [66], Kinect Azure [84], GoPro camera with a drone [85] and smartphone [86], [87], [88], [89], [90], [91], [92], [93], [94] cameras have been previously used to obtain a person’s vital signs. Table V and Table IV show different studies conducted with various equipment and the vital sign under observation.

The most common steps followed in an rPPG system are detailed below:

1) Face identification and ROI detection

2) Signal extraction

3) Reducing noise from signal

4) Applying Fast Fourier Transform (FFT)

5) Applying frequency filter

6) Extracting vital sign

7) Reliability check

A. Face and ROI Detection

The first step involves identifying the person’s face from the frame. The research stationed participants in front of the camera in a static position to acquire face image frames. The participants are required to be in a specific location with no or minimal movement. In the case of minimal movement, ROIs were selected manually [95], [96], [97] whereas to cater to head movement or rotation of the head to some degree, various algorithms were used, among which Viola and Jones [98], [99] were common. One of the major reasons researchers used this library was due to its availability in the OpenCV. Researchers used a neural network-based classifier to detect ROI [100], [101], [102], a statistical model to match a person’s face to an image frame [103] and different algorithms to track features points for head movement in image frames [104]. To cater to the movement problem and identify the correct ROI, one way would be to detect face and ROI for every single frame. However, the Viola-Jones bounding box is not very accurate and can report false positives and negatives [104] along with additional computation power in the case of realtime applications [104]. Kanade-Lucas Tomasi (KLT) was used by different researchers to track feature points from a face to update ROI location [100], [105], [106], [107], [108]. Different parts of the body have been used as ROI to obtain vitals [56], [109], [110], [111], such as the entire face, forehead, and palm, as shown in Table V.

B. Signal Extraction

Once the ROI has been identified and extracted. The next step is to extract the raw signal from the ROI image data over a specific time period. The raw signal from a colour image includes channels RGB and these values obtained from each channel of the image frame are averaged over all pixels at a time period. For example, red channel pixels averaged for ROI image data for a time period of 60 seconds (sR(t), sG(t), sB(t)) where s is the signal obtained over time t for RGB channels. This process is known as spatial pooling, and it helps average the noise contained in each pixel [51], [28], [112], [113], [114].

C. Reducing Noise

After the raw signal has been obtained, the signal (S) containing data from all channels [S = sr, sg, sb] is passed through a Blind Source Separation (BSS), for example, Independent Component Analysis (ICA) or Principal Component Analysis (PCA) algorithm. This is applied as the signal can contain additional noise due to skin pigmentation, light source or any movement. Applying BSS helps extract a clearer signal from the raw signal input [51], [28], [112], [113], [115].

D. FFT and filter

FFT is then applied to a raw signal to convert it into a frequency domain and apply a filter to remove unwanted frequencies. This is to increase SNR to enhance the quality of the obtained signal [116], [117]. The most common filter is a bandpass filter to remove low and high frequencies, for example, 0.65Hz and 3.5 Hz [51], [28], [112], [113], [118], [119], [120].

E. Extracting Vital

Finally, vital sign data is to be extracted from the signal [51], [28], [112], [113], [121], [122], [123], [124], [125], [126]. A peak is identified from the signal to obtain HR vital data. Where HR is calculated by multiplying the frequency value on that peak by sixty (60) to convert it into BPM, and for Respiratory Rate (RR), it is multiplied by the ratio of the number of peaks over a time period. Researchers used different methods to obtain SpO2 from the pulse signal, which is mentioned in section V-B.

F. Reliability Check

Only two research papers defined unique methods using SNR ratio, and previous estimates within a history window to perform consistency and reliability check to ensure historybased reliability of the vital sign data [127], [66].

IV. Methodology

In order to identify relevant published work, we conducted a literature review. PubMed, MedlinePlus, ACM, IEEE Xplore and Springer Articles were searched using the following keywords: (“remote”, “non-contact”) and (“heart rate monitoring”, “blood oxygenation level”, “rPPG”). Literature was identified that fit the criteria detailed in Table I. The date range for the review was January 1990 – October 2022 (Figure 2). This was refined further by developing stringent exclusion criteria (also in Table I). A second round was made based using the same criteria, to include more recent research (October 2022 – September 2023). Through this round, 57 additional papers were identified.

View this table:
TABLE I Inclusion and Exclusion Criteria for rPPG Literature Review
Fig. 2.
Fig. 2.

Research articles identified for the literature review based on the criteria defined in Table I.

V. Literature Review Findings: Previous Work With Its Limitations

This section provides details of the previous research on an rPPG system, its performance, conditions, and limitations. It was found that most of the research focuses only on HR, whereas very few focus on SpO2 and only a limited number of studies focus on both. The system performance metrics (Root Mean Square Error (RMSE), r-correlation and Standard Deviation Error (σ)) of previous work are listed in Table III for HR and Table II for SpO2. Previous research conditions and their participant information are in Table V and Table IV.

View this table:
TABLE II Previous Systems evaluation measures for SpO2 performance metrics of Root mean Square error (RMSE)
View this table:
TABLE III Lab-based HR systems performance: Standard Deviation Error (σ), Root Mean Square Error (RMSE), and r correlation
View this table:
TABLE IV Previous Systems, FPS, ROI and Tracking detail
View this table:
TABLE V Previous Systems and Experiment Detail (n: number of participants)

A. Remote HR Measurement

rPPG was introduced and demonstrated by using a thermal camera to record and analyse face data [67], [68]. This was to extract information about cardiac pulse, blood flow, and breathing rate. The researchers tracked the face movement and used a blood vessel (blood vessel registration) as ROI to extract a thermal signal. After extracting the thermal signal and removing noise, a Fourier Transform was applied to obtain a pulse signal. The performance of the system was reported to be 88% [67]. However, the research does not state RMSE, r-correlation or Mean Absolute Error (MAE) values. In addition to that, the research data was gathered in a controlled illumination environment where the participants’ movement was restricted (still state). The system might not perform well in a real-life scenario as illumination may vary and participants are not restricted from movement. Along with that, this thermal equipment is expensive [104], [69] which can be difficult to deploy on smaller budgets [104], [69].

Another study was also conducted utilising a thermal camera that used the forehead as ROI to measure and analyse signal data to generate HR. During data acquisition, if the participant changed the head position in any direction, the system failed to capture data from the defined ROI [69]. Researchers used the Auto-Regressive Moving Average (ARMA) model to obtain HR from the signal [160]. The biggest drawback is that even if the signal provides a reasonable result, the participant’s movement in real-life can fail the system since it only tracks the forehead. In the case of a headscarf or longer hair, it is also highly likely to produce a reading that is not accurate. Along with that, a person’s Body Temperature (BT) and environmental temperature could impact the signal extracted from a thermal video. Again, the expensive thermal equipment would also make it infeasible to deploy on a limited budget [104], [69], [161].

Remote HR measurement has also been accomplished using a CCD camera. Researchers used thirty (30) mean seconds of image data captured from the camera and analysed the change in brightness on participants’ face regions from the cheeks. After analysing the signal data obtained from participants, HR peak was identified [71]. Another research used a CCD camera to remote measure HR. It focused on participants’ movement and noise from light using chrominance rPPG to remove specular reflection variation. They used a combination of red and green channels to extract signals from defined ROI [72]. Once again, both these studies were conducted in a controlled setting which used specific lighting. Deploying this system in a real-life scenario could impact the results (inaccurate readings) due to various types of illumination present in realistic environments. CCD camera used in this research is also expensive and bulky [104], [71].

One of the early investigations to obtain HR used a laptop camera to acquire face colour images consisting of RGB channels. ICA, FFT and a frequency filter were applied to the channel data (RGB) to obtain the pulse signal. The second component (green channel) was found to typically produce the strongest pulse signal in this research study [73], [162]. This research was further refined by adding signal preprocessing such as detrending and smoothing the signal data. The system was updated to select a component with the highest peak after applying FFT and filter, which was further smoothed by moving average filter [74]. However, the study was conducted within a lab setting (in a controlled environment) with only a limited number of participants (n=12). In addition to that, restricted movement within a controlled environment does not replicate realistic environments. The face tracker used in the study detected multiple or no faces which the system handled by using the previous face captured; however, this method can fail in a real-life setting as it may not capture the real vital reading over time (using the previous face for a certain time) resulting in inaccurate HR readings.

Different researchers used various ROIs from face images obtained from participants to measure HR remotely. Face ROIs included full face, forehead, lips, eyes or nose and cheeks, which varied in pixel size. These ROIs may contain regions which cannot provide information related to HR for example eyes, teeth, beards etc., which can impact the overall performance of results. In the case, where only the mouth area is focused to obtain HR, it can impact the error rate when a person is talking or laughing as teeth do not provide a pulse signal to obtain HR. To cater to this problem, researchers [100] introduced neural networks for skin classification [163] to determine skin colour from the face image. This ROI (skin pixel) was labelled and tracked using a mean-shift tracker, which iteratively shifts points of data to the average of points of data nearby [108]. The HR was obtained after applying FFT and utilised a data adjustment scheme which resulted in obtaining HR in a short time. This work was further extended using multiple video data, which reduced the error rate [164]. However, these data do not cater to the different light sources and movement of participants and the system has not been validated in a real-life scenario.

Researchers most commonly used PCA [148], [97], [165], [166], ICA [167], [100], [168], [169], [170], [153], [171], [172], FastICA [53], [173], [151], [133], [138], [105], [174], [100], [168], [170], [153], [175], [176], RobustICA [149], Joint Approximation Diagonalization of Eigenmatrices (JADE) [147] and also Laplacian Eigenmap (LE) algorithms [150] for extracting source signals from RGB channels to obtain HR. One of the research used LE to extract signal sources from the face data obtained from participants. LE is also a dimensionality reduction technique but is nonlinear to find the internal structure of the data. The research revealed better results in comparison to other dimensionality reduction techniques. However, this has not been tested on a larger dataset obtained from realistic environments. In addition to that, the subject movement was still restricted in this research study.

A research study used Eulerian video magnification to measure changes in the face region of the participants. The data was obtained in the form of a video that was spatially decomposed into various frequency bands to which temporal filtering was applied and finally amplified by a factor. This was implemented on data where participants were in a still state [177]. Recent studies followed a similar approach to estimate HR and RR from video data as well [178], [179]. Since in real life, a person would be in motion, it is not possible to extract HR signal using this method. In addition to that, this method may not work well when there is high noise present in the signal. Research conducted also used machine learning techniques to measure remote HR [180], [181], [182], [183], [184], [185], [186]. Where the researcher [130] expanded on the work done by Poh et al. [73]. However, the accuracy drops for participants in motion and under different light sources. To counter this, the researchers [130] created a study set up to include more movement among participants. Compared to Poh et al. [73], instead of choosing a single pulse component to identify a peak, Monkaresi et al. [130] used a machine learning approach to select the pulse component among all channels (RGB components).

This approach also included applying power spectrum analysis, k-Nearest Neighbour (KNN) and linear regression to each component extracted after applying ICA to obtain features and classify them. KNN outperformed linear regression for selecting pulse signal components to obtain HR. However, generalising the model among the participants can bear different results or not perform well when new participants are introduced to the system on which it has not been trained yet and may therefore fail in a real-life scenario. Furthermore, the study was conducted in a controlled environment with specific light conditions.

To cater to challenges associated with different illumination and participant movement, Li et al. [133] suggested a technique based on Normalised Least Mean Square (NLMS) adaptive filtering and face tracking using the Viola-Jones face detector [152] to overcome these challenges. NLMS is an extension of the Least Mean Square (LMS) adaptive filter, which is used to simulate the selected filter. NLMS was used on raw signals to improve the data impacted by different light variations. To identify and locate face landmarks Discriminative Response Map Fitting (DRMF) the method was used in the initial image frame. KLT [154], [187] algorithm was then used to continue tracking the ROI. Researchers used the MAHNOB-HCI [188], [189], [190], [191] dataset for research studies to test their techniques. The results from their research exceeded the accuracy of previous studies. However, the database contains data obtained in a controlled environment with only slight movement in a lab setting. ROI tracking also fails when participants move at an extensive angle and participants’ facial expressions created noise which then shows high variations in the obtained signal. Different researchers have used datasets collected from lab setups to test and validate their methods; However, these have not been validated in realistic environments [192]. Various datasets include PURE [193], MAHNOB-HCI [194], [195], COHFACE [196], MMSE-HR [197], BH-rPPG [198], MPSC-rPPG Dataset[199] and UBFC-RPPG [200] and ARPOS dataset [66], [201].

Another study [140] suggested a method to select the most suitable ROI pixels. The researchers segmented the face region into various sub-regions on the basis of lightness dispersal. The most suitable sub-regions were then chosen and grouped by assessing their SNR [140]. Another study focused on improving signal quality by reducing noise obtained from participants’ face data by applying the block-based spatial-temporal on the face data. The spatial-temporal quality dissemination of the face ROI was then calculated depending on SNR. Adaptive ROIs were obtained by computing mean shift clustering and adaptive thresholding SNR maps, which then increased the accuracy of estimating HR [202], [203]. However, the system validation is limited to only controlled lab-based environments and needs to be stress tested for realistic environments. Another study conducted using Kinect V2 used skeleton tracking to obtain HR and RR by monitoring the chest region movement with and without a blanket over a participant (infant). This was to monitor cardiopulmonary irregularities such as bradycardia to process unclear ROI under well-lit and dark light conditions with different sleeping positions. In case of abnormalities were detected, an alarm would be triggered to notify the carer [82]. However, the system may fail if a participant is sleeping on their stomach. Another constraint is the reliability of movement features to obtain the vital, which can create inaccurate observations for different types of motions such as walking, talking or micro facial gestures. These voluntary movements can impact the reliability of the vital data obtained. A robot with a video camera incorporated has also been used to obtain HR. The aim was to design a robot for older adults to do some exercise and measure their vital [193]. However, the accuracy drops with movement and is not inexpensive which can be difficult to deploy on budget, especially for those within the middle to lower-income households around the world.

Studies conducted restrict participants from any internal or external movement. External movement includes a person moving their head or even walking around in the room, whereas internal movement is the motion of facial attributes such as talking where the mouth region is moving (moving lips) etc. In a real-life situation, these movements would be expected, which can increase noise when calculating vital signs. Research focusing on the head motion of participants from a video has been addressed by researchers [96], [204], [205], [105], [206], [207], [208], [209], [210], [211], [212], [213], [214], [215], [216], [217], [218], [219], [220], [221], [222], [223]. Where one of the researchers suggested tracking velocities of feature points on the face using Viola-Jones face detector and then tracking ROI by using KLT [96]. While another research focused only on a single ROI, specifically the forehead [224], [225], [226], this was as micro expressions below the eye line are prone to more motion as compared to forehead [225]. Researchers have also addressed this movement-based HR monitoring by using Face Quality Assessment (FQA), where low-quality image frames can be discarded, so the noise is not contributing to the final obtained values. FQA considers four parameters, including resolution, luminance, sharpness, and face trajectory, to determine if the face image requires to be removed. To track face feature points Good Feature Tracking (GFT) method was used on the image data obtained. Landmarks from the face were obtained using the Supervised Descent Method (SDM) combined with GFT to obtain face feature points and track movement and hence provided improved signal quality to obtain HR [139]. However, these motions were restricted and conducted in a controlled manner within a lab setting. The research systems were deployed by the researchers on specific hardware (labbased systems with enough RAM and Graphics Processing Unit (GPU)) to apply these techniques with controlled guided participation which does not necessarily reflect a real-life environment. People from lower to middle-income backgrounds also cannot take advantage of the system requiring expensive hardware [227], [228]. In addition to that tracking, a specific ROI such as only the forehead from the face can fail where a participant’s full face is not in front of the camera. In addition to that various research methods focusing on measuring HR need to be validated within realistic environments to check how different factors such as illumination, facial hair (such as beard), makeup and FPS impact signal data used to obtain vital sign data [229], [230], [231], [232]. To increase SNR it has been suggested to use a monochromatic camera with a green range filter. This study also used weighted average on the various face ROI. A deformable face fitting algorithm and KLT tracking [154], [233] algorithm tracked and extracted face features [137]. However, the RMSE obtained from this research is very high as mentioned in Table III which shows that it is not a feasible method to apply in a real-life system. A recent study proposed a method that uses Eulerian Video Magnification (EVM), Quality Assessment (QA) of signal, and Adaptive Chirp Model Decomposition (ACMD) to obtain HR. They validated the system in different illuminations and with some head motion [145]. Different researchers also looked into ambient, natural and varying illumination [234], [235], [236]; However, once again, the setup was within a controlled setting where participants were at a distance of 0.6m from the camera. The participant diversity information was not revealed in the research paper. In addition to that, only the forehead is selected as ROI which would fail if the participant is not facing the camera. Only one study was found that was conducted in participants’ home environments, involving different camera-to-subject distances, and shared its open dataset. However, it reported a higher error rate, particularly among individuals with darker skin pigmentation (Fitzpatrick scale IV and above) and those in an active state [66].

B. Remote SpO2 Measurement

Only a few studies published presented methods to obtain SpO2. A new method to cancel aliased frequency components induced by fluorescent light flickering has previously been proposed based on autoregressive (AR) modelling and pole cancellation, which improved the effectiveness of the method under fluorescent illumination [237]. The research was conducted on patients undergoing kidney dialysis (in a resting state with minimal movement) [237]. However, movement and illumination change increased noise, in turn impacting system accuracy. The research also does not provide RMSE, r-correlation or σ for SpO2. While using a webcam, another research measured SpO2 and HR by using an algorithm for noise removal based on Dual-Tree Complex Wavelet Transform (DTCWT) to fix motion artefacts and artificial illumination [37].

Researchers also used R, G, and B channels to obtain SpO2 by assessing the pulse signal at two wavelengths of 660nm and 940nm [157]. This was obtained by comparing red and blue wavelength bands. However these researches [37], [157], [128], only used a 0.5m range, which is short for a realistic scenario [37], [157], [128].

The ARPOS research presented an innovative approach to measuring SpO2, by utilising extinction spectra of oxyhaemoglobin and deoxy-haemoglobin. They used a ratiometric measurement technique, dividing the recorded pixel intensity between two distinct spectral ranges, red (600–700 nm) and infrared (800–900 nm) to obtain this vital data. The error rate from this research was impressively found to be consistent over all participants for different skin pigmentation in resting and active states of ±2%. However, this needs to be clinically validated to stress test the system before this relying on this technique.

Another research used hand palms to measure SpO2 by applying spatial averaging, obtaining R, G, and B time series and applying Convolutional Neural Network (CNN) structure. However, using palms under a camera is not very practical as people in real-life would be required to keep their hands still under a camera. In addition to that, hands are also a less exposed part of the body compared to that of the face, which would make it difficult to deploy it in a real-life scenario [86], [238].

Different equipment has been used for the purpose of rPPG, which includes a CCD camera, smartphone, laptop’s webcam, drone with camera and Kinect as mentioned above. Each device has its own characteristics such as resolution, dimensions, processing power, data type collected and cost etc. Low and high-resolution digital cameras, and smartphone cameras capture RGB channels have been used in most studies to obtain vital values; whereas thermal camera allows for capturing thermal data of a participant. However, expensive equipment such as thermal [104], [69] or CCD equipment [71] (as defined in the previous section V-A) would not make it feasible to deploy on a limited budget. Smartphones need to be continuously held in hand, which can be tiring and impractical for a longer period of time or due to any incapacity.

VI. Limitations of Existing Rppg Systems

There are several limitations in the existing rPPG research systems that researchers can focus on improving which are listed below:

A. Equipment limitations

Thermal and CCD cameras used in these research studies can be very expensive [104], [69], [227], [228], [161], [239] to use especially for middle to lower income countries. Another problem with using a thermal camera is that it can generate temperature from the body and also the environment, which can impact the data acquired from a participant, such as thermal noise [104]. Furthermore, ambient sensing of oxygen saturation through multispectral imaging has been explored but such systems require specialist scientific equipment which can be expensive to measure SpO2 [240], [241], [242], [243], [244].

The equipment employed in these studies has primarily been limited to laboratory settings, utilising specialised hardware with ample RAM and GPU resources. However, this lab-based approach may not readily translate to real-world environments, potentially causing system failures. In clinical or home settings, individuals may be positioned at varying distances from the equipment, making it essential for the system to deliver accurate readings under diverse circumstances. This becomes especially critical in triage settings where versatile performance is essential.

Most of the research studies except the ARPOS research conducted among participants’ homes used specific hardware to acquire data where FPS from a camera was 15, 30 or 60 as shown in Table IV. This means the signal obtained from each channel (RGB) always had a constant FPS of 15, 30 or 60. The system (code) and equipment (computer and camera) were set up by the researcher in all studies and had enough processing power, RAM, graphics card and storage space to collect this data. However, people in a real-life scenario may not have access to expensive scientific equipment where the RAM, graphics card and GPU may impact the FPS and in turn, impact the number of samples in a signal obtained for each channel and if the system does not cater to this, it may impact error rate of the system. ARPOS research found that FPS impacts signal quality that is used to obtain vital data obtained data from realistic scenarios [66].

B. Controlled Environments

rPPG studies have predominantly been conducted within controlled environments where only one study evaluated their system in participants’ homes. While it is important to initially test and validate these systems in controlled environments, it is equally crucial to evaluate their performance in realworld scenarios, particularly within clinical settings. Clinical environments encompass a wide range of heart rate and respiratory conditions, involving diverse participants. This real-world validation against clinical standards is vital for ensuring the effectiveness and reliability of such systems. Researchers need to ensure they share evaluation measures when discussing the results of their research systems to ensure further progression in the rPPG field.

Currently, there is no rPPG system that is currently commercially available that can monitor these vital signs remotely unobtrusively and has been clinically validated. It is crucial for such systems to be validated in real-life environments including labs, homes and clinics to stress these systems. Furthermore, the effectiveness of these systems for monitoring health in clinical settings has yet to be fully validated. It is crucial that rPPG systems demonstrate the ability to promptly identify patients at the highest risk of health deterioration, highlighting the significant advantages of utilising such remote technology in healthcare.

C. ROI Selection and Occlusion

Moreover, relying solely on a single Region of Interest (ROI), like the forehead or cheeks, can prove ineffective in measuring vital signs. This limitation becomes evident when the chosen ROI is obscured from the camera’s view, for instance, when a person wears a headscarf or has long hair that conceals the forehead, or when the individual is positioned at an oblique angle to the camera. Therefore, it is crucial to subject a system to comprehensive testing across diverse settings to validate its performance and evaluate its error rate. Tracking people can also be occluded by objects, other people or both from the camera’s view to send data to the system which is common for front-viewing cameras utilised by the systems [245]. Due to occlusion, a user’s skeleton cannot be identified [245]; however, existing research has identified several methods to help solve this issue which includes API tracking of multiple people’s positions in an environment where occlusion occurs [246], [247], [245], location-aware wearable haptics [248], sound localisation [249], using API, a toolkit to track and multiple cameras within an environment to track multiple peoples and joints using depth data [245] and using shadow and skeletal fusion data to track people within an environment [250]. Occlusion needs to be further studied related to the rPPG systems.

D. Participant Diversity and Experiment Conditions

Most of the research mentioned in Table V and Table II have a limited number of participants with a narrow range of skin pigmentations. Specifically, when examining studies related to SpO2, only one of them featured a substantial participant group of 40 [66], while the remaining studies included fewer than 15 participants, as indicated in Table II. For HR measurement, a similar trend emerges with most studies having a limited number of participants. Only three studies stood out with a larger number of participants [145], [66], [72]. Among these, two studies involved 40 participants each [145], [66]. However, only one of these studies collected vital sign data within realistic home environments, encompassing both HR and SpO2 measurements [66]. The third study [72], which included a substantial number of participants of 117, only focused on measuring HR and was conducted in a controlled environment resembling a studio setup with limited movement and utilising expensive equipment [72]. Whereas the rest of the research studies in the Table V had a much lower number of participants with restricted experimented conditions. Furthermore, an important aspect is that, most of the studies neglected to measure oxygenation levels at a distance, as shown in Table IV.

All the research considered only one participant at a time except a few studies which included 2, 3 or even 6 participants measuring at a time [51], [66]. Measuring multiple participants simultaneously is still a feature required, especially when multiple people are within an environment. All the research studies mentioned above had participants closer to the camera (up to a distance of 0.5 from the camera) and only two studies considered measuring HR up to 4m [82], [66]. Most of the studies also do not disclose participant characteristics. A wider range of demographics such as various skin pigmentation and ethnicity need to be included in the research studies so it’s not only validated and made available for one specific group of people but can benefit everyone.

In a real-time situation, participants’ environments may vary such as triage environments in clinics, care homes or people’s homes and different factors such as face rotation, facial expression, varied distance from the camera, illumination, beards, skin pigmentation etc. can be present which can impact the system’s obtained data. Parameters related to makeup and beard were only limited to two studies, however with only a small number of participants [136], [66] and one study also does not analyse or segment results based on these parameters [136]. These experiments are important to be conducted within different age groups, skin pigmentation and different environmental parameters.

E. Dataset

Most of the studies did not share open data and code from their research. Only a limited number of public databases are available; however, they also are conducted within a lab environment with restricted parameters such as light, movement and not realistic distance from the camera. There is a lack of publicly available data sets collected from a reallife environment, so the system evaluation measures can be tested on those to evaluate their system [51], [66]. Only one dataset was found collected from the home environments of participants but this does not include participants with black skin pigmentation [201]. Studies conducted restrict participants from micro facial gestures or physical movement during vital data acquisition. In addition to that, if the error rate is not evaluated for realistic environments with natural movement or for example during undergoing treatment, it can result in failure to accurately measure vitals due to the presence of noise. Therefore, it is essential to measure vital sign data in real-life environments [51], [66]. Face tracking will also impact ROI selection when movement takes place, depending upon the lighting, and exposure length of the camera, it could potentially create blurry frames, and it is important to analyse data with noise to ensure the system will be able to perform as required in a real-life environment [51]. It would be more beneficial if video streams of Colour, IR and Depth Videos would be made available open source so researchers can analyse the realistic data including various movements, illumination, environments, skin pigmentation and other factors with their respective timestamps to analyse their algorithms.

VII. Bias in Rppg Systems

Discussing rPPG systems presents several challenges, particularly because they rely on camera-based technology focused on capturing facial data. These challenges become especially noticeable due to the diversity of participants’ skin pigmentation. rPPG systems must be accurate, as they fall under medical technology upon which people may depend once they become commercially available. Therefore, rigorous validation and extensive testing across a diverse range of participants are imperative. Many publicly available databases primarily emphasise individuals with lighter skin pigmentation, typically categorised under the Fitzpatrick scale I-III. Neglecting diversity can pose a significant disadvantage when deploying rPPG systems; making it beneficial only for a specific group of people. One study [137], as shown in Table III, demonstrated a higher error rate in such cases. In contrast, a recent study, ARPOS [66], showcased a lower error rate and introduced effective techniques across white and darker skin pigmentations. However, the study’s dataset did not include any black participants.

Another critical observation from this review is the limited consideration given to participants who wear makeup, such as lipsticks or concealers, or those with facial hair, ranging from light to heavy beards. These factors can significantly influence the accuracy of data readings in rPPG systems. Only two studies were found which stated participants wearing makeup[66], [251]. Both research studies found makeup in Colour and IR impacted the accuracy of the rPPG systems and suggested further rigours testing. However, the ARPOS study only had one participant wearing lipstick [66] whereas the other research study only had participants wearing foundation [251]. Consequently, it becomes important to rigorously validate these systems, accounting for these various biases, and further evaluating their performance across different genders and age groups.

VIII. Summary

This paper presents an in-depth literature review on rPPGrelated research that can measure HR and/or SpO2 of people at a distance. Various processes, techniques, participant characteristics, equipment and experiment conditions used in the research studies were discussed. The performance and evaluation measures of previous research and its limitations were presented in this paper as well. Various equipment that was utilised included a thermal, CCD, web camera, Kinect V2, and camera installed in robots, drones and smartphone cameras have been detailed in this paper. The common process for extracting these vital signs usually involves ROI selection, extracting raw signals, preprocessing data, applying algorithms, FFT, filtering and identifying vitals. The review found that most research studies focused on measuring HR, a few on SpO2 and only a very limited number of research focused on measuring both vital sign data in a remote manner.

Most of the research studies for measuring HR had low participants and only a couple had a higher number of participants (as shown in Table V and discussed in section VI-D); but were conducted in a very controlled setup or lab environments which cannot be replicated in real-life scenarios. In addition to research data being gathered mostly in a controlled environment or a lab setting; it also restricted participants from micro facial gestures or physical movement of their arms or face rotations as in a real-life situation people cannot be expected to be very still for multiple minutes when their vitals are being measured as natural movement is normal (such as talking, rotation and movement due to facial expressions) which can increase noise in the data.

There is only one real-life scenario database publicly available collected to validate rPPG systems [66]. Previous research needs to consider various factors when designing an rPPG system including varying camera-to-subject distances, fluctuating frames per second (FPS), computational resource requirements, diverse environments, illumination conditions and participant diversity. Furthermore, it is crucial to validate and test for performance under conditions involving natural movements during vital sign measurements that may lead to inaccurate results. This issue becomes particularly critical when considering real-world deployments such as clinics and hospitals.

In conclusion, the field of rPPG has made significant progress in recent years. This review has highlighted various aspects of these methods for measuring HR and SpO2. Several important findings and research challenges were been identified. rPPG has enormous potential for various applications in both clinical and non-clinical settings. However, addressing the identified research challenges is crucial for advancing this field and ensuring the accuracy and reliability of these systems in real-world scenarios. This review serves as a valuable resource for researchers seeking to explore and contribute to this evolving field.

Data Availability

All data produced in the present work are contained in the manuscript.

IX. Acknowledgments

In order to meet institutional and research funder open access requirements, any accepted manuscript arising shall be open access under a Creative Commons Attribution (CC BY) reuse licence with zero embargo.

Footnotes

REFERENCES

  1. [1].↵
    C. Brüser, C. H. Antink, T. Wartzek, M. Walter, and S. Leonhardt, “Ambient and unobtrusive cardiorespiratory monitoring techniques,” IEEE Reviews in Biomedical Engineering, vol. 8, pp. 30–43, 2015.
    OpenUrl
  2. [2].↵
    T. Chakrabarti, S. Saha, S. Roy, and I. Chel, “Phonocardiogram signal analysis - practices, trends and challenges: A critical review,” in 2015 International Conference and Workshop on Computing and Communication (IEMCON), 2015, pp. 1–4.
  3. [3].↵
    A. A. Venessa L. Pinto, Prasanna Tadi, “Increased intracranial pressure,” NCBI, 2021.
  4. [4].↵
    R. A. Sharven Taghavi, “Hypovolemic shock,” NCBI, 2022.
  5. [5].↵
    D. H. Spodick, P. Raju, R. L. Bishop, and R. D. Rifkin, “Operational definition of normal sinus heart rate,” The American Journal of Cardiology, vol. 69, no. 14, pp. 1245–1246, 1992. [Online]. Available: https://www.sciencedirect.com/science/article/pii/000291499290947W
    OpenUrlCrossRefPubMedWeb of Science
  6. [6].↵
    A. S. Popel, “Theory of oxygen transport to tissue,” Critical reviews in biomedical engineering, vol. 17, no. 3, p. 257, 1989.
    OpenUrlPubMedWeb of Science
  7. [7].↵
    B. B. Hafen and S. Sharma, Oxygen Saturation. StatPearls Publishing, Treasure Island (FL), 2022. [Online]. Available: http://europepmc.org/books/NBK525974
  8. [8].↵
    S. R. Majumdar, D. T. Eurich, J.-M. Gamble, A. Senthilselvan, and T. J. Marrie, “Oxygen Saturations Less than 92% Are Associated with Major Adverse Events in Outpatients with Pneumonia: A Population-Based Cohort Study,” Clinical Infectious Diseases, vol. 52, no. 3, pp. 325–331, 02 2011. [Online]. Available: doi:10.1093/cid/ciq076
    OpenUrlCrossRefPubMedWeb of Science
  9. [9].↵
    P. Modi, R. B. Mark Munyaneza, E. Goldberg, G. Choy, R. Shailam, P. Sagar, S. J. Westra, S. Nyakubyara, M. Gakwerere, V. Wolfman, A. Vinograd, M. Moore, and A. C. Levine, “Oxygen saturation can predict pediatric pneumonia in a resource-limited setting,” The Journal of Emergency Medicine, vol. 45, no. 5, pp. 752–760, 2013. [Online]. Available: https://www.sciencedirect.com/science/article/pii/ S0736467913005969
    OpenUrlCrossRefPubMed
  10. [10].↵
    A. Plüddemann, M. Thompson, C. Heneghan, and C. Price, “Pulse oximetry in primary care: primary care diagnostic technology update,” British Journal of General Practice, vol. 61, no. 586, pp. 358–359, 2011.
    OpenUrlFREE Full Text
  11. [11].↵
    V. Kim, J. O. Benditt, R. A. Wise, and A. Sharafkhaneh, “Oxygen therapy in chronic obstructive pulmonary disease,” Proceedings of the American Thoracic Society, vol. 5, no. 4, pp. 513–518, 2008.
    OpenUrlCrossRefPubMed
  12. [12].↵
    J. Buekers, J. Theunis, P. De Boever, A. W. Vaes, M. Koopman, E. V. Janssen, E. F. Wouters, M. A. Spruit, J.-M. Aerts et al., “Wearable finger pulse oximetry for continuous oxygen saturation measurements during daily home routines of patients with chronic obstructive pulmonary disease (copd) over one week: observational study,” JMIR mHealth and uHealth, vol. 7, no. 6, p. e12866, 2019.
    OpenUrl
  13. [13].↵
    E. A. Kiyatkin, “Respiratory depression and brain hypoxia induced by opioid drugs: Morphine, oxycodone, heroin, and fentanyl,” Neuropharmacology, vol. 151, pp. 219–226, 2019. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0028390818306245
    OpenUrl
  14. [14].↵
    B. B. Eman Shebl, “Respiratory failure,” NCBI, 2021.
  15. [15].↵
    B. Govindaswami, P. Jegatheesan, and D. Song, “Oxygen Saturation Screening for Critical Congenital Heart Disease,” NeoReviews, vol. 13, no. 12, pp. e724–e731, 12 2012. [Online]. Available: doi:10.1542/neo.13-12-e724
    OpenUrlAbstract/FREE Full Text
  16. [16].↵
    E. Suprayitno, M. Marlianto, and M. Mauliana, “Measurement device for detecting oxygen saturation in blood, heart rate, and temperature of human body,” in Journal of Physics: Conference Series, vol. 1402, no. 3. IOP Publishing, 2019, p. 033110.
    OpenUrl
  17. [17].↵
    F. Chiappini and L. Fuso, “Usefulness of pulse oximetry in respiratory diseases,” Recenti Progressi in Medicina, vol. 88, no. 12, pp. 574–578, 1997.
    OpenUrlPubMed
  18. [18].↵
    A. Kamal, J. Harness, G. Irving, and A. Mearns, “Skin photo-plethysmography—a review,” Computer methods and programs in biomedicine, vol. 28, no. 4, pp. 257–269, 1989.
    OpenUrlCrossRefPubMedWeb of Science
  19. [19].↵
    J. Allen, “Photoplethysmography and its application in clinical physiological measurement,” Physiological measurement, vol. 28, no. 3, p. R1, 2007.
    OpenUrlCrossRefPubMedWeb of Science
  20. [20].↵
    A. B. Hertzman, “Observations on the finger volume pulse recorded photoelectrically,” Am. J. Physiol., vol. 119, pp. 334–335, 1937.
    OpenUrl
  21. [21].↵
    J. Allen, “Photoplethysmography and its application in clinical physiological measurement,” Physiological measurement, vol. 28, no. 3, p. R1, 2007.
    OpenUrlCrossRefPubMedWeb of Science
  22. [22].↵
    H. Lee, H. Ko, and J. Lee, “Reflectance pulse oximetry: Practical issues and limitations,” Ict Express, vol. 2, no. 4, pp. 195–198, 2016.
    OpenUrl
  23. [23].↵
    J. G. Webster, Design of pulse oximeters. CRC Press, 1997.
  24. [24].↵
    A. Jubran, “Pulse oximetry,” Critical care, vol. 3, pp. 1–7, 1999.
    OpenUrlCrossRefPubMedWeb of Science
  25. [25].
    S. Prahl, “Tabulated molar extinction coefficient for hemoglobin.” [Online]. Available: https://omlc.org/spectra/hemoglobin/summary.html
  26. [26].↵
    P. Ross, F. Hammes, M. Dignum, A. Magic-Knezev, B. Hambsch, and L. Rietveld, “A comparative study of three different assimilable organic carbon (aoc) methods: results of a round-robin test,” Water Science and Technology: Water Supply, vol. 13, no. 4, pp. 1024–1033, 2013.
    OpenUrlCrossRef
  27. [27].↵
    D. Castaneda, A. Esparza, M. Ghamari, C. Soltanpur, and H. Nazeran, “A review on wearable photoplethysmography sensors and their potential future applications in health care,” International journal of biosensors & bioelectronics, vol. 4, no. 4, p. 195, 2018.
    OpenUrl
  28. [28].↵
    P. V. Rouast, M. T. Adam, R. Chiong, D. Cornforth, and E. Lux, “Remote heart rate measurement using low-cost rgb face video: a technical literature review,” Frontiers of Computer Science, vol. 12, no. 5, pp. 858–872, 2018.
    OpenUrl
  29. [29].↵
    E. Nemati, M. J. Deen, and T. Mondal, “A wireless wearable ecg sensor for long-term applications,” IEEE Communications Magazine, vol. 50, no. 1, pp. 36–43, 2012.
    OpenUrlCrossRef
  30. [30].↵
    N. K. Rauniyar, S. Pujari, and P. Shrestha, “Study of oxygen saturation by pulse oximetry and arterial blood gas in icu patients: A descriptive cross-sectional study,” JNMA: Journal of the Nepal Medical Association, vol. 58, no. 230, p. 789, 2020.
    OpenUrl
  31. [31].↵
    Y. Mendelson, “Pulse oximetry: theory and applications for noninvasive monitoring,” Clinical chemistry, vol. 38, no. 9, pp. 1601–1607, 1992.
    OpenUrlAbstract/FREE Full Text
  32. [32].↵
    N. Joglekar, J. Chandran, R. Veljanovski, A. Stojcevski, A. Zayegh, and M. Dorairaj, “A review of blood gas sensors and a proposed portable solution,” in 2007 International Symposium on Integrated Circuits, 2007, pp. 244–247.
  33. [33].↵
    G. Pang and C. Ma, “A neo-reflective wrist pulse oximeter,” IEEE access, vol. 2, pp. 1562–1567, 2014.
    OpenUrl
  34. [34].↵
    Q. Cai, J. Sun, L. Xia, and X. Zhao, “Implementation of a wireless pulse oximeter based on wrist band sensor,” in 2010 3rd International Conference on Biomedical Engineering and Informatics, vol. 5. IEEE, 2010, pp. 1897–1900.
    OpenUrl
  35. [35].↵
    N. Mackenzie, “Comparison of a pulse oximeter with an ear oximeter and an in-vitro oximeter,” Journal of Clinical Monitoring, vol. 1, no. 3, pp. 156–160, 1985.
    OpenUrlPubMed
  36. [36].↵
    A. Azhari, S. Yoshimoto, T. Nezu, H. Iida, H. Ota, Y. Noda, T. Araki, T. Uemura, T. Sekitani, and K. Morii, “A patch-type wireless fore-head pulse oximeter for spo¡inf¿2¡/inf¿ measurement,” in 2017 IEEE Biomedical Circuits and Systems Conference (BioCAS), 2017, pp. 1–4.
  37. [37].↵
    U. Bal, “Non-contact estimation of heart rate and oxygen saturation using ambient light,” Biomedical optics express, vol. 6, no. 1, pp. 86–97, 2015.
    OpenUrl
  38. [38].↵
    K. D. Torp, P. Modi, and L. V. Simon, “Pulse oximetry,” in StatPearls [Internet]. StatPearls Publishing, 2021.
  39. [39].↵
    K. C. Badgujar, A. B. Badgujar, D. V. Dhangar, and V. C. Badgujar, “Importance and use of pulse oximeter in covid-19 pandemic: general factors affecting the sensitivity of pulse oximeter,” Indian Chemical Engineer, vol. 62, no. 4, pp. 374–384, 2020.
    OpenUrl
  40. [40].↵
    A. Jubran, “Pulse oximetry,” pp. 29–32, 2006. [Online]. Available: doi:10.1007/3-540-37363-28
    OpenUrlCrossRef
  41. [41].↵
    L. Martins, O. Gaidos, and I. dos Santos, “Pulse oximeter for cyclists in smartphone,” in 10th International Symposium on Medical Information Processing and Analysis,vol. 9287. International Society for Optics and Photonics, 2015, p. 92870X.
    OpenUrl
  42. [42].↵
    M. W. Wukitsch, M. T. Petterson, D. R. Tobler, and J. A. Pologe, “Pulse oximetry: analysis of theory, technology, and practice,” Journal of clinical monitoring, vol. 4, no. 4, pp. 290–301, 1988.
    OpenUrlCrossRefPubMedWeb of Science
  43. [43].↵
    J. Lopera, J. Diaz, M. Prieto, and F. Nuno, “Pulse oximeter for homecare,” in Proceedings of the Second Joint 24th Annual Conference and the Annual Fall Meeting of the Biomedical Engineering Society] [Engineering in Medicine and Biology, vol. 2, 2002, pp. 1738–1739 vol.2.
    OpenUrl
  44. [44].↵
    A. M. Luks and E. R. Swenson, “Pulse oximetry for monitoring patients with covid-19 at home. potential pitfalls and practical guidance,” Annals of the American Thoracic Society, vol. 17, no. 9, pp. 1040–1046, 2020.
    OpenUrl
  45. [45].↵
    Z. Yuan, Z. Fang, X. Chen, Z. Xu, D. Chen, Z. Zhao, and S. Xia, “A novel reflection pulse oximeter for home health care applications,” in 2014 Fifth International Conference on Intelligent Systems Design and Engineering Applications, 2014, pp. 64–68.
  46. [46].↵
    A. H. Işik and I. Güler, “Pulse oximeter based mobile biotelemetry application,” Stud Health Technol Inform, vol. 181, pp. 197–201, 2012.
    OpenUrlPubMed
  47. [47].↵
    M. Pflugradt, I. Fritzsch, S. Mann, T. Tigges, and R. Orglmeister, “A novel pulseoximeter for bluetooth synchronized measurements in a body sensor network,” in 2014 6th European Embedded Design in Education and Research Conference (EDERC). IEEE, 2014, pp. 21–25.
  48. [48].↵
    M. J. Moron, E. Casilari, R. Luque, and J. A. Gázquez, “A wireless monitoring system for pulse-oximetry sensors,” in 2005 Systems Communications (ICW’05, ICHSN’05, ICMCS’05, SENET’05). IEEE, 2005, pp. 79–84.
  49. [49].↵
    Y. Dai and J. Luo, “Design of noninvasive pulse oximeter based on bluetooth 4.0 ble,” in 2014 Seventh International Symposium on Computational Intelligence and Design, vol. 1. IEEE, 2014, pp. 100–103.
    OpenUrl
  50. [50].↵
    G. Mardirossian and R. E. Schneider, “Limitations of pulse oximetry.” Anesthesia progress, vol. 39, no. 6, p. 194, 1992.
    OpenUrlPubMed
  51. [51].↵
    F.-T.-Z. Khanam, A. Al-Naji, J. Chahl et al., “Remote monitoring of vital signs in diverse non-clinical and clinical scenarios using computer vision systems: A review,” Applied Sciences, vol. 9, no. 20, p. 4474, 2019.
    OpenUrl
  52. [52].↵
    K. M. van der Kooij and M. Naber, “An open-source remote heart rate imaging method with practical apparatus and algorithms,” Behavior research methods, vol. 51, no. 5, pp. 2106–2119, 2019.
    OpenUrl
  53. [53].↵
    L. A. Aarts, V. Jeanne, J. P. Cleary, C. Lieber, J. S. Nelson, S. Bambang Oetomo, and W. Verkruysse, “Non-contact heart rate monitoring utilizing camera photoplethysmography in the neonatal intensive care unit — a pilot study,” Early Human Development, vol. 89, no. 12, pp. 943–948, 2013. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0378378213002375
    OpenUrl
  54. [54].↵
    S. Phansalkar, J. Edworthy, E. Hellier, D. L. Seger, A. Schedlbauer, A. J. Avery, and D. W. Bates, “A review of human factors principles for the design and implementation of medication safety alerts in clinical information systems,” Journal of the American Medical Informatics Association, vol. 17, no. 5, pp. 493–501, 2010.
    OpenUrlCrossRefPubMed
  55. [55].↵
    J. D’Orazio, S. Jarrett, A. Amaro-Ortiz, and T. Scott, “Uv radiation and the skin,” International journal of molecular sciences, vol. 14, no. 6, pp. 12 222–12 248, 2013.
    OpenUrl
  56. [56].↵
    D.-Y. Kim, K. Lee, and C.-B. Sohn, “Assessment of ROI selection for facial video-based rPPG,” Sensors (Basel), vol. 21, no. 23, p. 7923, Nov. 2021.
    OpenUrl
  57. [57].↵
    K. E. Philip, R. Tidswell, and C. McFadyen, “Racial bias in pulse oximetry: more statistical detail may help tackle the problem,” bmj, vol. 372, 2021.
  58. [58].↵
    M. S. Arefin, A. P. Dumont, and C. A. Patil, “Monte carlo based simulations of racial bias in pulse oximetry,” in Design and Quality for Biomedical Technologies XV, vol. 11951. SPIE, 2022, pp. 8–12.
    OpenUrl
  59. [59].↵
    L. J. Mengelkoch, D. Martin, and J. Lawler, “A review of the principles of pulse oximetry and accuracy of pulse oximeter estimates during exercise,” Physical therapy, vol. 74, no. 1, pp. 40–49, 1994.
    OpenUrlAbstract/FREE Full Text
  60. [60].↵
    C. Cahan, M. J. Decker, P. L. Hoekje, and K. P. Strohl, “Agreement between noninvasive oximetric values for oxygen saturation,” Chest, vol. 97, no. 4, pp. 814–819, 1990.
    OpenUrlCrossRefPubMed
  61. [61].↵
    P. E. Bickler, J. R. Feiner, and J. W. Severinghaus, “Effects of skin pigmentation on pulse oximeter accuracy at low saturation,” The Journal of the American Society of Anesthesiologists, vol. 102, no. 4, pp. 715–719, 2005.
    OpenUrl
  62. [62].↵
    M. M. Chan, M. M. Chan, and E. D. Chan, “What is the effect of fingernail polish on pulse oximetry?” Chest, vol. 123, no. 6, pp. 2163–2164, 2003.
    OpenUrlCrossRefPubMedWeb of Science
  63. [63].↵
    A. Raipure, V. Ankalwar, N. G. Tirpude, and K. Gaikawad, “Effect of nail paint on pulse oximetry values & clinical importance,” 2019.
  64. [64].↵
    T. M. Brand, M. E. Brand, and G. D. Jay, “Enamel nail polish does not interfere with pulse oximetry among normoxic volunteers,” Journal of clinical monitoring and computing, vol. 17, no. 2, pp. 93–96, 2002.
    OpenUrlCrossRefPubMed
  65. [65].↵
    O. Jakpor, “Do artificial nails and nail polish interfere with the accurate measurement of oxygen saturation by pulse oximetry?” Young Scientists Journal, vol. 4, no. 9, p. 33, 2011.
    OpenUrl
  66. [66].↵
    P. Pirzada, D. Morrison, G. Doherty, D. Dhasmana, and D. Harris-Birtill, “Automated remote pulse oximetry system (arpos),” Sensors, vol. 22, no. 13, 2022. [Online]. Available: https://www.mdpi.com/1424-8220/22/13/4974
  67. [67].↵
    I. Pavlidis, “Continuous physiological monitoring,” in Proceedings of the 25th annual international conference of the IEEE engineering in medicine and biology society (IEEE Cat. No. 03CH37439), vol. 2. IEEE, 2003, pp. 1084–1087.
    OpenUrl
  68. [68].↵
    M. Garbey, N. Sun, A. Merla, and I. Pavlidis, “Contact-free measurement of cardiac pulse based on the analysis of thermal imagery,” IEEE transactions on Biomedical Engineering, vol. 54, no. 8, pp. 1418–1426, 2007.
    OpenUrlPubMed
  69. [69].↵
    R. G. Gatto, Estimation of instantaneous heart rate using video infrared thermography and arma models. University of Illinois at Chicago, 2009.
  70. [70].↵
    D.-Y. Chen, H.-S. Zou, and A.-T. Hsieh, “Thermal image based remote heart rate measurement on dynamic subjects using deep learning,” in 2020 IEEE International Conference on Consumer Electronics - Taiwan (ICCE-Taiwan). IEEE, Sep. 2020.
  71. [71].↵
    C. Takano and Y. Ohta, “Heart rate measurement based on a time-lapse image,” Medical engineering & physics, vol. 29, no. 8, pp. 853–857, 2007.
    OpenUrl
  72. [72].↵
    G. De Haan and V. Jeanne, “Robust pulse rate from chrominancebased rppg,” IEEE Transactions on Biomedical Engineering, vol. 60, no. 10, pp. 2878–2886, 2013.
    OpenUrlCrossRef
  73. [73].↵
    M.-Z. Poh, D. J. McDuff, and R. W. Picard, “Non-contact, automated cardiac pulse measurements using video imaging and blind source separation.” Optics express, vol. 18, no. 10, pp. 10 762–10 774, 2010.
    OpenUrlCrossRefPubMed
  74. [74].↵
    M.-Z. Poh, D. J. McDuff, and R. W. Picard, “Advancements in noncontact, multiparameter physiological measurements using a webcam,” IEEE transactions on biomedical engineering, vol. 58, no. 1, pp. 7–11, 2010.
    OpenUrl
  75. [75].↵
    M. Oviyaa, P. Renvitha, and R. Swathika, “Real time tracking of heart rate from facial video using webcam,” in 2020 Second International Conference on Inventive Research in Computing Applications (ICIRCA). IEEE, Jul. 2020.
  76. [76].↵
    M. Kopeliovich, M. Petrushan, and D. Shaposhnikov, “Approximation-based transformation of color signal for heart rate estimation with a webcam,” Pattern Recognition and Image Analysis, vol. 28, pp. 646–651, 2018.
    OpenUrl
  77. [77].↵
    Y. Muratov, M. Nikiforov, O. Melnik, and A. Loskutov, “Heart rate measurements with a web camera based on the facial image moving in the frame,” in 2021 10th Mediterranean Conference on Embedded Computing (MECO). IEEE, Jun. 2021.
  78. [78].↵
    H. Lee, H. Ko, H. Chung, Y. Nam, S. Hong, and J. Lee, “Real-time realizable mobile imaging photoplethysmography,” Sci. Rep., vol. 12, no. 1, p. 7141, May 2022.
    OpenUrl
  79. [79].↵
    D. Wang, X. Yang, X. Liu, J. Jing, and S. Fang, “Detail-preserving pulse wave extraction from facial videos using consumer-level camera,” Biomed. Opt. Express, vol. 11, no. 4, pp. 1876–1891, Apr. 2020.
    OpenUrl
  80. [80].↵
    M. Das, T. Choudhary, M. K. Bhuyan, and L. N. Sharma, “Non-contact heart rate measurement from facial video data using a 2D-VMD scheme,” IEEE Sens. J., vol. 22, no. 11, pp. 11 153–11 161, Jun. 2022.
    OpenUrl
  81. [81].↵
    R. Qian, T. Jin, H. Li, and Y. Dai, “WT-based data-length-variation technique for fast heart rate detection,” in 2018 Progress in Electro-magnetics Research Symposium (PIERS-Toyama). IEEE, Aug. 2018.
  82. [82].↵
    A. Al-Naji and J. Chahl, “Detection of cardiopulmonary activity and related abnormal events using microsoft kinect sensor,” Sensors, vol. 18, no. 3, p. 920, 2018.
    OpenUrl
  83. [83].↵
    L. Zhou, M. Yin, X. Xu, X. Yuan, and X. Liu, “Non-contact detection of human heart rate with kinect,” Cluster Computing, vol. 22, pp. 8199–8206, 2019.
    OpenUrl
  84. [84].↵
    A. Shokouhmand, S. Eckstrom, B. Gholami, and N. Tavassolian, “Camera-augmented non-contact vital sign monitoring in real time,” IEEE Sens. J., vol. 22, no. 12, pp. 11 965–11 978, Jun. 2022.
    OpenUrl
  85. [85].↵
    A. Al-Naji, A. G. Perera, and J. Chahl, “Remote monitoring of cardiorespiratory signals from a hovering unmanned aerial vehicle,” Biomedical engineering online, vol. 16, no. 1, pp. 1–20, 2017.
    OpenUrl
  86. [86].↵
    J. Mathew, X. Tian, M. Wu, and C.-W. Wong, “Remote blood oxygen estimation from videos using neural networks,” arXiv preprint arXiv:2107.05087, 2021.
  87. [87].↵
    N. Molinaro, E. Schena, S. Silvestri, and C. Massaroni, “Multi-ROI spectral approach for the continuous remote cardio-respiratory monitoring from mobile device built-in cameras,” Sensors (Basel), vol. 22, no. 7, p. 2539, Mar. 2022.
    OpenUrl
  88. [88].↵
    N. Q. Al-Naggar, H. M. Al-Hammadi, A. M. Al-Fusail, and Z. A. Al-Shaebi, “Design of a remote real-time monitoring system for multiple physiological parameters based on smartphone,” J. Healthc. Eng., vol. 2019, p. 5674673, Nov. 2019.
    OpenUrl
  89. [89].↵
    T. Burton, G. Saiko, M. Cao, and A. Douplik, “Remote photoplethys-mography with consumer smartphone reveals temporal differences between glabrous and nonglabrous skin: Pilot in vivo study,” J. Biophotonics, vol. 16, no. 1, p. e202200187, Jan. 2023.
    OpenUrl
  90. [90].↵
    S. Sanyal and K. K. Nundy, “Algorithms for monitoring heart rate and respiratory rate from the video of a user’s face,” IEEE J. Transl. Eng. Health Med., vol. 6, p. 2700111, Apr. 2018.
    OpenUrl
  91. [91].↵
    T. Coppetti, A. Brauchlin, S. Müggler, A. Attinger-Toller, C. Templin, F. Schönrath, J. Hellermann, T. F. Lüscher, P. Biaggi, and C. A. Wyss, “Accuracy of smartphone apps for heart rate measurement,” Eur. J. Prev. Cardiol., vol. 24, no. 12, pp. 1287–1293, Aug. 2017.
    OpenUrlCrossRef
  92. [92].↵
    Y. Nam, Y. Kong, B. Reyes, N. Reljin, and K. H. Chon, “Monitoring of heart and breathing rates using dual cameras on a smartphone,” PLoS One, vol. 11, no. 3, p. e0151013, Mar. 2016.
    OpenUrlCrossRef
  93. [93].↵
    S.-Y. Lin and M.-R. Tsai, “Non-contact heart rate monitoring with mobile phone camera,” in 2019 IEEE International Conference on Consumer Electronics - Taiwan (ICCE-TW). IEEE, May 2019.
  94. [94].↵
    M. Ben Ayed, S. A. Alshaya, and A. Alshammari, “Enhanced heart rate estimation based on face features,” in 2021 18th International Multi-Conference on Systems, Signals & Devices (SSD). IEEE, Mar. 2021.
  95. [95].↵
    W. Verkruysse, L. O. Svaasand, and J. S. Nelson, “Remote plethysmo-graphic imaging using ambient light.” Optics express, vol. 16, no. 26, pp. 21 434–21 445, 2008.
    OpenUrlCrossRefPubMed
  96. [96].↵
    G. Balakrishnan, F. Durand, and J. Guttag, “Detecting pulse from head motions in video,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2013, pp. 3430–3437.
  97. [97].↵
    M. Lewandowska, J. Ruminski, T. Kocejko, and J. Nowak, “Measuring pulse rate with a webcam — a non-contact method for evaluating cardiac activity,” in 2011 Federated Conference on Computer Science and Information Systems (FedCSIS), 2011, pp. 405–410.
  98. [98].↵
    M. Jones, “Rapid object detection using a boosted cascade of simple,” in Proceedings IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Kauai, USA: IEEE, 2001.
  99. [99].↵
    N. A. Akbar, A. Muneer, S. M. Taib, and S. Mohamed Fati, “Measuring accuracy towards facial video heart-rate estimation using haar-cascade and CNN method,” in 2022 International Conference on Decision Aid Sciences and Applications (DASA). IEEE, Mar. 2022.
  100. [100].↵
    K.-Z. Lee, P.-C. Hung, and L.-W. Tsai, “Contact-free heart rate measurement using a camera,” in 2012 Ninth Conference on Computer and Robot Vision. IEEE, 2012, pp. 147–152.
  101. [101].↵
    Y. Fujita, M. Hiromoto, and T. Sato, “Fast and robust heart rate estimation from videos through dynamic region selection,” in 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, Jul. 2018.
  102. [102].↵
    K. Zheng, K. Ci, H. Li, L. Shao, G. Sun, J. Liu, and J. Cui, “Heart rate prediction from facial video with masks using eye location and corrected by convolutional neural networks,” Biomed. Signal Process. Control, vol. 75, no. 103609, p. 103609, May 2022.
    OpenUrl
  103. [103].↵
    T. F. Cootes, G. J. Edwards, and C. J. Taylor, “Active appearance models,” in European conference on computer vision. Springer, 1998, pp. 484–498.
  104. [104].↵
    M. Waqar, “Contactfree heart rate measurement from human face videos and its biometric recognition application,” Ph.D. dissertation, Aberystwyth University, 2019.
  105. [105].↵
    L. Feng, L.-M. Po, X. Xu, Y. Li, and R. Ma, “Motion-resistant remote imaging photoplethysmography based on the optical properties of skin,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 25, no. 5, pp. 879–891, 2014.
    OpenUrl
  106. [106].↵
    L. Feng, L.-M. Po, X. Xu, and Y. Li, “Motion artifacts suppression for remote imaging photoplethysmography,” in 2014 19th International Conference on Digital Signal Processing. IEEE, 2014, pp. 18–23.
  107. [107].↵
    B. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision (ijcai),” vol. 81, 04 1981.
  108. [108].↵
    D. Comaniciu, V. Ramesh, and P. Meer, “Kernel-based object tracking,” IEEE Transactions on pattern analysis and machine intelligence, vol. 25, no. 5, pp. 564–577, 2003.
    OpenUrlCrossRefWeb of Science
  109. [109].↵
    S. Kwon, J. Kim, D. Lee, and K. Park, “ROI analysis for remote photoplethysmography on facial video,” Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., vol. 2015, pp. 4938–4941, Aug. 2015.
    OpenUrl
  110. [110].↵
    A. R. Arppana, N. K. Reshmma, G. Raghu, N. Mathew, H. R. Nair, and R. P. Aneesh, “Real time heart beat monitoring using computer vision,” in 2021 Seventh International conference on Bio Signals, Images, and Instrumentation (ICBSII). IEEE, Mar. 2021.
  111. [111].↵
    C. Wang, Y. Jiang, Z. Cai, and L. Lin, “Non-contact measurement of heart rate based on facial video,” in 2019 Photonics & Electromagnetics Research Symposium - Fall (PIERS - Fall). IEEE, Dec. 2019.
  112. [112].↵
    A. Ni, A. Azarang, and N. Kehtarnavaz, “A review of deep learning-based contactless heart rate measurement methods,” Sensors, vol. 21, p. 3719, 05 2021.
    OpenUrl
  113. [113].↵
    M. A. Hassan, A. S. Malik, D. Fofi, N. Saad, B. Karasfi, Y. S. Ali, and F. Meriaudeau, “Heart rate estimation using facial video: A review,” Biomedical Signal Processing and Control, vol. 38, pp. 346–360, 2017.
    OpenUrl
  114. [114].↵
    W. Wang, A. C. den Brinker, and G. de Haan, “Full video pulse extraction,” Biomed. Opt. Express, vol. 9, no. 8, pp. 3898–3914, Aug. 2018.
    OpenUrl
  115. [115].↵
    J. Cheng, P. Wang, R. Song, Y. Liu, C. Li, Y. Liu, and X. Chen, “Remote heart rate measurement from near-infrared videos based on joint blind source separation with delay-coordinate transformation,” IEEE Trans. Instrum. Meas., vol. 70, pp. 1–13, 2021.
    OpenUrl
  116. [116].↵
    H. Ernst, H. Malberg, and M. Schmidt, “More reliable remote heart rate measurement by signal quality indexes,” in 2020 Computing in Cardiology Conference (CinC). Computing in Cardiology, Dec. 2020.
  117. [117].↵
    H. Gao, X. Wu, J. Geng, and Y. Lv, “Remote heart rate estimation by signal quality attention network,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE, Jun. 2022.
  118. [118].↵
    H. Liu, Y. Wang, and L. Wang, “A review of non-contact, low-cost physiological information measurement based on photoplethys-mographic imaging,” in 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE, Aug. 2012.
  119. [119].↵
    G. Cennini, J. Arguel, K. Akşit, and A. van Leest, “Heart rate monitoring via remote photoplethysmography with motion artifacts reduction,” Opt. Express, vol. 18, no. 5, pp. 4867–4875, Mar. 2010.
    OpenUrlPubMed
  120. [120].↵
    W. Wang, A. C. den Brinker, S. Stuijk, and G. de Haan, “Amplitude-selective filtering for remote-PPG,” Biomed. Opt. Express, vol. 8, no. 3, pp. 1965–1980, Mar. 2017.
    OpenUrl
  121. [121].↵
    C. Pham, K. Poorzargar, M. Nagappa, A. Saripella, M. Parotto, M. Englesakis, K. Lee, and F. Chung, “Effectiveness of consumer-grade contactless vital signs monitors: a systematic review and meta-analysis,” J. Clin. Monit. Comput., vol. 36, no. 1, pp. 41–54, Feb. 2022.
    OpenUrlCrossRef
  122. [122].↵
    O. Havakuk, B. Sadeh, I. Merdler, Z. Zalevsky, J. Garcia-Monreal, S. Polani, and Y. Arbel, “Validation of a novel contact-free heart and respiratory rate monitor,” vol. 45, no. 5, pp. 344–350, Jul. 2021.
    OpenUrl
  123. [123].↵
    V. Saran, R. Kumar, G. Kumar, K. Chokalingam, M. Rawooth, and G. Parchani, “Validation of dozee, a ballistocardiography-based device, for contactless and continuous heart rate and respiratory rate measurement,” in 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE, Jul. 2022.
  124. [124].↵
    H. Lee, A. Cho, and M. Whang, “Fusion method to estimate heart rate from facial videos based on RPPG and RBCG,” Sensors (Basel), vol. 21, no. 20, p. 6764, Oct. 2021.
    OpenUrl
  125. [125].↵
    P. Li, Y. Benezeth, R. Macwan, K. Nakamura, R. Gomez, C. Li, and F. Yang, “Video-based pulse rate variability measurement using periodic variance maximization and adaptive Two-Window peak detection,” Sensors (Basel), vol. 20, no. 10, p. 2752, May 2020.
    OpenUrl
  126. [126].↵
    K. S. Alam, L. He, J. Ma, D. Das, M. Yap, B. Kerjner, S. Rezwan, A. Iqbal, and S. I. Ahamed, “Remote heart rate and heart rate variability detection and monitoring from face video with minimum resources,” in 2020 IEEE 44th Annual Computers, Software, and Applications Conference (COMPSAC). IEEE, Jul. 2020.
  127. [127].↵
    H. Demirezen and C. Eroglu Erdem, “Heart rate estimation from facial videos using nonlinear mode decomposition and improved consistency check,” Signal, Image and Video Processing, vol. 15, no. 7, pp. 1415–1423, 2021.
    OpenUrl
  128. [128].↵
    G. Casalino, G. Castellano, and G. Zaza, “Evaluating the robustness of a contact-less mhealth solution for personal and remote monitoring of blood oxygen saturation,” Journal of Ambient Intelligence and Humanized Computing, pp. 1–10, 2022.
  129. [129].
    M. Van Gastel, S. Stuijk, and G. De Haan, “New principle for measuring arterial blood oxygenation, enabling motion-robust remote monitoring,” Scientific reports, vol. 6, no. 1, pp. 1–16, 2016.
    OpenUrl
  130. [130].↵
    H. Monkaresi, R. A. Calvo, and H. Yan, “A machine learning approach to improve contactless heart rate monitoring using a webcam,” IEEE journal of biomedical and health informatics, vol. 18, no. 4, pp. 1153–1160, 2013.
    OpenUrl
  131. [131].
    B. D. Holton, K. Mannapperuma, P. J. Lesniewski, and J. C. Thomas, “Signal recovery in imaging photoplethysmography,” Physiological measurement, vol. 34, no. 11, p. 1499, 2013.
    OpenUrl
  132. [132].
    Y. Hsu, Y.-L. Lin, and W. Hsu, “Learning-based heart rate detection from remote photoplethysmography features,” in 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2014, pp. 4433–4437.
  133. [133].↵
    X. Li, J. Chen, G. Zhao, and M. Pietikainen, “Remote heart rate measurement from face videos under realistic situations,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2014, pp. 4264–4271.
  134. [134].
    Y. Yan, X. Ma, L. Yao, and J. Ouyang, “Noncontact measurement of heart rate using facial video illuminated under natural light and signal weighted analysis,” Bio-medical materials and engineering, vol. 26, no. 1, pp. S903–S909, 2015.
    OpenUrl
  135. [135].
    D. Lee, J. Kim, S. Kwon, and K. Park, “Heart rate estimation from facial photoplethysmography during dynamic illuminance changes,” in 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 2015, pp. 2758–2761.
  136. [136].↵
    D. N. Tran, H. Lee, and C. Kim, “A robust real time system for remote heart rate measurement via camera,” in 2015 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2015, pp. 1–6.
  137. [137].↵
    M. Kumar, A. Veeraraghavan, and A. Sabharwal, “Distanceppg: Robust non-contact vital signs monitoring using a camera,” Biomedical optics express, vol. 6, no. 5, pp. 1565–1588, 2015.
    OpenUrl
  138. [138].↵
    A. Lam and Y. Kuno, “Robust heart rate measurement from video using select random patches,” in Proceedings of the IEEE International Conference on Computer Vision, 015, pp. 3640–3648.
  139. [139].↵
    M. A. Haque, R. Irani, K. Nasrollahi, and T. B. Moeslund, “Heartbeat rate measurement from facial video,” IEEE Intelligent Systems, vol. 31, no. 3, pp. 40–48, 2016.
    OpenUrl
  140. [140].↵
    F. Bousefsaf, C. Maaoui, and A. Pruski, “Automatic selection of webcam photoplethysmographic pixels based on lightness criteria,” Journal of Medical and Biological Engineering, vol. 37, no. 3, pp. 374–385, 2017.
    OpenUrl
  141. [141].
    H. Qi, Z. Guo, X. Chen, Z. Shen, and Z. J. Wang, “Video-based human heart rate measurement using joint blind source separation,” Biomedical Signal Processing and Control, vol. 31, pp. 309–320, 2017.
    OpenUrl
  142. [142].
    L. Qi, H. Yu, L. Xu, R. S. Mpanda, and S. E. Greenwald, “Robust heartrate estimation from facial videos using project ica,” Physiological Measurement, vol. 40, no. 8, p. 085007, 2019.
    OpenUrl
  143. [143].
    H. Wang, Y. Zhou, and A. El Saddik, “Vitasi: A real-time contactless vital signs estimation system,” Computers & Electrical Engineering, vol. 95, p. 107392, 2021.
    OpenUrl
  144. [144].
    B. Huang, C.-L. Lin, W. Chen, C.-F. Juang, and X. Wu, “A novel one-stage framework for visual pulse rate estimation using deep neural networks,” Biomedical Signal Processing and Control, vol. 66, p. 102387, 2021.
    OpenUrl
  145. [145].↵
    X. Zheng, C. Zhang, H. Chen, Y. Zhang, and X. Yang, “Remote measurement of heart rate from facial video in different scenarios,” Measurement, vol. 188, p. 110243, 2022.
    OpenUrl
  146. [146].
    A. K. Noulas and B. J. Kröse, “Em detection of common origin of multi-modal cues,” in Proceedings of the 8th international conference on Multimodal interfaces, 2006, pp. 201–208.
  147. [147].↵
    S. Kwon, H. Kim, and K. S. Park, “Validation of heart rate extraction using video imaging on a built-in camera system of a smartphone,” in 2012 annual international conference of the IEEE engineering in medicine and biology society. IEEE, 2012, pp. 2174–2177.
  148. [148].↵
    L. Scalise, N. Bernacchia, I. Ercoli, and P. Marchionni, “Heart rate measurement in neonatal patients using a webcamera,” in 2012 IEEE International Symposium on Medical Measurements and Applications Proceedings. IEEE, 2012, pp. 1–4.
  149. [149].↵
    T. Pursche, J. Krajewski, and R. Moeller, “Video-based heart rate measurement from human faces,” in 2012 IEEE international conference on consumer electronics (ICCE). IEEE, 2012, pp. 544–545.
  150. [150].↵
    L. Wei, Y. Tian, Y. Wang, T. Ebrahimi, and T. Huang, “Automatic webcam-based human heart rate measurements using laplacian eigen-map,” in Asian Conference on Computer Vision. Springer, 2012, pp. 281–292.
  151. [151].↵
    Y.-P. Yu, B.-H. Kwan, C.-L. Lim, S.-L. Wong, and P. Raveendran, “Video-based heart rate measurement using short-time fourier transform,” in 2013 International Symposium on Intelligent Signal Processing and Communication Systems. IEEE, 2013, pp. 704–707.
  152. [152].↵
    P. Viola and M. J. Jones, “Robust real-time face detection,” International journal of computer vision, vol. 57, no. 2, pp. 137–154, 2004.
    OpenUrl
  153. [153].↵
    W. J. Jiang, S. C. Gao, P. Wittek, and L. Zhao, “Real-time quantifying heart beat rate from facial video recording on a smart phone using kalman filters,” in 2014 IEEE 16th International Conference on e-Health Networking, Applications and Services (Healthcom). IEEE, 2014, pp. 393–396.
  154. [154].↵
    S. Baker and I. Matthews, “Lucas-kanade 20 years on: A unifying framework,” International journal of computer vision, vol. 56, no. 3, pp. 221–255, 2004.
    OpenUrl
  155. [155].
    J. M. Saragih, S. Lucey, and J. F. Cohn, “Face alignment through subspace constrained mean-shifts,” in 2009 IEEE 12th International Conference on Computer Vision. Ieee, 2009, pp. 1034–1041.
  156. [156].
    J. Ryu, S. Hong, S. Liang, S. Pak, Q. Chen, and S. Yan, “A new framework for robust heart rate measurement based on the head motion state estimation,” IEEE Journal of Biomedical and Health Informatics, vol. 25, no. 9, pp. 3428–3437, 2021.
    OpenUrl
  157. [157].↵
    G. Casalino, G. Castellano, and G. Zaza, “A mhealth solution for contact-less self-monitoring of blood oxygen saturation,” in 2020 IEEE Symposium on Computers and Communications (ISCC), 2020, pp. 1–7.
  158. [158].
    K. Zhang, Z. Zhang, Z. Li, and Y. Qiao, “Joint face detection and alignment using multitask cascaded convolutional networks,” IEEE signal processing letters, vol. 23, no. 10, pp. 1499–1503, 2016.
    OpenUrlCrossRef
  159. [159].
    A. Zadeh, Y. Chong Lim, T. Baltrusaitis, and L.-P. Morency, “Con-volutional experts constrained local model for 3d facial landmark detection,” in Proceedings of the IEEE International Conference on Computer Vision Workshops, 2017, pp. 2519–2528.
  160. [160].↵
    G. E. Box, G. M. Jenkins, G. C. Reinsel, and G. M. Ljung, Time series analysis: forecasting and control. John Wiley & Sons, 2015.
  161. [161].↵
    A. K. Abbas, K. Heimann, K. Jergus, T. Orlikowsky, and S. Leonhardt, “Neonatal non-contact respiratory monitoring based on real-time infrared thermography,” Biomedical engineering online, vol. 10, no. 1, pp. 1–17, 2011.
    OpenUrl
  162. [162].↵
    S. Guler, O. Ozturk, A. Golparvar, H. Dogan, and M. K. Yapici, “Effects of illuminance intensity on the green channel of remote photoplethysmography (rPPG) signals,” Phys. Eng. Sci. Med., vol. 45, no. 4, pp. 1317–1323, Dec. 2022.
    OpenUrl
  163. [163].↵
    K. K. Bhoyar and O. Kakde, “Skin color detection model using neural networks and its performance evaluation,” in Journal of computer science. Citeseer, 2010.
  164. [164].↵
    P.-C. Hung, K.-Z. Lee, and L.-W. Tsai, “Contact-free heart rate measurement using multiple video data,” in AIP Conference Proceedings, vol. 1559, no. 1. American Institute of Physics, 2013, pp. 57–66.
    OpenUrl
  165. [165].↵
    W. Wang and A. C. den Brinker, “Modified RGB cameras for infrared remote-PPG,” IEEE Trans. Biomed. Eng., vol. 67, no. 10, pp. 2893–2904, Oct. 2020.
    OpenUrl
  166. [166].↵
    A. Panigrahi and H. Sharma, “Non-contact HR extraction from different color spaces using RGB camera,” in 2022 National Conference on Communications (NCC). IEEE, May 2022.
  167. [167].↵
    D. McDuff, S. Gontarek, and R. Picard, “Remote measurement of cognitive stress via heart rate variability,” in 2014 36th annual international conference of the IEEE engineering in medicine and biology society. IEEE, 2014, pp. 2957–2960.
  168. [168].↵
    C. Lueangwattana, T. Kondo, and H. Haneishi, “A comparative study of video signals for non-contact heart rate measurement,” in 2015 12th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON), 2015, pp. 1–5.
  169. [169].↵
    K. Mannapperuma, B. D. Holton, P. J. Lesniewski, and J. C. Thomas, “Performance limits of ica-based heart rate identification techniques in imaging photoplethysmography,” Physiological measurement, vol. 36, no. 1, p. 67, 2014.
    OpenUrl
  170. [170].↵
    E. Christinaki, G. Giannakakis, F. Chiarugi, M. Pediaditis, G. Iatraki, D. Manousos, K. Marias, and M. Tsiknakis, “Comparison of blind source separation algorithms for optical heart rate monitoring,” in 2014 4th International Conference on Wireless Mobile Communication and Healthcare-Transforming Healthcare Through Innovations in Mobile and Wireless Technologies (MOBIHEALTH). IEEE, 2014, pp. 339–342.
  171. [171].↵
    L. Qi, H. Yu, L. Xu, R. S. Mpanda, and S. E. Greenwald, “Robust heart-rate estimation from facial videos using Project ICA,” Physiol. Meas., vol. 40, no. 8, p. 085007, Sep. 2019.
    OpenUrl
  172. [172].↵
    R. Macwan, Y. Benezeth, and A. Mansouri, “Remote photoplethys-mography with constrained ICA using periodicity and chrominance constraints,” Biomed. Eng. Online, vol. 17, no. 1, p. 22, Feb. 2018.
    OpenUrl
  173. [173].↵
    L. K. Mestha, S. Kyal, B. Xu, L. E. Lewis, and V. Kumar, “Towards continuous monitoring of pulse rate in neonatal intensive care unit with a webcam,” in 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE, 2014, pp. 3817–3820.
  174. [174].↵
    A. V. Moco, S. Stuijk, and G. De Haan, “Ballistocardiographic artifacts in ppg imaging,” IEEE Transactions on Biomedical Engineering, vol. 63, no. 9, pp. 1804–1811, 2015.
    OpenUrl
  175. [175].↵
    J. R. Estepp, E. B. Blackford, and C. M. Meier, “Recovering pulse rate during motion artifact with a multi-imager array for non-contact imaging photoplethysmography,” in 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 2014, pp. 1462–1469.
  176. [176].↵
    S. M. Karmuse, A. L. Kakhandki, and M. Anandhalli, “A robust rppg approach for continuous heart rate measurement based on face,” Journal of The Institution of Engineers (India): Series B, pp. 1–12, 2022.
  177. [177].↵
    H.-Y. Wu, M. Rubinstein, E. Shih, J. Guttag, F. Durand, and W. Freeman, “Eulerian video magnification for revealing subtle changes in the world,” ACM transactions on graphics (TOG), vol. 31, no. 4, pp. 1–8, 2012.
    OpenUrl
  178. [178].↵
    B. Wallace, L. Y. Kassab, A. Law, R. Goubran, and F. Knoefel, “Contactless remote assessment of heart rate and respiration rate using video magnification,” IEEE Instrumentation & Measurement Magazine, vol. 25, no. 1, pp. 20–27, 2022.
    OpenUrl
  179. [179].↵
    J. Zhang, K. Zhang, X. Yang, and C. Wen, “Heart rate measurement based on video acceleration magnification,” in 2020 Chinese Control And Decision Conference (CCDC). IEEE, Aug. 2020.
  180. [180].↵
    C.-H. Cheng, K.-L. Wong, J.-W. Chin, T.-T. Chan, and R. H. Y. So, “Deep learning methods for remote heart rate measurement: A review and future research agenda,” Sensors (Basel), vol. 21, no. 18, p. 6296, Sep. 2021.
    OpenUrl
  181. [181].↵
    B. Li, W. Jiang, J. Peng, and X. Li, “Deep learning-based remote-photoplethysmography measurement from short-time facial video,” Physiol. Meas., vol. 43, no. 11, Nov. 2022.
  182. [182].↵
    Y. Sun, Y.-Y. Yang, B.-J. Wu, P.-W. Huang, S.-E. Cheng, B.-F. Wu, and C.-C. Chen, “Contactless facial video recording with deep learning models for the detection of atrial fibrillation,” Sci. Rep., vol. 12, no. 1, p. 281, Jan. 2022.
    OpenUrl
  183. [183].↵
    Q. E, “Pulse signal analysis based on deep learning network,” Biomed Res. Int., vol. 2022, p. 6256126, Sep. 2022.
  184. [184].↵
    B. Lokendra and G. Puneet, “AND-rPPG: A novel denoising-rPPG network for improving remote heart rate estimation,” Comput. Biol. Med., vol. 141, no. 105146, p. 105146, Feb. 2022.
    OpenUrl
  185. [185].↵
    L. C. Lampier, C. T. Valadaõ, L. A. Silva, D. Delisle-Rodríguez, E. M. d. O. Caldeira, and T. F. Bastos-Filho, “A deep learning approach to estimate pulse rate by remote photoplethysmography,” Physiol. Meas., vol. 43, no. 7, p. 075012, Jul. 2022.
    OpenUrl
  186. [186].↵
    A. Al-Naji, K. Gibson, and J. Chahl, “Remote sensing of physiological signs using a machine vision system,” J. Med. Eng. Technol., vol. 41, no. 5, pp. 396–405, Jul. 2017.
    OpenUrl
  187. [187].↵
    H. Wang, D. Gao, G. Chen, Y. Jing, and M. Wang, “KLT algorithm for non-contact heart rate detection based on image photoplethysmography,” in 2022 5th International Conference on Artificial Intelligence and Big Data (ICAIBD). IEEE, May 2022.
  188. [188].↵
    M. Soleymani, J. Lichtenauer, T. Pun, and M. Pantic, “A multimodal database for affect recognition and implicit tagging,” IEEE transactions on affective computing, vol. 3, no. 1, pp. 42–55, 2011.
    OpenUrl
  189. [189].↵
    M. Finžgar and P. Podržaj, “A wavelet-based decomposition method for a robust extraction of pulse rate from video recordings,” PeerJ, vol. 6, no. e5859, p. e5859, Nov. 2018.
    OpenUrl
  190. [190].↵
    X. Niu, H. Han, S. Shan, and X. Chen, “Continuous heart rate measurement from face: A robust rPPG approach with distribution learning,” in 2017 IEEE International Joint Conference on Biometrics (IJCB). IEEE, Oct. 2017.
  191. [191].↵
    R. Song, S. Zhang, C. Li, Y. Zhang, J. Cheng, and X. Chen, “Heart rate estimation from facial videos using a spatiotemporal representation with convolutional neural networks,” IEEE Trans. Instrum. Meas., vol. 69, no. 10, pp. 7411–7421, Oct. 2020.
    OpenUrl
  192. [192].↵
    M. A. Hassan, A. S. Malik, D. Fofi, B. Karasfi, and F. Meriaudeau, “Towards health monitoring using remote heart rate measurement using digital camera: A feasibility study,” Measurement (Lond.), vol. 149, no. 106804, p. 106804, Jan. 2020.
    OpenUrl
  193. [193].↵
    R. Stricker, S. Müller, and H.-M. Gross, “Non-contact video-based pulse rate measurement on a mobile service robot,” in The 23rd IEEE International Symposium on Robot and Human Interactive Communication. IEEE, 2014, pp. 1056–1062.
  194. [194].↵
    S. Petridis, B. Martinez, and M. Pantic, “The mahnob laughter database,” Image and Vision Computing, vol. 31, no. 2, pp. 186–202, 2013, affect Analysis In Continuous Input. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0262885612001461
    OpenUrl
  195. [195].↵
    J. Lichtenauer, J. Shen, M. Valstar, and M. Pantic, “Cost-effective solution to synchronised audio-visual data capture using multiple sensors,” Image and Vision Computing, vol. 29, no. 10, pp. 666–680, 2011.
    OpenUrl
  196. [196].↵
    G. Heusch, A. Anjos, and S. Marcel, “A reproducible study on remote heart rate measurement,” arXiv preprint arXiv:1709.00962, 2017.
  197. [197].↵
    Z. Zhang, J. M. Girard, Y. Wu, X. Zhang, P. Liu, U. Ciftci, S. Canavan, M. Reale, A. Horowitz, H. Yang et al., “Multimodal spontaneous emotion corpus for human behavior analysis,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 3438–3446.
  198. [198].↵
    Z. Yang, H. Wang, and F. Lu, “Assessment of deep learning-based heart rate estimation using remote photoplethysmography under different illuminations,” IEEE Transactions on Human-Machine Systems, vol. 52, no. 6, pp. 1236–1246, 2022.
    OpenUrl
  199. [199].↵
    Z. Hasan, S. R. Ramamurthy, and N. Roy, “Mpsc-rppg dataset,” 2021. [Online]. Available: doi:10.21227/ddgz-tx88
    OpenUrlCrossRef
  200. [200].↵
    S. Bobbia, R. Macwan, Y. Benezeth, A. Mansouri, and J. Dubois, “Unsupervised skin tissue segmentation for remote photoplethysmography,” Pattern Recognition Letters, vol. 124, pp. 82–90, 2019, award Winning Papers from the 23rd International Conference on Pattern Recognition (ICPR). [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0167865517303860
    OpenUrl
  201. [201].↵
    P. Pirzada, “Automated Remote Pulse Oximetry System (ARPOS),” May 2022. [Online]. Available: doi:10.5281/zenodo.6522389
    OpenUrlCrossRef
  202. [202].↵
    L.-M. Po, L. Feng, Y. Li, X. Xu, T. C.-H. Cheung, and K.-W. Cheung, “Block-based adaptive roi for remote photoplethysmography,” Multimedia Tools and Applications, vol. 77, no. 6, pp. 6503–6529, 2018.
    OpenUrl
  203. [203].↵
    A. Woyczyk, V. Fleischhauer, and S. Zaunseder, “Adaptive gaussian mixture model driven level set segmentation for remote pulse rate detection,” IEEE J. Biomed. Health Inform., vol. 25, no. 5, pp. 1361–1372, May 2021.
    OpenUrl
  204. [204].↵
    R. Irani, K. Nasrollahi, and T. B. Moeslund, “Improved pulse detection from head motions using dct,” in 2014 international conference on computer vision theory and applications (VISAPP), vol. 3. IEEE, 2014, pp. 118–124.
    OpenUrl
  205. [205].↵
    J.-P. Lomaliza and H. Park, “Detecting pulse from head motions using smartphone camera,” in International Conference on Advanced Engineering Theory and Applications. Springer, 2016, pp. 243–251.
  206. [206].↵
    G. Balakrishnan, “Analyzing pulse from head motions in video,” Ph.D. dissertation, Massachusetts Institute of Technology, 2014.
  207. [207].↵
    H. Lee, A. Cho, S. Lee, and M. Whang, “Vision-based measurement of heart rate from ballistocardiographic head movements using unsu-pervised clustering,” Sensors, vol. 19, no. 15, p. 3263, 2019.
    OpenUrl
  208. [208].↵
    A. Al-Naji and J. Chahl, “Contactless cardiac activity detection based on head motion magnification,” International journal of image and graphics, vol. 17, no. 01, p. 1750001, 2017.
    OpenUrl
  209. [209].↵
    X. Liu, X. Yang, Z. Meng, Y. Wang, J. Zhang, and A. Wong, “Manet: a motion-driven attention network for detecting the pulse from a facial video with drastic motions,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 2385–2390.
  210. [210].↵
    M. van Gastel, S. Stuijk, and G. de Haan, “Motion robust remote-PPG in infrared,” IEEE Trans. Biomed. Eng., vol. 62, no. 5, pp. 1425–1433, May 2015.
    OpenUrl
  211. [211].↵
    W. Wang, S. Stuijk, and G. de Haan, “Exploiting spatial redundancy of image sensor for motion robust rPPG,” IEEE Trans. Biomed. Eng., vol. 62, no. 2, pp. 415–425, Feb. 2015.
    OpenUrl
  212. [212].↵
    Y. Yang, C. Liu, H. Yu, D. Shao, F. Tsow, and N. Tao, “Motion robust remote photoplethysmography in CIELab color space,” J. Biomed. Opt., vol. 21, no. 11, p. 117001, Nov. 2016.
    OpenUrl
  213. [213].↵
    B.-F. Wu, B.-J. Wu, S.-E. Cheng, Y. Sun, and M.-L. Chung, “Motion-robust atrial fibrillation detection based on remote-photoplethysmography,” IEEE J. Biomed. Health Inform., vol. PP, May 2022.
  214. [214].↵
    H. Wang, X. Yang, X. Liu, and D. Wang, “Heart rate estimation from facial videos with motion interference using T-SNE-based signal separation,” Biomed. Opt. Express, vol. 13, no. 9, pp. 4494–4509, Sep. 2022.
    OpenUrl
  215. [215].↵
    Y. Liu, B. Qin, R. Li, X. Li, A. Huang, H. Liu, Y. Lv, and M. Liu, “Motion-robust multimodal heart rate estimation using BCG fused remote-PPG with deep facial ROI tracker and pose constrained kalman filter,” IEEE Trans. Instrum. Meas., vol. 70, pp. 1–15, 2021.
    OpenUrl
  216. [216].↵
    D. Cho, J. Kim, K. J. Lee, and S. Kim, “Reduction of motion artifacts from remote photoplethysmography using adaptive noise cancellation and modified HSI model,” IEEE Access, vol. 9, pp. 122 655–122 667, 2021.
    OpenUrl
  217. [217].↵
    M. A. Hassan, A. S. Malik, N. Saad, D. Fofi, and F. Meriaudeau, “Effect of motion artifact on digital camera based heart rate measurement,” Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., vol. 2017, pp. 2851–2854, Jul. 2017.
    OpenUrl
  218. [218].↵
    S. K. A. Prakash and C. S. Tucker, “Bounded kalman filter method for motion-robust, non-contact heart rate estimation,” Biomed. Opt. Express, vol. 9, no. 2, pp. 873–897, Feb. 2018.
    OpenUrl
  219. [219].↵
    Y. Sun, S. Hu, V. Azorin-Peris, S. Greenwald, J. Chambers, and Y. Zhu, “Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise,” J. Biomed. Opt., vol. 16, no. 7, p. 077010, Jul. 2011.
    OpenUrlPubMed
  220. [220].↵
    H. Rahman, M. U. Ahmed, and S. Begum, “Non-contact physiological parameters extraction using facial video considering illumination, motion, movement and vibration,” IEEE Trans. Biomed. Eng., vol. 67, no. 1, pp. 88–98, Jan. 2020.
    OpenUrl
  221. [221].↵
    N. S. Suriani, N. A. Jumain, A. A. Ali, and N. H. Mohd, “Facial video based heart rate estimation for physical exercise,” in 2021 IEEE Symposium on Industrial Electronics & Applications (ISIEA). IEEE, Jul. 2021.
  222. [222].↵
    Y. Napolean, A. Marwade, N. Tomen, P. Alkemade, T. Eijsvogels, and J. C. van Gemert, “Heart rate estimation in intense exercise videos,” in 2022 IEEE International Conference on Image Processing (ICIP). IEEE, Oct. 2022.
  223. [223].↵
    M. Hu, F. Qian, X. Wang, L. He, D. Guo, and F. Ren, “Robust heart rate estimation with spatial–temporal attention network from facial videos,” IEEE Trans. Cogn. Dev. Syst., vol. 14, no. 2, pp. 639–647, Jun. 2022.
    OpenUrl
  224. [224].↵
    R. P. Dresher and Y. Mendelson, “Reflectance forehead pulse oximetry: Effects of contact pressure during walking,” in 2006 International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE, Aug. 2006.
  225. [225].↵
    L. Shan and M. Yu, “Video-based heart rate measurement using head motion tracking and ica,” in 2013 6th International Congress on Image and Signal Processing (CISP), vol. 1. IEEE, 2013, pp. 160–164.
    OpenUrl
  226. [226].↵
    M. A. Adelabu, A. L. Imoize, and K. E. Adesoji, “Enhancement of a camera-based continuous heart rate measurement algorithm,” SN Computer Science, vol. 3, no. 4, p. 284, 2022.
    OpenUrl
  227. [227].↵
    1. J. Spigulis
    E. Kviesis-Kipge and U. Rubīns, “Remote photoplethysmography device with adaptive illumination for skin microcirculation assessment,” in Biophotonics—Riga 2020, J. Spigulis, Ed., vol. 11585, International Society for Optics and Photonics. SPIE, 2020, p. 1158505. [Online]. Available: doi:10.1117/12.2581933
    OpenUrlCrossRef
  228. [228].↵
    W. Wang, A. C. den Brinker, S. Stuijk, and G. de Haan, “Robust heart rate from fitness videos,” Physiological measurement, vol. 38, no. 6, p. 1023, 2017.
    OpenUrl
  229. [229].↵
    P. Gupta, B. Bhowmick, and A. Pal, “MOMBAT: Heart rate monitoring from face video using pulse modeling and bayesian tracking,” Comput. Biol. Med., vol. 121, no. 103813, p. 103813, Jun. 2020.
    OpenUrl
  230. [230].↵
    S. H. Fouladi, I. Balasingham, T. A. Ramstad, and K. Kansanen, “Accurate heart rate estimation from camera recording via MUSIC algorithm,” in 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, Aug. 2015.
  231. [231].↵
    W. Wang, L. Vosters, and A. C. den Brinker, “Modified camera setups for day-and-night pulse-rate monitoring,” in 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE, Nov. 2021.
  232. [232].↵
    X. Yang, Z. Zhang, Y. Huang, Y. Zheng, and Y. Shen, “Using a graph-based image segmentation algorithm for remote vital sign estimation and monitoring,” Sci. Rep., vol. 12, no. 1, p. 15197, Sep. 2022.
    OpenUrl
  233. [233].↵
    L. H. Nair and P. R. Aparna, “Non-contact heart-rate measurement using KLT-algorithm,” in 2018 Second International Conference on Electronics, Communication and Aerospace Technology (ICECA). IEEE, Mar. 2018.
  234. [234].↵
    R.-N. Yin, R.-S. Jia, Z. Cui, J.-T. Yu, Y.-B. Du, L. Gao, and H.-M. Sun, “Heart rate estimation based on face video under unstable illumination,” Applied Intelligence, pp. 1–17, 2021.
  235. [235].↵
    Yu-Chen Lin and Yuan-Hsiang Lin, “A study of color illumination effect on the SNR of rPPG signals,” Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., vol. 2017, pp. 4301–4304, Jul. 2017.
    OpenUrl
  236. [236].↵
    W. Verkruysse, L. O. Svaasand, and J. S. Nelson, “Remote plethysmographic imaging using ambient light,” Opt. Express, vol. 16, no. 26, pp. 21 434–21 445, Dec. 2008.
    OpenUrlCrossRefPubMed
  237. [237].↵
    L. Tarassenko, M. Villarroel, A. Guazzi, J. Jorge, D. Clifton, and C. Pugh, “Non-contact video-based vital sign monitoring using ambient light and auto-regressive models,” Physiological measurement, vol. 35, no. 5, p. 807, 2014.
    OpenUrlCrossRef
  238. [238].↵
    Y. Kurzweil-Segev, M. Brodsky, A. Polsman, E. Safrai, Y. Feldman, S. Einav, and P. Ben Ishai, “Remote monitoring of phasic heart rate changes from the palm,” IEEE Trans. Terahertz Sci. Technol., vol. 4, no. 5, pp. 618–623, Sep. 2014.
    OpenUrl
  239. [239].↵
    V. Selvaraju, N. Spicher, J. Wang, N. Ganapathy, J. M. Warnecke, S. Leonhardt, R. Swaminathan, and T. M. Deserno, “Continuous monitoring of vital signs using cameras: A systematic review,” Sensors (Basel), vol. 22, no. 11, p. 4097, May 2022.
    OpenUrl
  240. [240].↵
    M. Attas, M. Hewko, J. Payette, T. Posthumus, M. Sowa, and H. Mantsch, “Visualization of cutaneous hemoglobin oxygenation and skin hydration using near-infrared spectroscopic imaging,” Skin Research and Technology, vol. 7, no. 4, pp. 238–245, 2001.
    OpenUrlCrossRefPubMedWeb of Science
  241. [241].↵
    K. J. Zuzak, M. D. Schaeberle, E. N. Lewis, and I. W. Levin, “Visible reflectance hyperspectral imaging: characterization of a noninvasive, in vivo system for determining tissue perfusion,” Analytical chemistry, vol. 74, no. 9, pp. 2021–2028, 2002.
    OpenUrlCrossRefPubMed
  242. [242].↵
    K. J. Zuzak, R. P. Francis, E. F. Wehner, M. Litorja, J. A. Cadeddu, and E. H. Livingston, “Active dlp hyperspectral illumination: a noninvasive, in vivo, system characterization visualizing tissue oxygenation at near video rates,” Analytical chemistry, vol. 83, no. 19, pp. 7424–7430, 2011.
    OpenUrlPubMed
  243. [243].↵
    F. Wieringa, F. Mastik, R. H. Boks, A. Visscher, A. Bogers, and A. Van der Steen,In vitro demonstration of an SpO2-camera. IEEE, 2007.
  244. [244].↵
    F. P. Wieringa, F. Mastik, and A. F. van der Steen, “Contactless multiple wavelength photoplethysmographic imaging: a first step toward “spo2 camera” technology,” Annals of biomedical engineering, vol. 33, no. 8, pp. 1034–1041, 2005.
    OpenUrlPubMed
  245. [245].↵
    C.-J. Wu, A. Quigley, and D. Harris-Birtill, “Out of sight: A toolkit for tracking occluded human joint positions,” Personal and Ubiquitous Computing, vol. 21, no. 1, pp. 125–135, 2017.
    OpenUrl
  246. [246].↵
    D. Avola, L. Cinque, S. Levialdi, and G. Placidi, “Human body language analysis: a preliminary study based on kinect skeleton tracking,” in International Conference on Image Analysis and Processing. Springer, 2013, pp. 465–473.
  247. [247].↵
    T. Ballendat, N. Marquardt, and S. Greenberg, “Proxemic interaction: designing for a proximity and orientation-aware environment,” in ACM International Conference on Interactive Tabletops and Surfaces, 2010, pp. 121–130.
  248. [248].↵
    V. Frati and D. Prattichizzo, “Using kinect for hand tracking and rendering in wearable haptics,” in 2011 IEEE World Haptics Conference. IEEE, 2011, pp. 317–321.
  249. [249].↵
    L. A. Seewald, L. Gonzaga Jr, M. R. Veronez, V. P. Minotto, and C. R. Jung, “Combining srp-phat and two kinects for 3d sound source localization,” Expert systems with applications, vol. 41, no. 16, pp. 7106–7113, 2014.
    OpenUrl
  250. [250].↵
    W. Gai, M. Qi, M. Ma, L. Wang, C. Yang, J. Liu, Y. Bian, G. de Melo, S. Liu, and X. Meng, “Employing shadows for multi-person tracking based on a single rgb-d camera,” Sensors, vol. 20, no. 4, p. 1056, 2020.
    OpenUrl
  251. [251].↵
    W. Wang and C. Shan, “Impact of makeup on remote-PPG monitoring,” Biomed. Phys. Eng. Express, vol. 6, no. 3, p. 035004, Mar. 2020.
    OpenUrl
Back to top
Posted October 12, 2023.
Download PDF
Data/Code
Email
Remote Photoplethysmography (rPPG): A State-of-the-Art Review
Pireh Pirzada, Adriana Wilde, Gayle H Doherty, David Harris-Birtill
medRxiv 2023.10.12.23296882; doi: https://doi.org/10.1101/2023.10.12.23296882
Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools

Subject Area