Development and validation of a novel clinical risk score to predict hypoxemia in children with pneumonia using the WHO PREPARE dataset ======================================================================================================================================= * Rainer Tan * Arjun Chandna * Tim Colbourn * Shubhada Hooli * Carina King * Norman Lufesi * Eric D McCollum * Charles Mwansambo * Joseph L. Mathew * Clare Cutland * Shabir Ahmed Madhi * Marta Nunes * Sudha Basnet * Tor A. Strand * Kerry-Ann O’Grady * Brad Gessner * Emmanuel Addo-Yobo * Noel Chisaka * Patricia L. Hibberd * Prakash Jeena * Juan M. Lozano * William B. MacLeod * Archana Patel * Donald M. Thea * Ngoc Tuong Vy Nguyen * Marilla Lucero * Syed Mohammad Akram uz Zaman * Shinjini Bhatnagar * Nitya Wadhwa * Rakesh Lodha * Satinder Aneja * Mathuram Santosham * Shally Awasthi * Ashish Bavdekar * Monidarin Chou * Pagbajabyn Nymadawa * Jean-William Pape * Glaucia Paranhos-Baccala * Valentina S. Picot * Mala Rakoto-Andrianarivelo * Vanessa Rouzier * Graciela Russomando * Mariam Sylla * Philippe Vanhems * Jianwei Wang * Romina Libster * Alexey W. Clara * Fenella Beynon * Gillian Levine * Chris A Rees * Mark I Neuman * Shamim A. Qazi * Yasir Bin Nisar * World Health Organization PREPARE study group ## ABSTRACT **Background** Hypoxemia predicts mortality at all levels of care, and appropriate management can reduce preventable deaths. However, pulse oximetry and oxygen therapy remain inaccessible in many primary care health facilities. We aimed to develop and validate a simple risk score comprising commonly evaluated clinical features to predict hypoxemia in 2-59-month-old children with pneumonia. **Methods** Data from 7 studies conducted in 5 countries from the Pneumonia Research Partnership to Assess WHO Recommendations (PREPARE) dataset were included. Readily available clinical features and demographic variables were used to develop a multivariable logistic regression model to predict hypoxemia (SpO2<90%) at presentation to care. The adjusted log coefficients were transformed to derive the PREPARE hypoxemia risk score and its diagnostic value was assessed in a held-out, temporal validation dataset. **Results** We included 14,509 children in the analysis; 9.8% (n=2,515) were hypoxemic at presentation. The multivariable regression model to predict hypoxemia included age, sex, respiratory distress (nasal flaring, grunting and/or head nodding), lower chest indrawing, respiratory rate, body temperature and weight-for-age z-score. The model showed fair discrimination (area under the curve 0.70, 95% CI 0.67 to 0.73) and calibration in the validation dataset. The simplified PREPARE hypoxemia risk score includes 5 variables: age, respiratory distress, lower chest indrawing, respiratory rate and weight-for-age z-score. **Conclusion** The PREPARE hypoxemia risk score, comprising five easily available characteristics, can be used to identify hypoxemia in children with pneumonia with a fair degree of certainty for use in health facilities without pulse oximetry. Its implementation would require careful consideration to limit inappropriate referrals on patients and the health system. Further external validation in community settings in low-and middle-income countries is required. **What is already known on this topic** * Pulse oximetry is unavailable or underutilized in many resource-limited settings in low- and middle-income countries. * Hypoxemia is a good predictor of mortality and its early identification and further management can reduce mortality. **What this study adds** * The PREPARE hypoxemia risk score was developed using one of the largest and most geographically diverse datasets on childhood pneumonia to date. * Using age, lower chest indrawing, respiratory rate, respiratory distress and weight-for-age z-score to calculate the PREPARE hypoxemia risk score could help identify children with hypoxemia in settings without pulse oximeters. **How this study might affect research, practice or policy** * This study contributes to the important discussion on how best to identify hypoxemic children in the absence of pulse oximetry. * Further research is warranted to validate the findings in community settings * Operationalizing and integrating the score within existing clinical management pathways must be tailored to the setting of implementation. ## INTRODUCTION Hypoxemia – low blood oxygen levels, defined as an oxygen saturation (SpO2) of <90%, is common in children with World Health Organization (WHO) classified pneumonia in low- and middle-income countries (LMICs).1 It is a good predictor of preventable deaths,2-4 and early identification of hypoxemia and appropriate management can reduce mortality.5-8 The availability of pulse oximetry to measure SpO2 impacts clinicians’ assessment of illness severity, diagnosis, treatment and need for referral.6,9 As such, there is increasing rationale to support universal access to pulse oximetry in primary care settings to address the high burden of pneumonia-related mortality.10-13 Although the availability of pulse oximetry is increasing, in part due to increased focus during the COVID-19 pandemic,10 access is limited and inequitable and pulse oximetry remains underutilized in many LMIC settings, particularly in the community and primary care health facilities.14-17 Barriers to the implementation of pulse oximetry in LMICs include high cost of quality pediatric devices with appropriately sized probes, maintenance, requisite training, battery charging and patient-related factors (e.g. crying and movement), which can be particularly challenging among young children.18-20 Furthermore, performing pulse oximetry can take time, especially in agitated children; an important consideration for implementation in health facilities with high case-loads or insufficient staff-to-patient ratios. In primary care level health facilities without pulse oximetry, where the WHO’s Integrated Management of Childhood Illnesses (IMCI) chartbook21 is used to guide the management of sick children, many hypoxemic children are not identified.6,19 In health facilities where a pulse oximeter is not available, identifying children at the highest risk of being hypoxemic using clinical signs alone could improve the sensitivity of the IMCI chartbook for identification of children in need of oxygen therapy, hospitalization, or referral. Previous clinical scores to predict hypoxemia were limited in sample size, geographical representation, or included patients already identified as requiring referral based on other clinical features.22-26 We aimed to develop and validate a clinical risk score to predict hypoxemia (SpO2 <90%) at presentation to a health facility in children aged 2-59 months with pneumonia according to the 2014 WHO IMCI definition (i.e. cough and/or difficulty breathing with fast breathing and/or lower chest indrawing) using the Pneumonia REsearch Partnership to Assess WHO REcommendations (PREPARE) dataset.21,27 The goal is not to replace the pulse oximeter, but to guide referral to health facilities where children with suspected hypoxemia can be appropriately managed. ## METHODS ### Study design We used the WHO PREPARE dataset, to derive and validate a clinical risk score to identify hypoxemia in children 2-59 months with pneumonia. The PREPARE dataset included primary, patient-level data for children 0-59 months with pneumonia, from 45 separate studies conducted from 1994 to 2018, in over 20 countries in Asia, Africa, Latin America, Oceania and North America.27 Detailed description of this dataset is found elsewhere.27 We adhered to the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) guidelines.28 ### Patient and public involvement The development of the research question was informed by the high rates of pneumonia-related mortality, specifically in children with hypoxemia. Patients were neither advisers in this study nor were they involved in the design, recruitment or conduct of the study. Results of this study will be made publicly available through open-access publication. ### Study population For the present analysis, included studies that reported SpO2 measured at the same time as assessment of clinical symptoms and signs (within 2 hours) and for which pulse oximetry was performed in >=80% of cases. Studies in which fewer than 75% of patients had data for each candidate predictor were excluded. Patients aged 2-59 months with IMCI defined pneumonia (irrespective of SpO2 reading) presenting to health facilities were included. Patients with severe pneumonia (defined as cough and/or difficulty breathing with a general danger sign including stridor in a calm child), or complicated severe acute malnutrition were excluded, since they would already have met IMCI criteria for referral.21,29 ### Outcome The primary outcome was hypoxemia, defined as a SpO2 of <90%,21, 27 assessed at the same time as the clinical predictors (symptoms and signs). A sensitivity analysis assessed the development of a model for the prediction of blood oxygen saturation at a SpO2 value of <92%. This threshold was selected acknowledging the high risk of mortality in children at this threshold,13 and the use of this threshold by various clinical guidelines.30,31 ### Candidate Predictors Candidate predictors were selected a priori based on expert knowledge and previous diagnostic value for predicting hypoxemia, identified in the literature,26 and considering reliability, feasibility and resources available at typical primary care health facilities in LMICs. The seven candidate predictors were: sex, age, chest indrawing, respiratory rate, axillary temperature, weight-for-age z-score and respiratory distress (composite predictor defined as the presence of either nasal flaring, grunting and/or head nodding). A sensitivity analysis assessed the performance of the model without respiratory distress, with and without wheezing. Wheezing was included in a sensitivity analysis because it is difficult to identify without the use of a stethoscope, often absent or infrequently used in primary care health facilities in LMICs, however included within IMCI.21,32 In the analyses without respiratory distress as a candidate predictor, we excluded all patients with signs of respiratory distress. Given the variety of aims and methods among studies in the PREPARE dataset, some clinical predictors may have been measured with knowledge of the outcome status (peripheral oxygen saturation). Duration of cough or difficulty breathing, history of fever, duration of fever, mid-upper arm circumference, weight-for-height z-score, tracheal tugging, heart rate, cyanosis and capillary refill time, were considered as candidate predictors based on potential clinical relevance, but were not included due to high levels of missingness in the data. ### Sample size We used 75% of the dataset for the development of the score and 25% for validation. Each study dataset was split by reserving the last 25% of enrolled patients for the validation dataset. Such a temporal validation strategy is considered intermediate between internal and external validation.28 Using the approach outlined by Riley et al.,33 659 cases of hypoxemia would be required in order to have a sufficient sample size to include the 12 pre-specified parameters (from the 7 candidate predictors) to develop the score (54.9 events per parameter), considering a conservative R2 Nagelkerke of 0.15, shrinkage factor of 0.9 and outcome prevalence of 9.8% in the development dataset. The development dataset of 10,884 children contained 1,146 cases of hypoxemia (i.e. 95.5 outcome events per parameter). The temporal validation dataset included 275 outcome events, well above the recommended 100 events often recommended for a robust validation.34,35 ### Analysis Pulse oximetry measurements were not adjusted for altitude because no sites were greater than 2,500 meters above sea level.36 Continuous predictors were categorized a priori based on currently accepted thresholds in order to maximize interpretability, encourage uptake amongst healthcare providers and policy makers and align with current clinical practices: age: 2-5 months, 6-11 months, 12-59 months; temperature: <35.5, 35.5-37.4, >=37.5°C; respiratory rate: 0-9, 10-19, >=20 breaths/min above IMCI age-specific cut-offs (age 2-11 months >=50 breaths per minute; age 12-59 months >=40 breaths per minute); and weight-for-age z-score: >= -2, < -2 to -3, < -3. Patients with missing data were excluded as multiple imputation was not feasible as it was determined that data were unlikely to be missing at random.37 The model was developed using multivariable logistic regression with the least absolute shrinkage and selection operator (LASSO) with L1 regularization. The adjusted model included all parameters and was further adjusted for study. Associations with 95% confidence intervals (CI) for adjusted odds ratios (aOR) that did not cross 1 were considered significant. Discrimination was evaluated by calculating the area under the receiver operating characteristic (AUROC) curve of the model in both the development and validation datasets. Calibration was evaluated using calibration plots (observed risk on y-axis, predicted risk on the x-axis) in the validation dataset. In order to convert the model into a simple clinical risk score, the regression coefficient of each retained parameter with a statistically significant adjusted odds ratio (CI did not overlap with 1.0) was rounded to the nearest 0.5 and then doubled.38,39 Sensitivity, specificity, likelihood ratios, miss rate (proportion of patients with hypoxemia not identified), false discovery rate (proportion of patients inappropriately identified as having hypoxemia among all patients identified as having hypoxemia) and proportion of children identified as being hypoxemic at each of the score’s cut offs was evaluated in the temporal validation dataset. Recognizing that the appropriate threshold for referring a child with suspected hypoxemia would be context dependent, a decision curve analysis compared the clinical utility (net benefit) of the model and score across a range of clinically-plausible referral thresholds. The net benefit quantifies the trade-off between true positives (hypoxemic children predicted to have hypoxemia) and false positives (non-hypoxemic children incorrectly predicted to have hypoxemia), weighted according to the relative cost of a false positive (threshold probability), which would vary in different contexts. The threshold probability reflects the cut-off (predicted probability of hypoxemia) above which a given intervention (in this case, referral) might be considered. All analyses were performed using STATA version 16.40 ## RESULTS ### Baseline characteristics The PREPARE dataset included 294,968 children from 45 studies, of which 14,509 children from 7 studies met inclusion criteria and were included in this analysis (figure 1). Four studies were randomized controlled trials, and three were prospective cohort studies (table 1). All studies included outpatients presenting to hospitals, and one study also included patients presenting to health centers.41 Among the 7 studies included in this analysis, patients were recruited from 5 countries: Australia, Gambia, India, Malawi and Nepal (table 1). Of the 14,509 patients with pneumonia, 9.8% had a SpO2 less than 90% at presentation (table 2). View this table: [Table 1:](http://medrxiv.org/content/early/2024/08/20/2024.08.19.24312238/T1) Table 1: Characteristics of studies included in the development and validation of the PREPARE hypoxemia risk score View this table: [Table 2.](http://medrxiv.org/content/early/2024/08/20/2024.08.19.24312238/T2) Table 2. Baseline characteristics of children with pneumonia (fast breathing and/or chest indrawing) with pulse oximetry measurements and complete data on all candidate predictors (n=14,509) ![Figure 1:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/08/20/2024.08.19.24312238/F1.medium.gif) [Figure 1:](http://medrxiv.org/content/early/2024/08/20/2024.08.19.24312238/F1) Figure 1: Flow diagram of PREPARE study dataset describing which datasets were excluded/included, the split between development and validation datasets and proportion of patients meeting the primary outcome (SpO2 <90%) Danger signs were defined by the IMCI8 chartbook (i.e., inability to drink, lethargy or unconsciousness, convulsions, vomiting everything, or stridor in a calm child). Baseline characteristics between the development (n=10,884, 75%) and validation dataset (n=3,625, 25%) were similar (table 2). Within the total dataset (n=14,509), 54.9% of children were aged less than 1 year, there were more male patients than female patients, 62.0% of children had lower chest indrawing, 34.1% had a temperature of equal to or more than 37.5°C and 6.7% had a very low weight-for-age z-score (< -3). ### Model development The predictors most strongly associated with hypoxemia (SpO2 <90%) were respiratory rate >/= 20 breaths/minute above the IMCI age-adjusted cut-off (aOR 2.41, 95% CI 1.25 to 4.64), any sign of respiratory distress (aOR 2.21, 95% CI 1.90 to 2.58), weight for age z-score <-3 (aOR 1.96, 95% CI 1.56 to 2.42) and lower chest indrawing (aOR 1.62, 95% CI 1.35 to 1.95) (table 3). None of the 12 candidate parameters were eliminated by the LASSO penalty during model development and hence, all 7 candidate predictors were retained in the final model. View this table: [Table 3.](http://medrxiv.org/content/early/2024/08/20/2024.08.19.24312238/T3) Table 3. Multivariable regression model for predicting hypoxemia in the development dataset (n=10,884) ### Model performance The PREPARE hypoxemia clinical prediction model had an AUROC of 0.70 (95% CI 0.69 to 0.72) in the development dataset and an AUROC of 0.70 (95% CI 0.68 to 0.73) in the temporal validation dataset (figure 2). Calibration of the PREPARE hypoxemia prediction model in the validation dataset is illustrated in figure 3. ![Figure 2.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/08/20/2024.08.19.24312238/F2.medium.gif) [Figure 2.](http://medrxiv.org/content/early/2024/08/20/2024.08.19.24312238/F2) Figure 2. Receiver operating characteristic curve for the PREPARE hypoxemia clinical prediction model for children 2–59 months of age with pneumonia ![Figure 3:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/08/20/2024.08.19.24312238/F3.medium.gif) [Figure 3:](http://medrxiv.org/content/early/2024/08/20/2024.08.19.24312238/F3) Figure 3: Calibration plot of the PREPARE hypoxemia clinical prediction model in the validation dataset Reference (dashed line) indicates perfect calibration; Lowess (blue line) indicates calibration slope; Red rug plots indicate the distribution of predicted risk of participants with hypoxemia (above the line) and those without hypoxemia (below the line), E:O = expected to observed events ratio; CITL = calibration-in-the-large; AUC = area under the receiver operating characteristic curve. When converted into the PREPARE hypoxemia risk score (table 4), the retained predictors included age 2-11 months (+1 point), lower chest indrawing (+1 points), respiratory rate >/=20 breaths/min above age-adjusted tachypnea threshold (+2 points), any sign of respiratory distress (+2 points) and weight for age z-score <-2 (+1 point), with the score ranging from 0-7 points (table 4 and 5). Sensitivity, specificity, positive and negative likelihood ratios, miss rate, false discovery rate and the proportion of children identified as hypoxemic are included for each score cut-off (table 5). View this table: [Table 4:](http://medrxiv.org/content/early/2024/08/20/2024.08.19.24312238/T4) Table 4: Components of the PREPARE hypoxemia risk score in the development dataset (n=10,884) View this table: [Table 5.](http://medrxiv.org/content/early/2024/08/20/2024.08.19.24312238/T5) Table 5. Performance of the PREPARE hypoxemia risk score at each cut-off in the validation dataset (n=3,625) In the absence of pulse oximetry, the decision curve analysis suggests that in contexts where it might be acceptable to identify (and refer) up to six non-hypoxemic children for each hypoxemic child (up to a threshold probability of ∼17%) it would be optimal to use a cut-off of >/= 5 to guide referral decision for further management, whereas in contexts in which greater than 1-in-7 true positives are required, a cut-off of >/= six would be preferable (figure S1). The PREPARE hypoxemia model had an equivalent or higher net benefit than the PREPARE hypoxemia risk score (at any cut-off) across all threshold probabilities. ### Prediction of hypoxemia at SpO2 threshold of < 92% In our sensitivity analysis using the same dataset, 15.7% (2,281/14,509) children had SpO2 <92%. The proportion with SpO2 <92% was slightly lower in the validation dataset (14.2%, 514/3,625) compared to the development dataset (16.2%, 1,767/10,884). The predictive value of predictors in predicting SpO2 <92% compared to SpO2 <90% were similar (table S1, table 3). The PREPARE SpO2 <92% clinical prediction model had a similar AUROC (0.67, 95% CI 0.65 to 0.70) compared to the PREPARE hypoxemia (SpO2 <90%) prediction model (AUROC 0.70, 95% CI 0.68 to 0.73) (figure S2). The calibration plot for this model is shown in figure S3 and the performance of the risk score at each cut-off is presented in table S3. The PREPARE SpO2 <92% risk score integrates 5 clinical predictors for a total score of 6 points: Age 6-11 months (+1 point), lower chest indrawing (+1 point), respiratory rate >/= 20 breaths/min above age-adjusted tachypnea threshold (+2 points), any sign of respiratory distress (+1 point) and weight-for-age z-score <-3 (+1 point) (table S2). ### PREPARE hypoxemia risk score without respiratory distress When excluding respiratory distress as a candidate predictor, the dataset was larger with 15,592 children from 14 studies. Five studies were prospective cohorts, 7 were randomized controlled trials, one was a retrospective cohort and one was a prospective case series (table S4). Among the 14 studies included in this analysis, patients were recruited from 23 countries from North, Central and South America, Africa, Asia and Oceania (table S4). Of the 15,592 patients with pneumonia, 16.1% had a SpO2 <90% at presentation (table S5). Baseline characteristics were similar in the development and validation datasets (table S5) and did not differ from the primary dataset including respiratory distress. Without respiratory distress, lower chest indrawing (aOR 4.44, 95% CI 3.83 to 5.15) contributed more towards the prediction of hypoxemia (table S6). None of the 11 candidate parameters were eliminated by the LASSO penalty during model development. The PREPARE hypoxemia clinical prediction model excluding respiratory distress had a higher AUROC than the model that included respiratory distress as a candidate predictor (AUROC 0.78 vs. 0.70) (figure S4). The calibration plot of this model is shown in figure S5. The final PREPARE hypoxemia risk score when excluding respiratory distress integrated 4 clinical predictors: age 2-11 months (+1 point), lower chest indrawing (+3 points), respiratory rate >=20 breaths/min above age-adjusted tachypnea cut-off (+1 point) and weight for age z-score <-2 (+1 point) for a maximum total score of 6 points (table S7). ### PREPARE hypoxemia risk score with wheezing Using the same methodology as the primary analysis (i.e., with respiratory distress), including wheezing as a candidate predictor, resulted in a slightly smaller dataset of 13,792 children from 7 studies (table S9). In this dataset, 9.9% had a SpO2 of less than 90% at presentation (table S10). Baseline characteristics in these development and validation datasets were similar (table S10) and did not differ from the primary dataset, which excluded wheezing. Wheezing had an adjusted odds ratio of 1.28 (95% CI 1.11 to 1.47), sensitivity of 37.6% and specificity of 73.2% (table S11) for identification of hypoxemia. None of the 12 candidate parameters were eliminated by the LASSO penalty during model development. The PREPARE hypoxemia clinical prediction model that included wheezing performed similarly to the model without wheezing in the validation datasets (AUROC 0.72 versus 0.70) (figure S6). The calibration plot of the PREPARE hypoxemia clinical prediction model, including wheezing is shown in figure S3. The performance of the PREPARE hypoxemia risk score including wheezing at each cut-off is presented in table S10. The final PREPARE hypoxemia risk score when including wheezing integrated five clinical parameters: age 2-11 months (+1 points), chest indrawing (+1 point), respiratory distress (+2 point), respiratory rate >=20 breaths/min above cut-off (+2 points) and weight for age z-score <-2 (+1 point) for a maximum total score of 7 points (table S12). Wheezing was not retained in the risk score as the adjusted log coefficient was 0.24. A comparison of all risk score weights from the main analysis and sensitivity analyses can be found in table S13. ## DISCUSSION We developed the PREPARE hypoxemia risk score from one of the largest and most diverse pneumonia datasets, including over 14,500 childhood pneumonia cases (of which 1,421 had hypoxemia) from 7 studies in 5 countries and in 14 studies in 23 countries within a sensitivity analysis. The components of the score include age, respiratory distress, respiratory rate, lower chest-indrawing and weight-for-age z-score. The risk score demonstrated fair discrimination in predicting hypoxemia (SpO2 <90%) in children with pneumonia without danger signs or other indications for referral. Previous studies predicting hypoxemia included fewer patients from more restricted geographical settings.8,22-26,48-51 The use of very large and diverse datasets reduces instability of individual predictions, limits overfitting and provides the opportunity for generalizability to a wide range of settings.52,53 A model developed from a dataset with a similar number of patients, though exclusively evaluated in outpatient clinics from Bangladesh and Malawi, had a higher AUROC (0.79) compared to the PREPARE hypoxemia risk score (0.70), but had a similar AUROC to the PREPARE hypoxemia risk score excluding respiratory distress (0.78).22 However, this study and many others included patients with IMCI danger signs in whom hospitalization or referral would already be recommended by existing guidelines.21,22,29 IMCI danger signs were retained in these prior analyses to evaluate their performance in hypoxemia prediction models alongside other candidate parameters and these studies showed IMCI danger signs added little to no predictive value for hypoxemia.21 While excluding such patients in our study limits our ability to comment on the value of IMCI danger signs in hypoxemia prediction models it aligns more closely with the needs of healthcare providers currently using the IMCI framework, helping them to decide which patients with pneumonia not already identified for hospitalization or referral may benefit from higher-level care. As found in other studies, 21 this approach also underscores the high number of pneumonia patients with hypoxemia (1,421/14,509; 9.8%) that would not have been identified as requiring referral or hospitalization using the current IMCI algorithm in the absence of a pulse oximeter. In line with previous studies, greater tachypnea,23,24,49,51 chest indrawing,22,23,51 younger age,8,22 wheezing,22 respiratory distress,8,22,24,49,51 and low and very-low weight-for-age z-score,8,22 were identified as clinical signs predictive of hypoxemia in children with pneumonia. In the present study and previous studies, individual signs performed poorly in identifying hypoxemia, emphasizing the benefits of a multivariable clinical risk score. The use of the PREPARE hypoxemia risk score and selection of cut-off to guide referral of children with suspected hypoxemia would need to be tailored to the setting of implementation. Selecting a cut-off with the highest balance between sensitivity and specificity may not be the most appropriate referral threshold in many clinical contexts.54,55 Care must be taken when deciding to use such a score in order to limit referral of children without hypoxemia or danger signs which could potentially come at great cost to individual patients and/or the health system. Considering that all patients with hypoxemia identified using this score would have otherwise been misidentified as not requiring referral by IMCI without pulse oximetry, using cut-offs with lower sensitivity (and higher specificity) could still make a helpful contribution, limiting unnecessary referrals, whilst still identifying many children with hypoxemia who would have been missed by the IMCI algorithm in settings where pulse oximetry is not available. Nevertheless, even utilising cut-offs of 5 or 6, where between 4-12% of all children would be referred, the majority of referrals (74-81%) would still be for children who were not hypoxemic. It may be that depending on the setting of implementation, a cut-off with relatively low specificity should consider alternative approaches to referral, such as short admission or follow-up within the community, to observe clinical evolution as proposed by Graham et al.13 Growing accessibility to “smartphones” might increase feasibility of a digital tool, which would allow more precise predictions and enable referral thresholds to be better tailored to the specific context of implementation. More pragmatically, elements from the PREPARE hypoxemia risk score could be combined with data on their prognostic value to predict death,56 to update the IMCI criteria for severe pneumonia. The sensitivity analysis excluding respiratory distress had a higher AUROC value compared to the primary analysis including respiratory distress as a candidate predictor. This however, should not be interpreted to mean that respiratory distress does not add value in identifying hypoxemia. In fact respiratory distress had similar predictive value to identifying hypoxemia compared to lower chest indrawing and very fast breathing for age (i.e., a respiratory rate of 20 breaths or more above age cut-off values). Differences in AUROC values between our analyses and compared to other studies are influenced by differences in the underlying datasets.52 Indeed, this sensitivity analysis included twice the number of studies and thus the higher discrimination may reflect better generalizability of a score developed from a more diverse range of countries and settings. The value of respiratory distress should thus still be explored to improve the detection of children with hypoxemia and/or severe pneumonia within IMCI consultations, all the while considering the training, mentorship and resources required to reliably assess these signs. There are several limitations to this study. First, we intended to derive a clinical risk score using data from primary level health facilities since this is where pulse oximetry is often inaccessible and thus the clinical risk score would be most useful. Unfortunately, due to missing data on the outcome of interest and important clinical predictors, almost all studies from primary level health facilities were excluded, leaving a cohort derived predominantly from patients presenting to hospital outpatient departments. While the exclusion of children with danger signs may reduce this spectrum bias, the high proportion of pneumonia patients with chest indrawing (62%) and hypoxemia (9.8%) compared to what is expected at primary level health facilities, demonstrates the more severe presentation of included patients. Nonetheless, a similar study that was limited to ambulatory clinics found that the sensitivity and specificity of individual variables for predicting hypoxemia were similar to hospital-based studies.22 Secondly, it has been reported that pulse oximetry is less accurate in patients with darker skin,57,58 and this was not considered in the present clinical risk score. Third, only 9% of children within the target age range with pneumonia were included (14,509/157,433). The vast majority of children and studies were excluded due to missing data (predictors or outcome). While this limited the diversity of the dataset, this decision was taken in order to prioritize better quality data, limiting bias in order to retain studies that systematically assessed the predictors and outcome of interest. Further work is needed to confirm that the score is generalizable to the whole pneumonia population more widely – assuming that patients with pulse oximetry readings are likely to be more severe, the prevalence of hypoxemia in the pneumonia population is likely lower,59 and the score may therefore benefit from updating. ## CONCLUSION Hypoxemia predicts mortality at all levels of care,2-4 and pulse oximetry helps increase sensitivity to identify children with severe pneumonia as defined by IMCI.6,19 Identification of children with pneumonia who are at high risk of hypoxemia using the PREPARE hypoxemia risk score could improve recognition of children who would benefit from further management and may reduce morbidity and mortality in lower-level health facilities without access to pulse oximetry, even when other well-recognized indications of referral are absent. These benefits must be weighed against the burden of increased referrals on patients and health systems. Further validation and model updating in community settings is both a research and policy priority. ## Supporting information Supplementary Materials [[supplements/312238_file02.pdf]](pending:yes) ## Data Availability Data may be made available upon reasonable request to the corresponding author. ## FUNDING The study was funded by the Bill & Melinda Gates Foundation (#INV-007927) through a grant to the World Health Organization. The funders had no role in the study design or in the collection, analysis, or interpretation of the data. The funders did not write the report and had no role in the decision to submit the paper for publication. ## AUTHOR CONTRIBUTIONS RT, AC, FB, GL and YBN conceptualized and designed the study, interpreted the data and wrote the first draft of the manuscript. YBN verified the underlying data and conducted the statistical analyses. All authors oversaw data collection, verified the underlying data, assisted with the interpretation of the data and reviewed and provided input to the final draft. YBN had final responsibility for the decision to submit for publication. ## COMPETING INTERESTS We declare no competing interests. YBN is staff member of the World Health Organization. The authors alone are responsible for the views expressed in this article and they do not necessarily represent the views, decisions or policies of the institutions with which they are affiliated. ## ACKNOWLEDGEMENTS We would like to thank all study participants, clinicians and researchers involved in the included studies. * Received August 19, 2024. * Revision received August 19, 2024. * Accepted August 20, 2024. * © 2024, Posted by Cold Spring Harbor Laboratory This pre-print is available under a Creative Commons License (Attribution 4.0 International), CC BY 4.0, as described at [http://creativecommons.org/licenses/by/4.0/](http://creativecommons.org/licenses/by/4.0/) ## REFERENCES 1. 1.Rahman AE, Hossain AT, Nair H, et al. Prevalence of hypoxaemia in children with pneumonia in low-income and middle-income countries: a systematic review and meta-analysis. The Lancet Global health 2022;10(3):e348–e59. doi: 10.1016/s2214-109x(21)00586-6 [published Online First: 2022/02/19] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/s2214-109x(21)00586-6&link_type=DOI) 2. 2.Colbourn T, King C, Beard J, et al. Predictive value of pulse oximetry for mortality in infants and children presenting to primary care with clinical pneumonia in rural Malawi: A data linkage study. PLoS medicine 2020;17(10):e1003300. doi: 10.1371/journal.pmed.1003300 [published Online First: 2020/10/24] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pmed.1003300&link_type=DOI) 3. 3.Chandna A, Tan R, Carter M, et al. Predictors of disease severity in children presenting from the community with febrile illnesses: a systematic review of prognostic studies. BMJ Glob Health 2021;6(1) doi: 10.1136/bmjgh-2020-003451 [published Online First: 2021/01/22] [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiYm1qZ2giO3M6NToicmVzaWQiO3M6MTE6IjYvMS9lMDAzNDUxIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjQvMDgvMjAvMjAyNC4wOC4xOS4yNDMxMjIzOC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 4. 4.Lazzerini M, Sonego M, Pellegrin MC. Hypoxaemia as a Mortality Risk Factor in Acute Lower Respiratory Infections in Children in Low and Middle-Income Countries: Systematic Review and Meta-Analysis. PLoS One 2015;10(9):e0136166. doi: 10.1371/journal.pone.0136166 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pone.0136166&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=26372640&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F08%2F20%2F2024.08.19.24312238.atom) 5. 5.Enoch AJ, English M, Shepperd S. Does pulse oximeter use impact health outcomes? A systematic review. Archives of disease in childhood 2016;101(8):694–700. doi: 10.1136/archdischild-2015-309638 [published Online First: 2015/12/25] [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTI6ImFyY2hkaXNjaGlsZCI7czo1OiJyZXNpZCI7czo5OiIxMDEvOC82OTQiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyNC8wOC8yMC8yMDI0LjA4LjE5LjI0MzEyMjM4LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 6. 6.Hooli S, King C, McCollum ED, et al. In-hospital mortality risk stratification in children aged under 5 years with pneumonia with or without pulse oximetry: A secondary analysis of the Pneumonia REsearch Partnership to Assess WHO REcommendations (PREPARE) dataset. Int J Infect Dis 2023;129:240–50. doi: 10.1016/j.ijid.2023.02.005 [published Online First: 2023/02/23] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.ijid.2023.02.005&link_type=DOI) 7. 7.Duke T, Graham S, Cherian M, et al. Oxygen is an essential medicine: a call for international action [Unresolved issues]. The International journal of tuberculosis and lung disease 2010;14(11):1362–68. 8. 8.Awasthi S, Rastogi T, Pandey AK, et al. Epidemiology of Hypoxic Community-Acquired Pneumonia in Children Under 5 Years of Age: An Observational Study in Northern India. Frontiers in pediatrics 2021;9:790109. doi: 10.3389/fped.2021.790109 [published Online First: 2022/03/01] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3389/fped.2021.790109&link_type=DOI) 9. 9.Sylvies F, Nyirenda L, Blair A, et al. The impact of pulse oximetry and Integrated Management of Childhood Illness (IMCI) training on antibiotic prescribing practices in rural Malawi: A mixed-methods study. PLoS One 2020;15(11):e0242440. doi: 10.1371/journal.pone.0242440 [published Online First: 2020/11/20] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pone.0242440&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F08%2F20%2F2024.08.19.24312238.atom) 10. 10.UNITAID. The ACT-A Oxygen Emergency Taskforce… one year on 2022 [updated 23.02.2022. Available from: [https://unitaid.org/news-blog/the-act-a-oxygen-emergency-taskforceone-year-on/#en](https://unitaid.org/news-blog/the-act-a-oxygen-emergency-taskforceone-year-on/#en) accessed 09.11.2022 2022. 11. 11.Källander K, Young M, Qazi S. Universal access to pneumonia prevention and care: a call for action. The Lancet Respiratory Medicine 2014;2(12):950–52. 12. 12.Ansermino JM. Universal access to essential vital signs monitoring. Anesthesia & Analgesia 2013;117(4):883–90. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1213/ANE.0b013e3182a1f22f&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24023026&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F08%2F20%2F2024.08.19.24312238.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000330435000017&link_type=ISI) 13. 13.Graham HR, King C, Duke T, et al. Hypoxaemia and risk of death among children: rethinking oxygen saturation, risk-stratification, and the role of pulse oximetry in primary care. The Lancet Global Health 2024;12(8):e1359–e64. doi: 10.1016/S2214-109X(24)00209-2 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S2214-109X(24)00209-2&link_type=DOI) 14. 14.Ginsburg AS, Van Cleve WC, Thompson MIW, et al. Oxygen and Pulse Oximetry in Childhood Pneumonia: A Survey of Healthcare Providers in Resource-limited Settings. Journal of tropical pediatrics 2012;58(5):389–93. doi: 10.1093/tropej/fmr103 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/tropej/fmr103&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=22170511&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F08%2F20%2F2024.08.19.24312238.atom) 15. 15.McCollum ED, Bjornstad E, Preidis GA, et al. Multicenter study of hypoxemia prevalence and quality of oxygen treatment for hospitalized Malawian children. Transactions of the Royal Society of Tropical Medicine and Hygiene 2013;107(5):285–92. doi: 10.1093/trstmh/trt017 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/trstmh/trt017&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=23584373&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F08%2F20%2F2024.08.19.24312238.atom) 16. 16.English M, Gathara D, Mwinga S, et al. Adoption of recommended practices and basic technologies in a low-income setting. Archives of disease in childhood 2014;99(5):452. doi: 10.1136/archdischild-2013-305561 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTI6ImFyY2hkaXNjaGlsZCI7czo1OiJyZXNpZCI7czo4OiI5OS81LzQ1MiI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDI0LzA4LzIwLzIwMjQuMDguMTkuMjQzMTIyMzguYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 17. 17.Simkovich SM, Underhill LJ, Kirby MA, et al. Resources and Geographic Access to Care for Severe Pediatric Pneumonia in Four Resource-limited Settings. Am J Respir Crit Care Med 2022;205(2):183–97. doi: 10.1164/rccm.202104-1013OC [published Online First: 2021/10/19] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1164/rccm.202104-1013OC&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=34662531&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F08%2F20%2F2024.08.19.24312238.atom) 18. 18.Enoch AJ, English M, McGivern G, et al. Variability in the use of pulse oximeters with children in Kenyan hospitals: A mixed-methods analysis. PLoS medicine 2019;16(12) doi: 10.1371/journal.pmed.1002987; 10.1371/journal.pmed.1002987.r001; 10.1371/journal.pmed.1002987.r002; 10.1371/journal.pmed.1002987.r003; 10.1371/journal.pmed.1002987.r004; 10.1371/journal.pmed.1002987.r005 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pmed.1002987&link_type=DOI) 19. 19.McCollum ED, King C, Deula R, et al. Pulse oximetry for children with pneumonia treated as outpatients in rural Malawi. Bulletin of the World Health Organization 2016;94(12):893–902. doi: 10.2471/blt.16.173401 [published Online First: 2016/12/21] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2471/blt.16.173401&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F08%2F20%2F2024.08.19.24312238.atom) 20. 20.King C, Boyd N, Walker I, et al. Opportunities and barriers in paediatric pulse oximetry for pneumonia in low-resource clinical settings: a qualitative evaluation from Malawi and Bangladesh. BMJ open 2018;8(1):e019177. doi: 10.1136/bmjopen-2017-019177 [published Online First: 2018/02/01] [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMToiOC8xL2UwMTkxNzciO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyNC8wOC8yMC8yMDI0LjA4LjE5LjI0MzEyMjM4LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 21. 21.World Health Organization. Integrated Management of Childhood Ilness Chartbook. Geneva, 2014. 22. 22.Schuh HB, Hooli S, Ahmed S, et al. Clinical hypoxemia score for outpatient child pneumonia care lacking pulse oximetry in Africa and South Asia. Frontiers in pediatrics 2023;11 doi: 10.3389/fped.2023.1233532 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3389/fped.2023.1233532&link_type=DOI) 23. 23.Lodha R, Bhadauria PS, Kuttikat AV, et al. Can clinical symptoms or signs accurately predict hypoxemia in children with acute lower respiratory tract infections? Indian pediatrics 2004;41(2):129–35. [published Online First: 2004/03/09] [PubMed](http://medrxiv.org/lookup/external-ref?access_num=15004298&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F08%2F20%2F2024.08.19.24312238.atom) 24. 24.Usen S, Weber M, Mulholland K, et al. Clinical predictors of hypoxaemia in Gambian children with acute lower respiratory tract infection: prospective cohort study. Bmj 1999;318(7176):86–91. doi: 10.1136/bmj.318.7176.86 [published Online First: 1999/01/08] [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjExOiIzMTgvNzE3Ni84NiI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDI0LzA4LzIwLzIwMjQuMDguMTkuMjQzMTIyMzguYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 25. 25.von der Weid L, Gehri M, Camara B, et al. Clinical signs of hypoxaemia in children aged 2 months to 5 years with acute respiratory distress in Switzerland and Senegal. Paediatrics and International Child Health 2018;38(2):113–20. doi: 10.1080/20469047.2017.1390828 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/20469047.2017.1390828&link_type=DOI) 26. 26.Zhang L, Mendoza-Sassi R, Santos JC, et al. Accuracy of symptoms and signs in predicting hypoxaemia among young children with acute respiratory infection: a meta-analysis. Int J Tuberc Lung Dis 2011;15(3):317–25. [published Online First: 2011/02/22] [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21333097&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F08%2F20%2F2024.08.19.24312238.atom) 27. 27.Martin H, Falconer J, Addo-Yobo E, et al. Assembling a global database of child pneumonia studies to inform WHO pneumonia management algorithm: Methodology and applications. J Glob Health 2022;12:04075. doi: 10.7189/jogh.12.04075 [published Online First: 2022/12/30] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7189/jogh.12.04075&link_type=DOI) 28. 28.Moons KGM, Altman DG, Reitsma JB, et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): Explanation and Elaboration. Annals of internal medicine 2015;162(1):W1–W73. doi: 10.7326/M14-0698 %J Annals of Internal Medicine [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7326/M14-0698&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25560730&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F08%2F20%2F2024.08.19.24312238.atom) 29. 29.World Health Organization. Pocket Book of Hospital Care for Children: Guidelines for the Management of Common Childhood Illnesses. 2nd ed. Geneva: World Health Organization 2013. 30. 30.Fields E, Chard J, Murphy MS, et al. Assessment and initial management of feverish illness in children younger than 5 years: summary of updated NICE guidance. Bmj 2013;346:f2866. doi: 10.1136/bmj.f2866 [published Online First: 2013/05/24] [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE4OiIzNDYvbWF5MjJfMTYvZjI4NjYiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyNC8wOC8yMC8yMDI0LjA4LjE5LjI0MzEyMjM4LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 31. 31.Harris M, Clark J, Coote N, et al. British Thoracic Society guidelines for the management of community acquired pneumonia in children: update 2011. Thorax 2011;66 Suppl 2:ii1–23. doi: 10.1136/thoraxjnl-2011-200598 [published Online First: 2011/10/19] [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToidGhvcmF4am5sIjtzOjU6InJlc2lkIjtzOjE0OiI2Ni9TdXBwbF8yL2lpMSI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDI0LzA4LzIwLzIwMjQuMDguMTkuMjQzMTIyMzguYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 32. 32.Fung JST, Akech S, Kissoon N, et al. Determining predictors of sepsis at triage among children under 5 years of age in resource-limited settings: A modified Delphi process. PLoS One 2019;14(1):e0211274. doi: 10.1371/journal.pone.0211274 [published Online First: 2019/01/29] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pone.0211274&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F08%2F20%2F2024.08.19.24312238.atom) 33. 33.Riley RD, Snell KI, Ensor J, et al. Minimum sample size for developing a multivariable prediction model: PART II - binary and time-to-event outcomes. Stat Med 2019;38(7):1276–96. doi: 10.1002/sim.7992 [published Online First: 2018/10/26] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/sim.7992&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=30357870&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F08%2F20%2F2024.08.19.24312238.atom) 34. 34.Collins GS, Ogundimu EO, Altman DG. Sample size considerations for the external validation of a multivariable prognostic model: a resampling study. Statistics in medicine 2016;35(2):214–26. doi: 10.1002/sim.6787 [published Online First: 2015/11/09] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/sim.6787&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=26553135&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F08%2F20%2F2024.08.19.24312238.atom) 35. 35.Vergouwe Y, Steyerberg EW, Eijkemans MJ, et al. Substantial effective sample sizes were required for external validation studies of predictive logistic regression models. J Clin Epidemiol 2005;58(5):475–83. doi: 10.1016/j.jclinepi.2004.06.017 [published Online First: 2005/04/23] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jclinepi.2004.06.017&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=15845334&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F08%2F20%2F2024.08.19.24312238.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000229008000007&link_type=ISI) 36. 36.World Health Organization. Oxygen therapy for children: a manual for health workers. 2016 37. 37.Jakobsen JC, Gluud C, Wetterslev J, et al. When and how should multiple imputation be used for handling missing data in randomised clinical trials – a practical guide with flowcharts. BMC Medical Research Methodology 2017;17(1):162. doi: 10.1186/s12874-017-0442-1 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s12874-017-0442-1&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F08%2F20%2F2024.08.19.24312238.atom) 38. 38.Bonnett LJ, Snell KIE, Collins GS, et al. Guide to presenting clinical prediction models for use in clinical settings. BMJ 2019;365:l737. doi: 10.1136/bmj.l737 [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE2OiIzNjUvYXByMTdfMi9sNzM3IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjQvMDgvMjAvMjAyNC4wOC4xOS4yNDMxMjIzOC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 39. 39.Sullivan LM, Massaro JM, D’Agostino Sr RB. Presentation of multivariate data for clinical use: The Framingham Study risk score functions. Statistics in medicine 2004;23(10):1631–60. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/sim.1742&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=15122742&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F08%2F20%2F2024.08.19.24312238.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000221233900017&link_type=ISI) 40. 40.Stata Statistical Software: Release 16. [program]. College Station, TX: StataCorp LLC., 2019. 41. 41.Cutts F, Zaman S, Enwere Gy, et al. Efficacy of nine-valent pneumococcal conjugate vaccine against pneumonia and invasive pneumococcal disease in The Gambia: randomised, double-blind, placebo-controlled trial. The Lancet 2005;365(9465):1139–46. 42. 42.Basnet S, Shrestha PS, Sharma A, et al. A randomized controlled trial of zinc as adjuvant therapy for severe pneumonia in young children. Pediatrics 2012;129(4):701–08. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1542/peds.2010-3091&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=22392179&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F08%2F20%2F2024.08.19.24312238.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000302541700049&link_type=ISI) 43. 43.Lazzerini M, Seward N, Lufesi N, et al. Mortality and its risk factors in Malawian children admitted to hospital with clinical pneumonia, 2001–12: a retrospective observational study. The Lancet Global health 2016;4(1):e57–68. doi: 10.1016/s2214-109x(15)00215-6 [published Online First: 2016/01/01] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/s2214-109x(15)00215-6&link_type=DOI) 44. 44.Mathew JL, Singhi S, Ray P, et al. Etiology of community acquired pneumonia among children in India: prospective, cohort study. Journal of global health 2015;5(2) 45. 45.McCollum ED, Nambiar B, Deula R, et al. Impact of the 13-valent pneumococcal conjugate vaccine on clinical and hypoxemic childhood pneumonia over three years in central Malawi: an observational study. PLoS One 2017;12(1):e0168209. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F08%2F20%2F2024.08.19.24312238.atom) 46. 46.O’Grady KAF, Torzillo PJ, Ruben AR, et al. Identification of radiological alveolar pneumonia in children with high rates of hospitalized respiratory infections: Comparison of WHO-defined and pediatric pulmonologist diagnosis in the clinical context. Pediatric pulmonology 2012;47(4):386–92. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21916025&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F08%2F20%2F2024.08.19.24312238.atom) 47. 47.Wadhwa N, Chandran A, Aneja S, et al. Efficacy of zinc given as an adjunct in the treatment of severe and very severe pneumonia in hospitalized children 2–24 mo of age: a randomized, double-blind, placebo-controlled trial. The American journal of clinical nutrition 2013;97(6):1387–94. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYWpjbiI7czo1OiJyZXNpZCI7czo5OiI5Ny82LzEzODciO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyNC8wOC8yMC8yMDI0LjA4LjE5LjI0MzEyMjM4LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 48. 48.Rao YK, Midha T, Kumar P, et al. Clinical predictors of hypoxemia in Indian children with acute respiratory tract infection presenting to pediatric emergency department. World J Pediatr 2012;8(3):247–51. doi: 10.1007/s12519-012-0365-1 [published Online First: 2012/08/14] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s12519-012-0365-1&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=22886198&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F08%2F20%2F2024.08.19.24312238.atom) 49. 49.Alwadhi V, Dewan P, Malhotra RK, et al. Tachypnea and other danger signs vs pulse oximetry for prediction of hypoxia in severe pneumonia/very severe disease. Indian pediatrics 2017;54(9):729–34. doi: 10.1007/s13312-017-1163-6 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s13312-017-1163-6&link_type=DOI) 50. 50.Tesfaye SH, Gebeyehu Y, Loha E, et al. Pulse oximeter with integrated management of childhood illness for diagnosis of severe childhood pneumonia at rural health institutions in Southern Ethiopia: results from a cluster-randomised controlled trial. BMJ open 2020;10(6):e036814. doi: 10.1136/bmjopen-2020-036814 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMjoiMTAvNi9lMDM2ODE0IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjQvMDgvMjAvMjAyNC4wOC4xOS4yNDMxMjIzOC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 51. 51.Onyango FE, Steinhoff MC, Wafula EM, et al. Hypoxaemia in young Kenyan children with acute lower respiratory infection. British Medical Journal 1993;306(6878):612. doi: 10.1136/bmj.306.6878.612 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMDYvNjg3OC82MTIiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyNC8wOC8yMC8yMDI0LjA4LjE5LjI0MzEyMjM4LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 52. 52.Riley RD, Pate A, Dhiman P, et al. Clinical prediction models and the multiverse of madness. BMC Medicine 2023;21(1):502. doi: 10.1186/s12916-023-03212-y [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s12916-023-03212-y&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=38110939&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F08%2F20%2F2024.08.19.24312238.atom) 53. 53.Collins GS, Dhiman P, Ma J, et al. Evaluation of clinical prediction models (part 1): from development to external validation. BMJ 2024;384:e074819. doi: 10.1136/bmj-2023-074819 [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE5OiIzODQvamFuMDhfMi9lMDc0ODE5IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjQvMDgvMjAvMjAyNC4wOC4xOS4yNDMxMjIzOC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 54. 54.Fackler JC, Rehman M, Winslow RL. Please Welcome the New Team Member: The Algorithm. Pediatric critical care medicine : a journal of the Society of Critical Care Medicine and the World Federation of Pediatric Intensive and Critical Care Societies 2019;20(12):1200–01. doi: 10.1097/pcc.0000000000002149 [published Online First: 2019/12/06] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/pcc.0000000000002149&link_type=DOI) 55. 55.Tustumi F. Choosing the most appropriate cut-point for continuous variables. Revista do Colégio Brasileiro de Cirurgiões 2022;49:e20223346. 56. 56.Rees CA, Colbourn T, Hooli S, et al. Derivation and validation of a novel risk assessment tool to identify children aged 2–59 months at risk of hospitalised pneumonia-related mortality in 20 countries. BMJ Glob Health 2022;7(4) doi: 10.1136/bmjgh-2021-008143 [published Online First: 2022/04/17] [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiYm1qZ2giO3M6NToicmVzaWQiO3M6MTE6IjcvNC9lMDA4MTQzIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjQvMDgvMjAvMjAyNC4wOC4xOS4yNDMxMjIzOC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 57. 57.Feiner JR, Severinghaus JW, Bickler PE. Dark skin decreases the accuracy of pulse oximeters at low oxygen saturation: the effects of oximeter probe type and gender. Anesth Analg 2007;105(6 Suppl):S18–s23. doi: 10.1213/01.ane.0000285988.35174.d9 [published Online First: 2007/12/06] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1213/01.ane.0000285988.35174.d9&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=18048893&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F08%2F20%2F2024.08.19.24312238.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000205336800004&link_type=ISI) 58. 58.Sjoding MW, Dickson RP, Iwashyna TJ, et al. Racial Bias in Pulse Oximetry Measurement. The New England journal of medicine 2020;383(25):2477–78. doi: 10.1056/NEJMc2029240 [published Online First: 2020/12/17] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMc2029240&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33326721&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F08%2F20%2F2024.08.19.24312238.atom) 59. 59.EMPIC Study Group. Innovative, enhanced community management of non-hypoxaemic chest-indrawing pneumonia in 2–59-month-old children: a cluster-randomised trial in Africa and Asia. BMJ Global Health 2022;7(1):e006405. doi: 10.1136/bmjgh-2021-006405 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiYm1qZ2giO3M6NToicmVzaWQiO3M6MTE6IjcvMS9lMDA2NDA1IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjQvMDgvMjAvMjAyNC4wOC4xOS4yNDMxMjIzOC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=)