Accuracy and clinical effectiveness of risk prediction tools for pressure injury occurrence: An umbrella review
===============================================================================================================
* Bethany Hillier
* Katie Scandrett
* April Coombe
* Tina Hernandez-Boussard
* Ewout Steyerberg
* Yemisi Takwoingi
* Vladica Velickovic
* Jacqueline Dinnes
## ABSTRACT
**Background** Pressure injuries (PIs) pose a substantial healthcare burden and incur significant costs worldwide. Several risk prediction tools to allow timely implementation of preventive measures and a subsequent reduction in healthcare system burden are available and in use. The ability of risk prediction tools to correctly identify those at high risk of PI (prognostic accuracy) and to have a clinically significant impact on patient management and outcomes (effectiveness) is not clear.
We aimed to evaluate the prognostic accuracy and clinical effectiveness of risk prediction tools for PI, and to identify gaps in the literature.
**Methods and Findings** The umbrella review was conducted according to Cochrane guidance. MEDLINE, Embase, CINAHL, EPISTEMONIKOS, Google Scholar and reference lists were searched to identify relevant systematic reviews. Methodological quality was assessed using adapted AMSTAR-2 criteria. Results were described narratively.
We identified 19 reviews that assessed prognostic accuracy and 11 that assessed clinical effectiveness of risk prediction tools for PI. The 19 reviews of prognostic accuracy evaluated 70 tools (39 scales and 31 machine learning models), with the Braden, Norton, Waterlow, Cubbin-Jackson scales (and modifications thereof) the most evaluated tools. Meta-analyses from a focused set of included reviews showed that the scales had sensitivities and specificities ranging from 53%-97% and 46%-84%, respectively. Only 2/19 reviews performed appropriate statistical synthesis and quality assessment. Two reviews assessing machine learning based algorithms reported high prognostic accuracy estimates, but some of which were sourced from the same data within which the models were developed, leading to potentially overoptimistic results.
Two randomised trials assessing the effect of PI risk assessment tools (within the full test-intervention-outcome pathway) on the incidence of PIs were identified from the 11 systematic reviews of clinical effectiveness; both were included in a Cochrane review and assessed as high risk of bias. Both trials found no evidence of an effect on PI incidence.
**Conclusions** Available systematic reviews suggest a lack of high-quality evidence for the accuracy of risk prediction tools for PI and limited reliable evidence for their use leading to a reduction in incidence of PI. Further research is needed to establish the clinical effectiveness of appropriately developed and validated risk prediction tools for PI.
**Why was this study done?**
* Pressure injuries (PIs) are injuries to and below the skin caused by prolonged pressure, especially on bony areas, and people who spend extensive periods in a bed or chair are particularly vulnerable.
* The majority of pressure injuries are preventable if appropriate preventive measures are put into place, but it is crucial to conduct risk stratification of individuals in order to appropriately allocate preventive measures.
* Numerous tools that give patients a score (or probability) to signify their risk of developing a PI exist. However, there is a lack of clarity on how accurate the risk scores are, and how effective the scores are at improving patient outcomes (the clinical effectiveness) when patient management is subsequently changed for patients classified as high-risk.
**What did the researchers do and find?**
* We conducted an umbrella review (an overview of existing systematic reviews), identifying 26 systematic reviews which included 70 risk prediction tools.
* Of these 70 risk prediction tools, 31 were developed using machine learning methods, while the remainder were derived from statistical modelling and/or clinical expertise.
* Risk prediction tools demonstrated moderate to high accuracy, as measured by a variety of metrics. However, there were concerns regarding the quality of both the systematic reviews, and the primary studies included in these reviews, as reported by the systematic review authors.
* There were only two randomised controlled trials that investigated the clinical effectiveness of risk prediction tools and subsequent changes in PI management, and neither trial found that use of the tools had an impact on the incidence of PIs.
**What do these findings mean?**
* Whilst an abundance of risk prediction tools exists, it is unclear how accurate they are due to poor quality evidence and poor reporting, so it is difficult to recommend a particular tool/tools.
* Even if the tools are shown to be accurate, they are not useful unless they lead to improvement in patient outcomes. There is very limited evidence to determine whether the tools are clinically effective and the evidence that does exist suggests that the tools did not lead to improved patient outcomes.
* More research into the clinical effectiveness of appropriately developed and evaluated tools, when they are adopted within the clinical pathway, is needed.
Keywords
* Sensitivity
* specificity
* AUC
* AUROC
* prognostic model
* clinical scale
* pressure injury
* pressure ulcer
* incidence
* umbrella review
* overview
## INTRODUCTION
Pressure injuries (PI), also known as pressure ulcers or decubitus ulcers, have an estimated global prevalence of 12.8% among hospitalised adults,1 and place a significant burden on healthcare systems (estimated at $26.8 billion per year in the US alone2). PIs are most common in individuals with reduced mobility, limited sensation, poor circulation, or compromised skin integrity, and can affect those in community settings and long-term care as well as hospital settings. Effective prevention of PI requires multicomponent preventive strategies such as mattresses, overlays, and other support systems, nutritional supplementation, repositioning, dressings, creams, lotions, and cleansers.3 4 Health economic models have suggested that providing baseline preventive interventions for all with daily risk assessments is more cost-effective than either a less standardised prevention protocol or a targeted risk-stratified prevention strategy.5 Nevertheless, the stratification of patients by risk could further improve outcomes by allowing timely and targeted implementation of additional or greater intensity preventive measures in those most at risk, to reduce harm and consequently burden to healthcare systems.6
Numerous clinical assessment scales and statistical risk prediction models for assessing the risk of PI are available. However, the methodology underlying their development is not always explicit, with scales in routine clinical usage apparently based on epidemiological evidence and clinical judgment about predictors that may not meet accepted principles for the development and reporting of risk prediction models.7 The Braden8 9, Norton10 and Waterlow11 scales are recommended by NICE guidelines12 in the UK and referenced in international guidelines for PI prevention.13 In some hospitals and long-term care settings in the US, healthcare professionals must conduct mandatory risk assessments for PI for all patients for the purposes of risk stratification and clinical triage. The Braden scale, developed in 1987 using a sample of 102 elderly hospital patients in the US includes sensory perception, moisture, activity, mobility, nutrition, friction and shear as predictors.8 9 The Norton scale, based on a sample of 250 elderly hospital patients in the UK and published in 1962, includes physical condition, mental status, activity, mobility and continence domains.10 The Waterlow scale was published in 1985 for use by Waterlow’s nursing students in the UK14, and assesses BMI, assessment of the skin, gender, age, malnutrition, incontinence, mobility, tissue malnutrition, neurological deficits, major surgery or trauma and medication.11
Despite the apparent lack of reporting of now standard methods for development and validation (including external validation) of available risk prediction tools, there is a considerable body of evidence evaluating their clinical utility, much of which has been synthesised in systematic reviews and meta-analyses.7 Clinical utility includes both prognostic accuracy and clinical effectiveness.
Prognostic accuracy is estimated by applying a numeric threshold above (or below) which there is a greater risk of PI, with study results presented using accuracy metrics such as sensitivity, specificity or the area under the receiver operating characteristic (ROC) curve.15 Resulting accuracy is driven not only by the nominated threshold for defining participants as at low or high risk for PI but by other study factors including population and setting.16 Clinical effectiveness, or the ability of a tool to ultimately impact on health outcomes such as the incidence or severity of PI, is related both to the accuracy of the tool (or its ability to correctly identify those most likely to develop PI), to the uptake and implementation of the tool in practice and to the consequential changes in PI management based on tool predictions. Demonstrating a change in health outcomes as a result of use of a risk prediction tool is vital to encourage implementation.17
Using an umbrella review approach, we aimed to provide a comprehensive overview of available systematic reviews that consider the prognostic accuracy and clinical effectiveness of PI risk prediction tools.
## METHODS
### Protocol registration and reporting of findings
We followed Cochrane guidance for conducting umbrella reviews18, and ‘Preferred Reporting Items for Systematic Reviews and Meta-Analyses of Diagnostic Test Accuracy Studies’ (PRISMA-DTA) reporting guidelines19 (see Appendix 1 in S1 File). The protocol was registered on Open Science Framework ([https://osf.io/tepyk](https://osf.io/tepyk)).
### Literature search
Electronic searches of MEDLINE, Embase via Ovid and CINAHL Plus EBSCO from inception to June 2024 were developed and conducted by an experienced information specialist (AC), employing well- established systematic review and prognostic search filters,20–22 combined with appropriate keywords related to PIs. Simplified supplementary searches in EPISTEMONIKOS and Google Scholar were also undertaken, with the latter covering the years 2013 to June 2024 (see Appendix 2 in S1 File for further details). Screening of search results and full texts were conducted independently and in duplicate by any two from a group of four reviewers (BH, JD, YT, KS), with arbitration by a third reviewer where necessary (any one of the four reviewers not involved in the independent screening).
### Eligibility criteria for this umbrella review
Published English-language systematic reviews of risk prediction tools developed for adult patients at risk of PI in any setting were included. Clinical risk assessment scales and models developed using statistical or machine learning (ML) methods were eligible (models exclusively using pressure sensor data were not considered). Risk prediction tools could be applied by any healthcare professional using any threshold for classifying patients as high or low risk and using any PI classification system13 23–25 as a reference standard. For prognostic accuracy, we required accuracy metrics, such as sensitivity and specificity, to be presented but did not require full 2x2 classification tables to be reported. Reviews on diagnosing or staging suspected or existing PIs were excluded.
To be considered ‘systematic’, reviews were required to report a thorough search of at least two electronic databases and at least one other indication of systematic methods (e.g. explicit eligibility criteria, formal quality assessment of included studies, adequate data presentation for reproducibility of results, or review stages (e.g. search screening) conducted independently in duplicate).
### Data extraction and quality assessment
Data extraction forms (Appendix 3) were informed by the CHARMS checklist (CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies) and Cochrane Prognosis group template.26 27 Data extraction items included review characteristics, number of studies and participants, study quality and results.
The methodological quality of included systematic reviews was assessed using an adapted version of AMSTAR-2 (A Measurement Tool to Assess Systematic Reviews).28 For example, for reviews evaluating the prognostic accuracy of risk prediction tools we assessed eligibility criteria using the PIRT framework (Population, Index test, Reference standard, Target condition)29 and POII framework (Population, Outcome to be predicted, Intended use of model, Intended moment in time)30 and required methodological quality assessment to be conducted using validated and appropriate tools such as QUADAS31, QUADAS-232 or PROBAST33. We omitted the AMSTAR-2 item relating to publication bias (Item 15) because of the lack of empirical evidence for the effect of publication bias on test accuracy estimates, and limitations in statistical methods for identifying publication bias.19 34 Our adapted AMSTAR-2 contains six critical items, and limitations in any of these items reduces the overall validity of a review.28 Full details can be found in Appendix 4 in S1 File. Quality assessment and data extraction were conducted by one reviewer and checked by a second (BH, JD, KS), with disagreements resolved by consensus.
### Synthesis methods
Reviews about prognostic accuracy and clinical effectiveness of risk prediction tools were considered separately. Review methods and results were tabulated, and a narrative synthesis provided.
Prognostic accuracy results from reviews including a statistical synthesis were tabulated according to risk prediction tool.
Considerable overlap in risk prediction tools and included primary studies was noted between reviews. For risk prediction tools that were included in multiple meta-analyses, we focused our synthesis on the review(s) with the most recent search date or most comprehensive (based on number of included studies) and most robust estimate of prognostic accuracy (judged according to the appropriateness of the meta-analytic method used, e.g. use of recommended hierarchical approaches for test accuracy data35). The prognostic accuracy of risk prediction tools that were included in three or fewer reviews, was reported only if an appropriate method of statistical synthesis18 was used.
For clinical effectiveness results, reviews with the most recent search date or most comprehensive overview of available studies, that assessed PI incidence outcomes and that at least partially met more of the AMSTAR-2 criteria28 were prioritised for narrative synthesis.
## RESULTS
### Characteristics of included reviews
A total of 118 records were selected for full-text assessment from 7200 unique records. We could obtain the full text of 111 publications, of which 26 reviews met all eligibility criteria (Figure 1), 19 reported accuracy data36–54 and 11 reported clinical effectiveness data38 42 43 49 55–61 (four reported both accuracy and effectiveness data38 42 43 49). Tables 1-2 provide an overview of the characteristics, methods and methodological quality of all 26 reviews (see Appendix 5 in S1 File for full details).
[Figure 1.](http://medrxiv.org/content/early/2024/09/25/2024.05.07.24307001/F1)
Figure 1.
PRISMA flowchart: identification, screening and selection process
List of full-text articles excluded, with reasons, is given in Appendix 5 in S1 File.
View this table:
[Table 1.](http://medrxiv.org/content/early/2024/09/25/2024.05.07.24307001/T1)
Table 1. Summary of included systematic review characteristics
View this table:
[Table 2.](http://medrxiv.org/content/early/2024/09/25/2024.05.07.24307001/T2)
Table 2. Summary of AMSTAR-2 assessment results.
Reviews were published between 2006 and 2024. Over half (15/26, 58%) restricted inclusion to adult populations (Table 1), two (8%) included any age group, and nine (35%) did not report any age restrictions. Six reviews (6/26, 23%) only included study populations with no PI at baseline. Acute care was the most frequent setting across both review questions (7/19 (37%) accuracy reviews and 3/11 (27%) effectiveness reviews). Quality assessment tools included QUADAS-2 (n=8) or QUADAS (n=2) in more than half of reviews of accuracy (10/19, 53%). One review47 utilised and reported PROBAST assessments for risk of bias. Another review48 reported using QUADAS-2 and PROBAST tools in their methods, but only reported QUADAS-2 results.
Reviews of accuracy either included studies evaluating any tool (5/19, 26%) or pre-specified tools (10/19, 53%); two47 48 included only ML-based prediction models, while the remaining two49 50 did not specify the tools to be included. A total of 70 risk prediction tools were reported across the reviews (from one37 40 41 46 51 52 to 2839 tools included per review), including 31 ML models. Only two reviews reported eligibility criteria related to the development or validation of the risk prediction tools. One43 (6%) excluded evaluation studies that used the same data that was used to develop the tool and the other38 included only “validated risk assessment instruments” with no further definition (yet included studies reporting original tool development).
The majority (15/19, 79%) of accuracy reviews conducted a meta-analysis, but only two utilized currently recommended hierarchical approaches for the meta-analysis of test accuracy data.41 53 Eight reviews conducted univariate meta-analysis of individual accuracy measures (e.g. sensitivity and specificity separately, or area under the curve (AUC)50, risk ratios (RR)39 or odds ratio43) and five did not clearly report the type of analysis approach used.
Of the 11 systematic reviews evaluating clinical effectiveness, two only considered the reliability of risk assessment scales49 58, one considered reliability and other ‘psychometric’ properties42, and eight considered effects on patient outcomes (one of which also considered tool reliability55). More than half of reviews (6, 55%) compared use of PI risk assessment scales to clinical judgement alone or ‘standard care’. The number of included studies ranged from one56 to 2060, with sample sizes ranging from one (one subject and 110 raters, in an inter-rater reliability study62) to 4,137 patients. Reported outcomes included the incidence of PIs (7/11), preventive interventions prescribed (5/11) and interrater reliability (4/11), internal consistency, measurement error and convergent validity (1/11) (latter four properties reported in Appendix 5 in S1 File). One review61 used the Cochrane risk of bias (RoB) tool for quality assessment of included studies, and three used JBI (n=2) or CASP (n=1) tools. Due to heterogeneity in study design, risk prediction tools and outcomes evaluated, none of the included reviews provided any form of statistical synthesis of study results.
### Methodological quality of included reviews
The quality of included reviews was generally poor (Table 2; Appendix 5 in S1 File). The AMSTAR-2 items that were most consistently met (yes or partial yes) were: comprehensiveness of the search (21/26, 81%), study selection independently in duplicate (17/26, 65%), data extraction in independently in duplicate (15/26, 58%), and conflicts of interest reported (20/26, 77%).
Six (32%) accuracy reviews 36 40 41 47 48 53 and two (18%) effectiveness reviews used an appropriate method of quality assessment of included studies (i.e. QUADAS or QUADAS-2 dependent on publication year, or PROBAST for accuracy and the Cochrane tool for assessing risk of bias64 and criteria consistent with AHRQ Methods Guide for Effectiveness and Comparative Effectiveness Reviews65 for effectiveness reviews) and also presented judgements per study. Five reviews either reported quality assessment results per study (n=442 58–60) or were considered to use an appropriate quality assessment tool (n=143) (AMSTAR-2 criterion partially met).
Of the accuracy reviews that included a statistical synthesis, 25% (4/16)39 41 50 53 used an appropriate meta-analytic method and investigated sources of heterogeneity. Two reviews41 53 used recommended hierarchical approaches to meta-analysis of test accuracy data (the bivariate model41 and hierarchical summary ROC (HSROC) model53) and two reviews calculated summary estimates of individual measures, using random effects meta-analyses (AUC50 or RR66).
Compared to the reviews of accuracy, reviews of effectiveness more commonly provided adequate descriptions of primary studies (8/11, 73% vs 1/19, 5%) and adequately defined their inclusion criteria (4/11, 36% vs 1/19, 5%) (Table 2). No other major differences across review questions were noted.
### Results from reviews evaluating the prognostic accuracy of risk prediction tools
Seven of 19 accuracy reviews were prioritised for narrative synthesis (Tables 3-4) and are reported below according to risk prediction tool. Five of the seven reviews did not include development study estimates within their meta-analyses, one review of ML models did not report this information48 and one47 restricted inclusion to studies reporting model development studies. The latter review was the only one to consider the effect of study quality in their statistical syntheses.
View this table:
[Table 3.](http://medrxiv.org/content/early/2024/09/25/2024.05.07.24307001/T3)
Table 3. Findings related to prognostic accuracy, by model: Characteristics and quality of studies included within reviews
View this table:
[Table 4.](http://medrxiv.org/content/early/2024/09/25/2024.05.07.24307001/T4)
Table 4. Summary estimates of accuracy parameters (main results from statistical syntheses), by prediction tool
### Braden, and modified Braden scales
The most recent and largest review41 of the Braden scale (60 studies, including 49,326 patients), which used hierarchical bivariate meta-analysis, reported an overall summary sensitivity of 0.78 (95% CI 0.74, 0.82; 15,241 patients) and specificity of 0.72 (95% CI 0.66, 0.78; 34,085 patients) across all reported thresholds (range ≤10 to ≤20). Summary sensitivities and specificities ranged from 0.79 (95% CI 0.76, 0.82) and 0.66 (95% CI 0.55, 0.75) at the lowest cut-offs for identification of high-risk patients (≤15 in 15 studies) to 0.82 (95% CI 0.73, 0.89) and 0.70 (95% CI 0.62, 0.77) using a cut-off of 18 (15 studies), respectively. Heterogeneity investigations suggested higher accuracy for predicting PI risk in patients with a mean age of 60 years or less, in hospitalised patients (compared to long-term care facility residents) and in Caucasian populations (compared to Asian populations).41 The review noted a high risk of bias for the ’index test’ section of the QUADAS-2 assessment in approximately a third of included studies, but failed to provide further details.
Two modified versions of the Braden scale67 68 were included in another review.44 Summary sensitivities were 0.97 (95% CI 0.92, 0.99; 125 patients from four studies)67 and 0.89 (95% CI 0.71, 0.98; 27 patients from two studies)68, and summary specificities were 0.70 (95% CI 0.66, 0.73; 563 patients)67 and 0.71 (95% CI 0.67, 0.75; 599 patients).68 The review was rated critically low on the AMSTAR-2 assessment, with only 1/15 (13%) criteria fulfilled. QUADAS-2 was reportedly used but results not reported in any detail, other than to indicate that none of the included studies were considered at high risk of bias.
### Cubbin & Jackson scale
The most recent and comprehensive review36 of the Cubbin & Jackson scale (9 studies, including 7,684 patients) reported summary sensitivity of 0.81 (95% CI 0.51, 0.95; 1,558 patients) and specificity of 0.76 (95% CI 0.58, 0.88; 6,126 patients). However, this review scored critically low on AMSTAR-2 (3/15, 20%, criteria fulfilled), with authors utilising inappropriate methods for statistical synthesis, not investigating causes of heterogeneity and poor reporting of results throughout. Their meta-analysis approach was also not clearly reported, but it appears that univariate meta-analyses were conducted separately for sensitivity and specificity, across studies with different Cubbin & Jackson thresholds.
Zhang and colleagues53 included six studies evaluating the original Cubbin & Jackson scale69 (800 patients). Summary sensitivity and specificity were both reported as 0.84 (95% CIs 0.59, 0.95 and 0.66, 0.93, respectively)53 suggesting that this represents the point on the HSROC curve where sensitivity equals specificity, particularly as reported thresholds ranged from 24 to 34. The review authors concluded that although the accuracy of the Cubbin & Jackson scale was higher than the EVARUCI scale and the Braden scale, low quality of evidence and significant heterogeneity limit the strength of conclusions that can be drawn.
### Norton scale
Park and colleagues44 synthesised data from seven studies (2,899 participants) evaluating the Norton scale, across thresholds ranging from <14 to <16. They reported summary sensitivity of 0.75 (95% CI 0.70, 0.79) and specificity 0.57 (95% CI 0.55, 0.59). A further four reviews presented statistically synthesised results for the Norton scale (Appendix 5 in S1 File), including one review by Chou and colleagues38 which included nine studies (5,444 participants) but only reported median values for accuracy parameters.
### Waterlow scale
Although Zhang and colleagues53 included the fewest participants (4 studies; 1,000 participants) of all six reviews that conducted a statistical synthesis of the accuracy of the Waterlow scale11, they provided the most recent review. It was rated highest on AMSTAR-2 criteria and appropriately used the HSROC model for meta-analysis across thresholds ranging from 12 to 25. Summary sensitivity was 0.63 (95% CI 0.48, 0.76) and summary specificity 0.46 (95% CI 0.22, 0.71) (Table 4). A second review44 reported contrasting results with summary sensitivity of 0.55 (95% CI 0.49, 0.62) and specificity 0.82 (95% CI 0.80, 0.85) (6 studies; 1268 participants), however authors synthesised data across multiple thresholds without utilising hierarchical methods.
### Machine learning algorithms
Pei and colleagues47 included 18 ML models, seven of which were not covered by any other included review. Accuracy measures were combined across all models that provided 2x2 data (n=14 models). The summary AUC across the 14 models was 0.94, summary sensitivity was 0.79 (95% CI 0.78, 0.80) and summary specificity was 0.87 (95% CI 0.88, 0.87) (Table 4). Meta-regression found no significant effect by ML algorithm or data type. Clinical heterogeneity was not investigated. The majority of studies (89%, 16/18) were considered at high risk of bias based on PROBAST. Our confidence in the review was critically low, with only 6/15 (40%) AMSTAR-2 criteria fulfilled. One critical flaw was the use of inappropriate meta-analysis methods (failing to use a hierarchical model for synthesising sensitivity and specificity).
Qu and colleagues48 conducted separate meta-analyses of 25 studies by ML algorithm type using Bayesian hierarchical methods (Table 3). The review rated critically low on AMSTAR-2 items, with only 6/15 (40%) criteria fulfilled. The review did not restrict inclusion to external evaluations of the models, and the authors did not report which estimates were sourced from development data or external data. The summary AUC for the five algorithms ranged from 0.82 (95% CI 0.79, 0.85; 9 studies with 97,815 participants) for neural network-based models to 0.95 (95% CI 0.93, 0.97; 7 studies with 161,334 participants) for random forest models (Table 4).
The latter approach also had the highest summary specificity 0.96 (95% CI 0.80, 0.99), with sensitivity 0.72 (95% CI 0.26, 0.95). The highest summary sensitivity was observed for support vector machine models (0.81, 95% CI 0.69, 0.90) with summary specificity 0.81 (95% CI 0.59, 0.93) (9 studies, 152,068 participants). The remaining algorithms had summary sensitivities ranging from 0.66 (decision tree models) to 0.73 (neural network models) (Table 4). Two additional ML algorithms evaluated in the included studies (Bayesian networks and LOS (abbreviation not explained)) had too few studies to allow meta-analysis (Appendix 5 in S1 File).
### Other scales
In addition to the risk prediction tools reported above, Zhang and colleagues53 reported on the EVARUCI scale70, presenting summary sensitivity and specificity of 0.84 (95% CI 0.79, 0.89) and 0.68 (95% CI 0.66, 0.70), respectively (3 studies; 3,063 participants). These results were synthesised across thresholds, 11 and 11.5 (one not reported).
Additional statistical syntheses covering three further modifications of the Braden scale (Braden modified by Kwong71, the 4-factor model72 and ‘extended Braden’72), two modified versions of the Norton scale (by Ek73, and by Bienstein74), a revised “Jackson & Cubbin”75, and the EMINA76 and PSPS77 tools ) were also identified.39 38 49 These analyses showed variable performance, often with high uncertainty. Full details can be found in Table A4 in S1 File.
Table A5 in S1 File reports data for another 17 risk prediction tools, each associated with a single primary study (therefore not covered in detail in the text above), and another two tools, Sunderland78 and RAPS79, which are assessed in two primary studies each.
### Results from reviews evaluating the clinical effectiveness of risk prediction tools
The 11 reviews reporting clinical effectiveness, used a range of eligibility criteria and a number of different quality assessment tools, leading to varying conclusions about the methodological quality of the same studies across reviews. Given the overlap in study inclusion between reviews Table 5 provides an overview of results from four38 57 59 61 of the 11 reviews, and a summary of the included comparative studies is provided below.
View this table:
[Table 5.](http://medrxiv.org/content/early/2024/09/25/2024.05.07.24307001/T5)
Table 5. Systematic reviews evaluating clinical effectiveness
Two randomised controlled trials (RCTs) of risk prediction tools83 84 were identified, both of which were considered at high risk of bias in the Cochrane review (assessed using the Cochrane RoB tool64). One of the trials (an individually randomised study83) was included in a further three reviews which considered it to be ‘good quality’38, ‘valid’56, or ‘high quality’59. The trial was conducted in 1,231 hospital inpatients and the only intervention was that the staff must use the tool that was allocated to them, with no other protocol prescribed changes made to routine care. However, no evidence of a difference in PI incidence was found between patients assessed with either the Waterlow scale or Ramstadius tool compared with clinical judgment alone (RR 1.10 (95% CI 0.68, 1.81) and RR 0.79 (95% CI 0.46, 1.35), respectively). The trial further showed no evidence of a difference in patient management or in PI severity when using a risk assessment tool compared to clinical judgement.
A further cluster randomised trial84 was considered to be of poor methodological quality both in the Cochrane review38 and one other review61. The trial included 521 patients at a military hospital and compared nurse training with mandatory use of the Braden scale, to nurse training and optional use of the Braden scale, to no training. No evidence of a difference in PI incidence was observed between the three groups: incidence rates were 22%, 22% and 15% (p=0.38), respectively.
Two reviews by Lovegrove and colleagues59 60 included an uncontrolled comparison study85 rated as high quality59. The study compared the clinical effectiveness of the Maelor scale86 used in an Irish hospital (121 patients) with nurses’ clinical judgement at a Norwegian hospital (59 patients). A higher rate of preventive strategies, as well as a lower PI prevalence (12% vs. 54%), was reported for the Irish hospital. However, these results are likely to be highly confounded by inherent differences in population and setting.
A non-randomised study by Gunningberg and colleagues87 included in two reviews43 57 was considered by review authors to be of relatively high quality. The study was conducted in 124 patients in emergency and orthopaedic units and compared the use of a PI risk alarm sticker for patients with a modified Norton Score of <21 (indicating high-risk patients) to standard care. No significant difference in the incidence of PIs between the Norton scale and standard care groups was observed.
A non-randomised study88 conducted in 233 hospice inpatients was included in three reviews,38 43 57 one of which is reported in Table 5.57 The study met six of eight quality criteria used by Health Quality Ontario.57 Use of a modified version of the Norton scale (Norton modified by Bale), in conjunction with standardised use of preventive interventions based on risk score, was found to be associated with lower risk of PIs when compared with nurses’ clinical judgment alone (RR 0.11, 95% CI 0.03, 0.46). The lack of randomisation limits the reliability of this result, and review authors report that the modified Norton scale had not been validated.
Finally, a ’before-and-after’ study89 of 181 patients in various hospital settings was included in two reviews,43 57 one of which considered the study to meet all quality criteria.57 Use of the Norton scale with additional training for staff was associated with significant differences in the number of preventive interventions prescribed compared to standard care (18.96 vs. 10.75, respectively).
Preventive interventions were also introduced earlier in the intervention group (on day 1, 61% vs. 50%, p<0.002 for Norton and usual care, respectively). However, no significant difference in the incidence of PIs was detected between the groups.
## DISCUSSION
This umbrella review summarises data from a total of 26 systematic reviews of studies evaluating the prognostic accuracy and clinical effectiveness of a total of 70 PI risk prediction tools. Despite the large number of available reviews, quality assessment using an adaptation of AMSTAR-2 suggested that the majority were conducted to a relatively poor standard or did not meet reporting standards for systematic reviews.19 90 Of the 15 AMSTAR-2 items assessed, only two (for accuracy reviews) and four (for effectiveness reviews) criteria were more consistently met (more than 60% of reviews scoring ‘Yes’). Whilst AMSTAR-2 Item 6 (data extraction independent in duplicate) was fulfilled by over half of all reviews (15/26, 58%), and Item 14 (adequate heterogeneity investigation) was fulfilled by around half of the accuracy reviews (10/19, 53%), all other criteria were fully met by less than half of the reviews. The primary studies included in the reviews were particularly poorly described in the accuracy reviews, making it difficult to determine exactly what was evaluated and in whom. The extent to which we could reliably describe and comment on the content of the reviews is limited and high-quality evidence for the accuracy and clinical effectiveness of PI risk prediction tools may be lacking.
### Prognostic accuracy of risk prediction tools
Of the 19 reviews reporting the accuracy of included tools, only two used appropriate methods for both quality assessment and statistical synthesis of accuracy data41 53, one of which41 evaluated only the Braden scale. Only two reviews42 43 pre-specified the exclusion of studies reporting accuracy data from tool development studies, one review restricted to “validated risk assessment instruments” only38 and one review47 was limited to development studies only. This was the only review47, that discussed the importance of appropriate validation of prediction tools. Only two reviews conducted meta-analyses at different cut-offs for determination of high risk38 41; the remaining reviews combined data regardless of the threshold used. Combining data across different thresholds to estimate summary sensitivity and specificity yields clinically uninterpretable and non-generalisable estimates that do not relate to a particular threshold.35 Only one review38 considered timing in their inclusion criteria or in the description of primary studies. It is important to interpret the findings below with these limitations in mind.
The included meta-analyses consistently suggested that risk prediction scales have moderate sensitivities and somewhat lower specificities, typically in the range of around 70% to 85% for sensitivity and as low as 30% to 40% for specificity for some tools. Although these ranges in sensitivities and specificities would be considered on the lower end of acceptable within a diagnostic accuracy paradigm, they may have greater utility in a prognostic context. Without a detailed review of the primary study publications for these tools, it is not possible to assess which, if any, of these risk assessment scales might outperform the others. It seems that limited comparative studies comparing the accuracy of different tools are available.
For the ML-based models, one review47 combined multiple ML models into one meta-analysis and another48 meta-analysed accuracy data by algorithm type. The results of the latter meta-analyses are not informative for clinical practice but may be a useful way of identifying which ML algorithms may be more suited to the data. Results suggested that specificities for random forest or decision tree models could reach 90% or above with associated sensitivities in the range of 66% to 72%, however relatively wide confidence intervals around these summary estimates reflect considerable variation in model performance. Moreover, some of these estimates came from internal validations within model development studies, and may not be transferable to other settings.91 Authors should make it clear where accuracy estimates are derived from to avoid overinterpretation of results.
Diagnostic accuracy studies are typically cross-sectional in the sense that there should be no, or only minimal delay between application of the test and the reference standard.92 93 For prognostic accuracy however, there is a time delay between the application of the test and the outcome that the tool aims to predict. If the use of an accurate PI risk prediction tool is combined with effective and appropriate preventive measures in those identified as most at risk, the incidence of PI would decline, reducing the positive predictive value of the original risk assessment and potentially the sensitivity of the tool.94 Sensitivity and specificity can be optimised by methods which directly consider the cost of misclassification, including both the harms associated with applying more intensive prevention in those with a false positive result and the benefits of preventive measures in those with a true positive result. One solution to determine the preventive treatment threshold risk is through net benefit calculations,95 96 which can be visualised in decision curves and are common in prognostic research. These calculations can assist in providing a balanced use of resources while maximising positive health outcomes, such as lowering incidence of PI.
It is important to also consider that not all predictors have a causal relationship with the outcome, therefore, not every predictor will be a clinical risk modifier. Risk assessment tools that allow a more personalised-risk approach, i.e. that identify and flag predictors that are risk modifiers to the end- users of the tool, would make predictions more interpretable and actionable. Some such developments exist,97 98 but future validation of these methods is needed. Where risk assessment tools are developed for enriching study design (for example, as a means of recruiting only high-risk patients to studies evaluating preventive measures), a different approach and optimisation of performance metrics would be needed. Risk prediction models should therefore pre-specify their intended application before development to allow their clinical utility for a given context to be addressed.99
### Clinical effectiveness of risk prediction scales
Prediction models, like any test used for diagnostic or prognostic purposes, require evaluation in the care pathway to identify the extent to which their use can impact on health outcomes.100 Of the 11 reviews assessing clinical effectiveness of PI risk prediction tools, the only primary studies suggesting potential patient benefits from the use of risk prediction tools85 88 89 were non-randomised and are likely to be at high risk of bias. In contrast, two randomised trials83 84 (both considered at high risk of bias by the Cochrane review61) suggest that use of structured risk assessment tools does not ultimately lead to the reduction in incidence of PIs. We should recognise that effectiveness outcomes from using a risk prediction tool depend on the timely implementation of effective preventive measures, a step that is frequently poorly described in studies evaluating the effectiveness of risk assessment tools, restricting the conclusions that can be drawn from the limited evidence available. One possible explanation for the lack of differences in PI incidence is the implementation of preventive measures that have not been proven effective in preventing PIs, such as alternating air- mattresses.4 All reviews included studies that assessed the use of risk assessment scales developed by clinical experts, and no evidence is available evaluating the clinical effectiveness of empirically derived prediction models or ML algorithms.
### Other existing evidence
We have separately reviewed7 available evidence for the development and validation of risk prediction tools for PI occurrence. Almost half (60/124, 48%) of available tools were developed using ML methods (as defined by review authors), 37% (46/124) were based on clinical expertise or unclear methods, and only 18 (15%) were identified as having used statistical modelling methods. The reviews varied in methodological quality and reporting; however, the reporting of prediction model development in the original primary studies appears to be poor. For example, across all prediction tools identified, the internal validation approach was unclear and unidentifiable for 72% (89/124) of tools, and only one review101 identified and included external validation studies (n=7 studies).
ML-based models may have potential for identifying those at risk of PI, as suggested by two reviews47 48 included in this umbrella review. However, it is important to consider the lack of transparency in reporting of model development methods and model performance, and the concerning lack of model validation in populations outside of the original model development sample.7
### Strengths and limitations
We have conducted the first umbrella review that summarises the prognostic accuracy and clinical effectiveness of prediction tools for risk of PI. We followed Cochrane guidance18, with a highly sensitive search strategy designed by an experienced information specialist. Although we excluded non-English publications due to time and resource constraints, where possible these publications were used to identify additional eligible risk prediction tools.
To some extent, our review is limited by the use of AMSTAR-2 for quality assessment of included reviews. AMSTAR-2 was not designed for assessing systematic reviews of diagnostic or prognostic studies. Although we made some adaptations, many of the existing and amended criteria relate to the quality of reporting of the reviews as opposed to methodological quality. There is scope for further work to establish criteria for assessing systematic reviews of prediction tools. Additionally, we chose not to exclude reviews based on low AMSTAR-2 ratings to provide a comprehensive overview of all available evidence. However, by doing so, we acknowledge that many included reviews are of poor quality (with critically low confidence in 81%, 21/26, reviews), reducing the reliability of the evidence presented and the ability to make conclusions or recommendations based on this evidence.
The primary limitation of our study lies in the limited detail available on risk prediction tools and their performance within the included systematic reviews. To ensure comprehensive model identification, we adopted a broad definition of ’systematic’, potentially influencing the depth of information provided in the reviews, and the reporting quality in many primary studies contributing to these reviews may be suboptimal.
Although standards for reporting of test accuracy studies have been available since the year 2000,92 standards for reporting risk prediction models were not published until 2015.102 Similarly, quality assessment tools highlighting important areas for consideration in primary studies have been available for DTA studies since 2003, with an adaption to prognostic accuracy published in 2022,103 and PROBAST for prediction model studies in 2019.33 This lag in methodological developments for studies and systematic reviews of risk prediction tools has likely contributed to the observed emphasis on the application of DTA principles in our set of reviews, without sufficient consideration of the prognostic context and effect on accuracy of intervening and effective preventive interventions.
While 18/19 (95%) accuracy reviews aimed to evaluate the ‘predictive’ validity of PI risk assessment tools, the majority (16/19, 84%) relied on DTA principles without any consideration of the time interval between the test and the outcome, i.e. occurrence of PI. This approach does not account for the prognostic nature of these tools or address longitudinal questions, such as censoring and competing events.103 Another fundamental flaw in these accuracy assessments is that risk scales may actually appear to perform worse in settings where risk prediction and preventive care are most effective, as accurate risk prediction combined with effective preventive measures may prevent patients classified as ‘high-risk’ from developing PIs.94
## CONCLUSIONS
In conclusion, this umbrella review comprehensively summarises the prognostic accuracy and clinical effectiveness of risk prediction tools for developing PIs. The included systematic reviews used poor methodology and reporting, limiting our ability to reliably describe and evaluate their content. ML- based models demonstrated potential, with high specificity reported for some models. Wide confidence intervals highlight the variability in current evaluations, and external validation of ML tools may be lacking. The prognostic accuracy of clinical scales and statistically derived prediction models has a substantial range of specificities and sensitivities, motivating further model development with high quality data and appropriate statistical methods.
Regarding clinical effectiveness, a reduction of PI incidence is unclear due the overall uncertainty and potential biases in available studies. This underscores the need for further research in this critical area, once promising prediction tools have been developed and appropriately validated. In particular, the clinical impact of newer ML-based models currently remains largely unexplored. Despite these limitations, our umbrella review provides valuable insights into the current state of PI risk prediction tools, emphasising the need for robust research methods to be used in future evaluations.
## Supporting information
Appendices [[supplements/307001_file03.pdf]](pending:yes)
## Data Availability
All data produced in the present work are contained in the manuscript and supplementary file
## Supporting Information
**S1 File. Appendices.**
## Author Contributions
**Conceptualisation:** Bethany Hillier, Katie Scandrett, April Coombe, Tina Hernandez-Boussard, Ewout Steyerberg, Yemisi Takwoingi, Vladica Velickovic, Jacqueline Dinnes
**Data curation:** Bethany Hillier, Katie Scandrett, April Coombe, Jacqueline Dinnes
**Formal analysis:** Bethany Hillier, Katie Scandrett, Jacqueline Dinnes
**Funding acquisition:** Yemisi Takwoingi, Vladica Velickovic, Jacqueline Dinnes
**Investigation:** Bethany Hillier, Katie Scandrett, April Coombe, Yemisi Takwoingi, Jacqueline Dinnes
**Methodology:** Bethany Hillier, Katie Scandrett, April Coombe, Tina Hernandez-Boussard, Ewout Steyerberg, Yemisi Takwoingi, Vladica Velickovic, Jacqueline Dinnes
**Project administration:** Bethany Hillier, Yemisi Takwoingi, Jacqueline Dinnes
**Resources:** Bethany Hillier, Katie Scandrett
**Supervision:** Yemisi Takwoingi, Jacqueline Dinnes
**Writing – original draft:** Bethany Hillier, Katie Scandrett, April Coombe, Jacqueline Dinnes
**Writing – review & editing:** Bethany Hillier, Katie Scandrett, April Coombe, Tina Hernandez-Boussard, Ewout Steyerberg, Yemisi Takwoingi, Vladica Velickovic, Jacqueline Dinnes
## Funding
This work was commissioned and supported by Paul Hartmann AG (Heidenheim, Germany), part of HARTMANN GROUP. The contract with the University of Birmingham was agreed on the legal understanding that the authors had the freedom to publish results regardless of the findings.
YT, JD and BH are funded by the National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre (BRC). This paper presents independent research supported by the NIHR Birmingham BRC at the University Hospitals Birmingham NHS Foundation Trust and the University of Birmingham. The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care.
## Conflicting Interests
I have read the journal’s policy, and the authors of this manuscript have the following competing interests: VV is an employee of Paul Hartmann AG; ES and THB received consultancy fees from Paul Hartmann AG. VV, ES and THB were not involved in data curation, screening, data extraction, analysis of results or writing of the original draft. These roles were conducted independently by authors at the University of Birmingham. All other authors received no personal funding or personal compensation from Paul Hartmann AG and have declared that no competing interests exist.
## Acknowledgements
We would like to thank Mrs. Rosie Boodell (University of Birmingham, UK) for her help in acquiring the publications necessary to complete this piece of work.
## Footnotes
* We have made several amendments to our paper to ensure its suitability for publication. In particular, we have updated the search of our umbrella review (in June 2024) and added critical discussions and limitations of the reliance on diagnostic test accuracy metrics for prediction tools. We also expanded our discussions on the importance on using risk prediction tools to help guide effective preventive measures for pressure injuries, and added various clarifications where suggested by reviewers. An Author Summary has also been added.
* Received May 7, 2024.
* Revision received September 25, 2024.
* Accepted September 25, 2024.
* © 2024, Posted by Cold Spring Harbor Laboratory
This pre-print is available under a Creative Commons License (Attribution 4.0 International), CC BY 4.0, as described at [http://creativecommons.org/licenses/by/4.0/](http://creativecommons.org/licenses/by/4.0/)
## References
1. 1.Li Z, Lin F, Thalib L, et al. Global prevalence and incidence of pressure injuries in hospitalised adult patients: A systematic review and meta-analysis. International Journal of Nursing Studies 2020;105:103–546. doi: 10.1016/j.ijnurstu.2020.103546
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.ijnurstu.2020.103546&link_type=DOI)
2. 2.Padula WV, Delarmente BA. The national cost of hospital-acquired pressure injuries in the United States. Int Wound J 2019;16(3):634–40. doi: 10.1111/iwj.13071 [published Online First: 2019/01/28]
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/iwj.13071&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
3. 3.Sullivan N, Schoelles K. Preventing In-Facility Pressure Ulcers as a Patient Safety Strategy. Annals of Internal Medicine 2013;158(5.2):410-16. doi: 10.7326/0003-4819-158-5-201303051-00008
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7326/0003-4819-158-5-201303051-00008&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=23460098&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
[Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000316058600008&link_type=ISI)
4. 4.Qaseem A, Mir TP, Starkey M, et al. Risk Assessment and Prevention of Pressure Ulcers: A Clinical Practice Guideline From the American College of Physicians. Annals of Internal Medicine 2015;162(5):359–69. doi: 10.7326/m14-1567
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7326/M14-1567&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=25732278&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
5. 5.Padula WV, Pronovost PJ, Makic MBF, et al. Value of hospital resources for effective pressure injury prevention: a cost-effectiveness analysis. BMJ Quality & Safety 2019;28(2):132. doi: 10.1136/bmjqs-2017-007505
[Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjI4LzIvMTMyIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjQvMDkvMjUvMjAyNC4wNS4wNy4yNDMwNzAwMS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=)
6. 6.Institute for Quality and Efficiency in Health Care (IQWiG). Preventing pressure ulcers. Cologne, Germany 2006 [updated 2018 Nov 15. Available from: [https://www.ncbi.nlm.nih.gov/books/NBK326430/?report=classic](https://www.ncbi.nlm.nih.gov/books/NBK326430/?report=classic) accessed Feb 2023].
7. 7.Hillier B, Scandrett K, Coombe A, et al. Development and validation of risk prediction tools for pressure injury occurrence: An umbrella review (pre-print). MedRxiv 2024 doi: 10.1101/2024.05.07.24306999
[Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoibWVkcnhpdiI7czo1OiJyZXNpZCI7czoyMToiMjAyNC4wNS4wNy4yNDMwNjk5OXYxIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjQvMDkvMjUvMjAyNC4wNS4wNy4yNDMwNzAwMS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=)
8. 8.Braden B, Bergstrom N. A Conceptual Schema for the Study of the Etiology of Pressure Sores. Rehabilitation Nursing 1987;12(1):8–16. doi: 10.1002/j.2048-7940.1987.tb00541.x
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/j.2048-7940.1987.tb00541.x&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=3643620&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
9. 9.Bergstrom N, Braden BJ, Laguzza A, et al. The Braden Scale for Predicting Pressure Sore Risk. Nurs Res 1987;36(4):205–10.
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/00006199-198705000-00025&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=3299278&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
[Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1987J233400002&link_type=ISI)
10. 10.Norton D. Geriatric nursing problems. Int Nurs Rev 1962;9:39–41.
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=14480428&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
11. 11.Waterlow J. Pressure sores: a risk assessment card. Nursing Times 1985;81:49–55.
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=3844179&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
12. 12.NICE. Pressure ulcers: prevention and management. Clinical guideline [CG179]. 2014 [Available from: [https://www.nice.org.uk/guidance/cg179](https://www.nice.org.uk/guidance/cg179) accessed Aug 2024].
13. 13.Haesler E. European Pressure Ulcer Advisory Panel, National Pressure Injury Advisory Panel and Pan Pacific Pressure Injury Alliance. Prevention and Treatment of Pressure Ulcers/Injuries: Clinical Practice Guideline. 2019 [Available from: [https://internationalguideline.com/2019](https://internationalguideline.com/2019) accessed Feb 2023].
14. 14.Scott K, Longstaffe S. Judy Waterlow. 2020 [Available from: [https://litfl.com/judy-waterlow/](https://litfl.com/judy-waterlow/) accessed Aug 2024].
15. 15.Šimundić AM. Measures of Diagnostic Accuracy: Basic Definitions. EJIFCC 2009;19(4):203-11. [published Online First: 2009/01/20]
16. 16.Leeflang MM, Rutjes AW, Reitsma JB, et al. Variation of a test’s sensitivity and specificity with disease prevalence. CMAJ 2013;185(11):E537–44. doi: 10.1503/cmaj.121286 [published Online First: 2013/06/24]
[Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMToiMTg1LzExL0U1MzciO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyNC8wOS8yNS8yMDI0LjA1LjA3LjI0MzA3MDAxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==)
17. 17.Maiga A, Farjah F, Blume J, et al. Risk Prediction in Clinical Practice: A Practical Guide for Cardiothoracic Surgeons. Ann Thorac Surg 2019;108(5):1573–82. doi: 10.1016/j.athoracsur.2019.04.126 [published Online First: 2019/06/27]
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.athoracsur.2019.04.126&link_type=DOI)
18. 18.Pollock M, Fernandes RM BL, Pieper D, Hartling L,. Chapter V: Overviews of Reviews. In: Higgins JPT TJ, Chandler J, Cumpston M, Li T, Page MJ, Welch VA ed. Cochrane Handbook for Systematic Reviews of Interventions version 63 (updated February 2022). Available from [www.training.cochrane.org/handbook](https://www.training.cochrane.org/handbook): Cochrane 2022.
19. 19.McInnes MDF, Moher D, Thombs BD, et al. Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies: The PRISMA-DTA Statement. JAMA 2018;319(4):388–96. doi: 10.1001/jama.2017.19163
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.2017.19163&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=29362800&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
20. 20.Ingui BJ, Rogers MA. Searching for clinical prediction rules in MEDLINE. J Am Med Inform Assoc 2001;8(4):391–7. doi: 10.1136/jamia.2001.0080391 [published Online First: 2001/06/22]
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1136/jamia.2001.0080391&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=11418546&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
21. 21.Wilczynski NL, Haynes RB. Optimal Search Strategies for Detecting Clinically Sound Prognostic Studies in EMBASE: An Analytic Survey. Journal of the American Medical Informatics Association 2005;12(4):481–85. doi: 10.1197/jamia.M1752
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1197/jamia.M1752&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=15802476&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
22. 22.Geersing G-J, Bouwmeester W, Zuithoff P, et al. Search Filters for Finding Prognostic and Diagnostic Prediction Studies in Medline to Enhance Systematic Reviews. PLOS ONE 2012;7(2):e32844. doi: 10.1371/journal.pone.0032844
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pone.0032844&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=22393453&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
23. 23.NHS. Pressure ulcers: revised definition and measurement. Summary and recommendations 2018 [Available from: [https://www.england.nhs.uk/wp-content/uploads/2021/09/NSTPP-summary-recommendations.pdf](https://www.england.nhs.uk/wp-content/uploads/2021/09/NSTPP-summary-recommendations.pdf) accessed Feb 2023].
24. 24.AHCPR. Pressure ulcer treatment. : Agency for Health Care Policy and Research 1994:1–25.
25. 25.Harker J. Pressure ulcer classification: the Torrance system. Journal of Wound Care 2000;9(6):275–77. doi: 10.12968/jowc.2000.9.6.26233
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.12968/jowc.2000.9.6.26233&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=11933341&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
26. 26.Moons KGM, de Groot JAH, Bouwmeester W, et al. Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies: The CHARMS Checklist. PLOS Medicine 2014;11(10):e1001744. doi: 10.1371/journal.pmed.1001744
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pmed.1001744&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=25314315&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
27. 27.Cochrane. DE form example prognostic models - scoping review: The Cochrane Collaboration: The Prognosis Methods Group; [Available from: [https://methods.cochrane.org/prognosis/tools](https://methods.cochrane.org/prognosis/tools) accessed Feb 2023].
28. 28.Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ 2017;358:j4008. doi: 10.1136/bmj.j4008
[FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE4OiIzNTgvc2VwMjFfMTYvajQwMDgiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyNC8wOS8yNS8yMDI0LjA1LjA3LjI0MzA3MDAxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==)
29. 29. World Health O. WHO handbook for guideline development: Chapter 17: developing guideline recommendations for tests or diagnostic tools. 2nd ed. Geneva: World Health Organization 2014:218.
30. 30.Whiting P, Savović J, Higgins JP, et al. ROBIS: A new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol 2016;69:225–34. doi: 10.1016/j.jclinepi.2015.06.005 [published Online First: 20150616]
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jclinepi.2015.06.005&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=26092286&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
31. 31.Whiting P, Rutjes AWS, Reitsma JB, et al. The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol 2003;3(25) doi: 10.1186/1471-2288-3-25
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/1471-2288-3-25&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=14606960&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
32. 32.Whiting PF, Rutjes AW, Westwood ME, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011;155(8):529–36. doi: 10.7326/0003-4819-155-8-201110180-00009
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1059/0003-4819-155-8-201110180-00009&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=22007046&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
[Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000296066300018&link_type=ISI)
33. 33.Wolff RF, Moons KGM, Riley RD, et al. PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies. Annals of Internal Medicine 2019;170(1):51–58. doi: 10.7326/M18-1376
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7326/M18-1376&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=30596875&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
34. 34.Macaskill P, Gatsonis C, Deeks J, et al. Cochrane handbook for systematic reviews of diagnostic test accuracy: Version, 2010.
35. 35.Macaskill P, Takwoingi Y, Deeks J, et al. Chapter 9: Understanding meta-analysis. In: Deeks J, Bossuyt P, Leeflang M, et al., eds. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy. Version 2.0 ed: Cochrane, 2023 (updated July 2023).
36. 36.Chen X, Diao D, Ye L. Predictive validity of the Jackson–Cubbin scale for pressure ulcers in intensive care unit patients: A meta-analysis. Nursing in Critical Care 2023;28(3):370–78. doi: 10.1111/nicc.12818
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/nicc.12818&link_type=DOI)
37. 37.Chen HL, Shen WQ, Liu P. A Meta-analysis to Evaluate the Predictive Validity of the Braden Scale for Pressure Ulcer Risk Assessment in Long-term Care. Ostomy/wound management 2016;62(9):20–8.
38. 38.Chou R, Dana T, Bougatsos C, et al. Pressure ulcer risk assessment and prevention: a systematic comparative effectiveness review. Annals of internal medicine 2013;159(1):28–38.
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7326/0003-4819-159-1-201307020-00006&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=23817702&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
[Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000321516200004&link_type=ISI)
39. 39.García-Fernández FP, Pancorbo-Hidalgo PL, Agreda JJS. Predictive Capacity of Risk Assessment Scales and Clinical Judgment for Pressure Ulcers: A Meta-analysis. Journal of Wound Ostomy & Continence Nursing 2014;41(1):24–34. doi: 10.1097/01.WON.0000438014.90734.a2
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/01.WON.0000438014.90734.a2&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=24280770&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
40. 40.He W, Liu P, Chen HL. The Braden Scale cannot be used alone for assessing pressure ulcer risk in surgical patients: a meta-analysis. Ostomy/wound management 2012;58:34–40.
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=22316631&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
41. 41.Huang C, Ma Y, Wang C, et al. Predictive validity of the braden scale for pressure injury risk assessment in adults: A systematic review and meta-analysis. Nursing open 2021;8:2194–207. doi: 10.1002/nop2.792
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/nop2.792&link_type=DOI)
42. 42.Mehicic A, Burston A, Fulbrook P. Psychometric properties of the Braden scale to assess pressure injury risk in intensive care: A systematic review. Intensive & critical care nursing 2024;83:103686. doi: 10.1016/j.iccn.2024.103686
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.iccn.2024.103686&link_type=DOI)
43. 43.Pancorbo-Hidalgo PL, Garcia-Fernandez FP, Lopez-Medina IM, et al. Risk assessment scales for pressure ulcer prevention: a systematic review. J Adv Nurs 2006;54(1):94–110. doi: 10.1111/j.1365-2648.2006.03794.x
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1365-2648.2006.03794.x&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=16553695&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
[Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000236246800012&link_type=ISI)
44. 44.Park SH, Lee HS. Assessing Predictive Validity of Pressure Ulcer Risk Scales- A Systematic Review and Meta-Analysis. Iranian journal of public health 2016;45(2):122–33.
45. 45.Park SH, Lee YS, Kwon YM. Predictive Validity of Pressure Ulcer Risk Assessment Tools for Elderly: A Meta-Analysis. Western journal of nursing research 2016;38:459–83. doi: 10.1177/0193945915602259
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/0193945915602259&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=26337859&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
46. 46.Park SH, Choi YK, Kang CB. Predictive validity of the Braden Scale for pressure ulcer risk in hospitalized patients. Journal of Tissue Viability 2015;24:102–13. doi: 10.1016/j.jtv.2015.05.001
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jtv.2015.05.001&link_type=DOI)
47. 47.Pei J, Guo X, Tao H, et al. Machine learning-based prediction models for pressure injury: A systematic review and meta-analysis. Int Wound J 2023 doi: 10.1111/iwj.14280 [published Online First: 2023/06/20]
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/iwj.14280&link_type=DOI)
48. 48.Qu C, Luo W, Zeng Z, et al. The predictive effect of different machine learning algorithms for pressure injuries in hospitalized patients: A network meta-analyses. Heliyon 2022;8(11):e11361. doi: 10.1016/j.heliyon.2022.e11361
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.heliyon.2022.e11361&link_type=DOI)
49. 49.Tayyib NAH, Coyer F, Lewis P. Pressure ulcers in the adult intensive care unit: a literature review of patient risk factors and risk assessment scales. Journal of Nursing Education and Practice 2013;3(11):28–42.
50. 50.Wang N, Lv L, Yan F, et al. Biomarkers for the early detection of pressure injury: A systematic review and meta-analysis. Journal of Tissue Viability 2022;31:259–67. doi: 10.1016/j.jtv.2022.02.005
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jtv.2022.02.005&link_type=DOI)
51. 51.Wei M, Wu L, Chen Y, et al. Predictive Validity of the Braden Scale for Pressure Ulcer Risk in Critical Care: A Meta-Analysis. Nursing in critical care 2020;25:165–70. doi: 10.1111/nicc.12500
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/nicc.12500&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
52. 52.Wilchesky M, Lungu O. Predictive and concurrent validity of the Braden scale in long-term care: A meta-analysis. Wound Repair and Regeneration 2015;23:44–56. doi: 10.1111/wrr.12261
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/wrr.12261&link_type=DOI)
53. 53.Zhang Y, Zhuang Y, Shen J, et al. Value of pressure injury assessment scales for patients in the intensive care unit: Systematic review and diagnostic test accuracy meta-analysis. Intensive & critical care nursing 2021;64:103009. doi: 10.1016/j.iccn.2020.103009
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.iccn.2020.103009&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
54. 54.Zimmermann GS, Cremasco MF, Zanei SSV, et al. Pressure injury risk prediction in critical care patients: an integrative review. Texto & Contexto-Enfermagem 2018;27(3)
55. 55.Baris N, Karabacak BG, Alpar SE. The Use of the Braden Scale in Assessing Pressure Ulcers in Turkey: A Systematic Review. Advances in skin & wound care 2015;28:349–57. doi: 10.1097/01.ASW.0000465299.99194.e6
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/01.ASW.0000465299.99194.e6&link_type=DOI)
56. 56.Gaspar S, Peralta M, Marques A, et al. Effectiveness on hospital-acquired pressure ulcers prevention: a systematic review. International Wound Journal 2019;16(5):1087–102. doi: 10.1111/iwj.13147
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/iwj.13147&link_type=DOI)
57. 57.Ontario HQ. Pressure ulcer prevention: an evidence-based analysis. Ontario health technology assessment series 2009;9(2):1–104.
58. 58.Kottner J, Dassen T, Tannen A. Inter- and intrarater reliability of the Waterlow pressure sore risk scale: A systematic review. International Journal of Nursing Studies 2009;46:369–79. doi: 10.1016/j.ijnurstu.2008.09.010
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.ijnurstu.2008.09.010&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=18986650&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
59. 59.Lovegrove J, Ven S, Miles SJ, et al. Comparison of pressure injury risk assessment outcomes using a structured assessment tool versus clinical judgement: A systematic review. Journal of Clinical Nursing 2021 doi: 10.1111/jocn.16154 [published Online First: 2021/12/01]
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/jocn.16154&link_type=DOI)
60. 60.Lovegrove J, Miles S, Fulbrook P. The relationship between pressure ulcer risk assessment and preventative interventions: a systematic review. Journal of wound care 2018;27(12):862–75.
61. 61.Moore ZEH, Patton D. Risk assessment tools for the prevention of pressure ulcers. Cochrane Database of Systematic Reviews 2019 doi: 10.1002/14651858.CD006471.pub4
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/14651858.CD006471.pub4&link_type=DOI)
62. 62.Kelly J. Inter-rater reliability and Waterlow’s pressure ulcer risk assessment tool. Nurs Stand 2005;19(32):86–7, 90-2. doi: 10.7748/ns2005.04.19.32.86.c3851
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7748/ns2005.04.19.32.86.c3851&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=15875591&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
63. 63.Munoz N, Posthauer ME. Nutrition strategies for pressure injury management: Implementing the 2019 International Clinical Practice Guideline. Nutrition in Clinical Practice 2022;37(3):567-82.
64. 64.Higgins JPT, Altman DG, Gøtzsche PC, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 2011;343:d5928. doi: 10.1136/bmj.d5928
[FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNDMvb2N0MThfMi9kNTkyOCI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDI0LzA5LzI1LzIwMjQuMDUuMDcuMjQzMDcwMDEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9)
65. 65.AHRQ Methods for Effective Health Care. Methods Guide for Effectiveness and Comparative Effectiveness Reviews. Rockville (MD): Agency for Healthcare Research and Quality (US) 2008.
66. 66.Zahia S, Garcia Zapirain MB, Sevillano X, et al. Pressure injury image analysis with machine learning techniques: A systematic review on previous and possible future methods. Artificial Intelligence in Medicine 2020;102:101742. doi: 10.1016/j.artmed.2019.101742
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.artmed.2019.101742&link_type=DOI)
67. 67.Song M, Choi KS. Factors predicting development of decubitus ulcers among patients admitted for neurological problems. The Journal of Nurses Academic Society 1991;21(1):16–26.
68. 68.Pang SM, Wong TK. Predicting pressure sore risk with the Norton, Braden, and Waterlow scales in a Hong Kong rehabilitation hospital. Nursing Research 1998;47(3):147–53.
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/00006199-199805000-00005&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=9610648&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
[Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000073637900004&link_type=ISI)
69. 69.Cubbin B, Jackson C. Trial of a pressure area risk calculator for intensive therapy patients. Intensive Care Nursing 1991;7(1):40–44.
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/0266-612X(91)90032-M&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=2019734&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
70. 70.González-Ruiz J, Carrero AG, Blázquez MH, et al. Factores de riesgo de las úlceras por presión en pacientes críticos. Enfermería Clinica 2001;11(5):184–90.
71. 71.Kwong E, Pang S, Wong T, et al. Predicting pressure ulcer risk with the modified Braden, Braden, and Norton scales in acute care hospitals in Mainland China. Appl Nurs Res 2005;18(2):122–8. doi: 10.1016/j.apnr.2005.01.001
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.apnr.2005.01.001&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=15991112&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
[Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000230365700013&link_type=ISI)
72. 72.Halfens R, Van Achterberg T, Bal R. Validity and reliability of the Braden scale and the influence of other risk factors: a multi-centre prospective study. International Journal of Nursing Studies 2000;37(4):313–19.
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0020-7489(00)00010-9&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=10760538&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
73. 73.Ek AC. Prediction of pressure sore development. Scand J Caring Sci 1987;1(2):77–84. doi: 10.1111/j.1471-6712.1987.tb00603.x
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1471-6712.1987.tb00603.x&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=3134685&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
74. 74.Bienstein C. Risikopatienten erkennen mit der erweiterten Nortonskala [Risk patients detected with the extended Norton scale]. Dekubitus - Prophylaxe undTherapie. Frankfurt/Main: Verlag Krankenpflege 1991.
75. 75.Jackson C. The revised Jackson/Cubbin Pressure Area Risk Calculator. Intensive Crit Care Nurs 1999;15(3):169-75. doi: 10.1016/s0964-3397(99)80048-2
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/s0964-3397(99)80048-2&link_type=DOI)
76. 76.Fuentelsaz C. Validation of the EMINA scale: tool for the evaluation of risk of developing pressure ulcers in hospitalized patients. Enferm Clin [Internet*]* 2001;11(3):97–103.
77. 77.Lowthian P. The practical assessment of pressure sore risk. Care–Science and Practice 1987;5(4):3–7.
78. 78.Lowery MT. A pressure sore risk calculator for intensive care patients: ’the Sunderland experience’. Intensive Crit Care Nurs 1995;11(6):344–53. doi: 10.1016/s0964-3397(95)80452-8
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/s0964-3397(95)80452-8&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=8574087&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
79. 79.Lindgren M, Unosson M, Krantz AM, et al. A risk assessment scale for the prediction of pressure sore development: reliability and validity. Journal of advanced nursing 2002;38(2):190–99.
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1046/j.1365-2648.2002.02163.x&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=11940132&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
80. 80.Walter SD. Properties of the summary receiver operating characteristic (SROC) curve for diagnostic test data. Stat Med 2002;21(9):1237–56. doi: 10.1002/sim.1099
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/sim.1099&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=12111876&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
[Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000175217600004&link_type=ISI)
81. 81.Moses LE, Shapiro D, Littenberg B. Combining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerations. Stat Med 1993;12(14):1293–316. doi: 10.1002/sim.4780121403
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/sim.4780121403&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=8210827&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
[Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1993LQ06900002&link_type=ISI)
82. 82.Littenberg B, Moses LE. Estimating diagnostic accuracy from multiple conflicting reports: a new meta-analytic method. Med Decis Making 1993;13(4):313–21. doi: 10.1177/0272989x9301300408
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/0272989X9301300408&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=8246704&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
[Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1993MA38600008&link_type=ISI)
83. 83.Webster J, Coleman K, Mudge A, et al. Pressure ulcers: effectiveness of risk-assessment tools. A randomised controlled trial (the ULCER trial). BMJ Quality & Safety 2011;20(4):297. doi: 10.1136/bmjqs.2010.043109
[Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjIwLzQvMjk3IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjQvMDkvMjUvMjAyNC4wNS4wNy4yNDMwNzAwMS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=)
84. 84.Saleh M, Anthony D, Parboteeah S. The impact of pressure ulcer risk assessment on patient outcomes among hospitalised patients. J Clin Nurs 2009;18(13):1923–9. doi: 10.1111/j.1365-2702.2008.02717.x [published Online First: 2009/04/03]
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1365-2702.2008.02717.x&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=19374691&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
85. 85.Moore Z, Johansen E, Etten Mv, et al. Pressure ulcer prevalence and prevention practices: a cross- sectional comparative survey in Norway and Ireland. Journal of Wound Care 2015;24(8):333–39. doi: 10.12968/jowc.2015.24.8.333
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.12968/jowc.2015.24.8.333&link_type=DOI)
86. 86.Moore Z, Pitman S. Towards establishing a pressure sore prevention and management policy in an acute hospital setting. The All Ireland Journal of Nursing and Midwifery 2000;1(1):7–11.
87. 87.Gunningberg L, Lindholm C, Carlsson M, et al. Implementation of risk assessment and classification of pressure ulcers as quality indicators for patients with hip fractures. J Clin Nurs 1999;8(4):396–406. doi: 10.1046/j.1365-2702.1999.00287.x
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1046/j.1365-2702.1999.00287.x&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=10624256&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
[Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000082096100010&link_type=ISI)
88. 88.Bale S, Finlay I, Harding KG. Pressure sore prevention in a hospice. J Wound Care 1995;4(10):465–8. doi: 10.12968/jowc.1995.4.10.465
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.12968/jowc.1995.4.10.465&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=8548573&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
89. 89.Hodge J, Mounter J, Gardner G, et al. Clinical trial of the Norton Scale in acute care settings. Aust J Adv Nurs 1990;8(1):39–46.
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=2091682&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
90. 90.Moher D, Liberati A, Tetzlaff J, et al. Preferred Reporting Items for Systematic Reviews and Meta- Analyses: The PRISMA Statement. PLOS Medicine 2009;6(7):e1000097. doi: 10.1371/journal.pmed.1000097
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pmed.1000097&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=19621072&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
91. 91.Steyerberg EW, Harrell FE, Jr.. Prediction models need appropriate internal, internal-external, and external validation. J Clin Epidemiol 2016;69:245–7. doi: 10.1016/j.jclinepi.2015.04.005 [published Online First: 2015/04/18]
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jclinepi.2015.04.005&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=25981519&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
92. 92.Bossuyt PM, Reitsma JB, Bruns DE, et al. The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Ann Intern Med 2003;138(1):W1–12. doi: 10.7326/0003-4819-138-1-200301070-00012-w1
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7326/0003-4819-138-1-200301070-00012-w1&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=12513067&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
93. 93.Reitsma J, Rutjes A, Whiting P, et al. Chapter 8: Assessing risk of bias and applicability. In: Deeks J, Bossuyt P, Leeflang M, et al., eds. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy. Version 2.0 (updated July 2023) ed. Cochrane, 2023.
94. 94.Deeks JJ, Dealey C. Pressure sore prevention: using and evaluating risk assessment tools. British Journal of Nursing 1996;5(5):313–20. doi: 10.12968/bjon.1996.5.5.313
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.12968/bjon.1996.5.5.313&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=8715749&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
95. 95.Vickers AJ, Van Calster B, Steyerberg EW. Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests. BMJ 2016;352:i6. doi: 10.1136/bmj.i6
[FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE0OiIzNTIvamFuMjVfMi9pNiI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDI0LzA5LzI1LzIwMjQuMDUuMDcuMjQzMDcwMDEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9)
96. 96.Trikalinos TA, Siebert U, Lau J. Decision-Analytic Modeling to Evaluate Benefits and Harms of Medical Tests: Uses and Limitations. Medical Decision Making 2009;29(5):E22–E29. doi: 10.1177/0272989X09345022
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/0272989X09345022&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=19734441&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F09%2F25%2F2024.05.07.24307001.atom)
97. 97.Dweekat OY, Lam SS, McGrath L. Machine Learning Techniques, Applications, and Potential Future Opportunities in Pressure Injuries (Bedsores) Management: A Systematic Review. International journal of environmental research and public health 2023;20(1) doi: 10.3390/ijerph20010796
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3390/ijerph20010796&link_type=DOI)
98. 98.Berlowitz DR, VanDeusen Lukas C, Parker V, et al. 3F: Care Plan. Preventing pressure ulcers in hospitals: A toolkit for improving quality of care: Agency for Healthcare Research and Quality, 2014:140–42.
99. 99.Hingorani AD, Windt DAvd, Riley RD, et al. Prognosis research strategy (PROGRESS) 4: Stratified medicine research. BMJ : British Medical Journal 2013;346:e5793. doi: 10.1136/bmj.e5793
[FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNDYvZmViMDVfMS9lNTc5MyI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDI0LzA5LzI1LzIwMjQuMDUuMDcuMjQzMDcwMDEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9)
100.100.Moons KG, Kengne AP, Grobbee DE, et al. Risk prediction models: II. External validation, model updating, and impact assessment. Heart 2012;98(9):691–8. doi: 10.1136/heartjnl-2011-301247 [published Online First: 2012/03/07]
[Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiaGVhcnRqbmwiO3M6NToicmVzaWQiO3M6ODoiOTgvOS82OTEiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyNC8wOS8yNS8yMDI0LjA1LjA3LjI0MzA3MDAxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==)
101.101.Shi C, Dumville JC, Cullum N. Evaluating the development and validation of empirically-derived prognostic models for pressure ulcer risk assessment: A systematic review. International journal of nursing studies 2019;89:88–103. doi: 10.1016/j.ijnurstu.2018.08.005
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.ijnurstu.2018.08.005&link_type=DOI)
102.102.Moons KG, Altman DG, Reitsma JB, et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med 2015;162(1):W1–73. doi: 10.7326/m14-0698
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7326/m14-0698&link_type=DOI)
103.103.Lee J, Mulder F, Leeflang M, et al. QUAPAS: An Adaptation of the QUADAS-2 Tool to Assess Prognostic Accuracy Studies. Annals of Internal Medicine 2022;175(7):1010–18. doi: 10.7326/m22-0276
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7326/m22-0276&link_type=DOI)