A prediction algorithm to improve the accuracy of the Gold Standard Framework Surprise Question end-of-life prognostic categories in an acute hospital admission cohort-controlled study. The Proactive Risk-Based and Data-Driven Assessment of Patients at the End of Life (PRADA) ==================================================================================================================================================================================================================================================================================== * Baldev Singh * Nisha Kumari-Dewat * Adam Ryder * Vijay Klaire * Gemma Bennion * Hannah Jennens * Dawn Matthews * Sophie Rayner * Benoit Ritzenthaler * Jean Shears * Kamran Ahmed * Mona Sidhu * Ananth Viswanath * Kate Warren * Emma Parry ## Abstract **Objective** To determine the accuracy of a clinical data algorithm allocated end-of-life prognosis amongst hospital inpatients. **Method** The model allocated a predicted Gold Standard Framework end-of-life prognosis to all acute medical patients admitted over a 2-year period. Mortality was determined at 1 year. **Results** Of 18,838 patients, end-of-life prognosis was unknown in 67.9%. A binary logistic regression model calculated 1-year mortality probability (*X2*=6650.2, p<0.001, r2 = 0.43). Probability cut off points were used to triage those with unknown prognosis using the GSF Surprise Question “Yes” or “No” survival categories (> or < 1 year respectively), with subsidiary classification of “No” to Green (months), Amber (weeks) or Red (days). This digitally driven prognosis allocation (100% vs baseline 32.1%) yielded cohorts of GSFSQ-Yes 15,264 (81%), GSFSQ-No Green 1,771 (9.4%) and GSFSQ-No Amber or Red 1,803 (9.6%). There were 5,043 (26.8%) deaths at 1 year. In Cox’s survival, model allocated cohorts were discrete for mortality (GSFSQ-Yes 16.4% v GSFSQ-No 71.0% (p<0.001). For the GSFSQ-No classification, the mortality Odds Ratio was 12.4 (11.4 – 13.5) (p<0.001) vs GSFSQ-Yes (c-statistic of 0.71 (0.70 – 0.73), p<0.001; accuracy, positive and negative predictive values of 81.2%, 83.6%, 83.6% respectively. If this tool had been utilised at the time of admission, the potential to reduce subsequent hospital admissions, death-in-hospital, and bed days was all p<0.001. **Conclusions** The defined model successfully allocated end-of-life prognosis in cohorts of hospitalised patients with strong performance metrics for prospective 1 year mortality, yielding the potential to provide anticipatory care and improve outcomes. **What is already known about this topic?**End-of-life care is fragmented with excessive hospital admission and death in hospital. Current processes to determine end-of-life prognosis and promote anticipatory care for better outcomes are of limited utility. **What this paper adds?**A patient centric data integration model permitted the development of a digital health care system (PRADA) which allows the use of advanced analytics to accurately determine end-of-life prognosis among those where it was otherwise unknown. This paper demonstrates the potential benefit of integrating this prediction tool into routine care, at scale, in large population-level cohorts. **Implications for practice, theory, or policy** In an era of advancing opportunity from informatics driven heath care, NHS policy, through commissioning to direct care, must now actively deploy such evidence-based digital systems into direct care, most specifically in data sharing across provider boundaries. We particularly hope the research community might consider testing and validating this approach. Key Words * Advance Care Planning (D032722) * Algorithms (D000465) * Clinical Decision Rules (D000081415) * Decision Making (D003657) * Emergency care (D004632) * Health informatics (D008490) * Hospitals (D006761) * Mortality (D009026) * Palliative care (D010166) * Precision Medicine (D057285) ## Introduction The NHS is under increasing pressure as the population ages and their needs become more complex, fragmented, and poorly integrated between service providers (1). End-of-life (EOL) is interposed by escalating rates of non-elective admissions (NEA) and death in hospital (DIH) (2–5). Guidance for better EOL care targets the reduction of NEA and DIH (6–14) from which there are emergent models of care (15). Hospital Specialist Palliative Care Teams (SPCT) improve EOL outcomes (16–17), but they may not be supported by effective prediction engines and clinical prognosis protocols (18,19). Specifically, the commended GSF Surprise Question (GSFSQ), establishing EOL prognosis, has shown variable performance (20,21). Current mortality prediction tools are not always automated and have uncertain performance characteristics (22–29), though the utilisation of modern-day informatics to support care provision through a population-based approach with early identification and systemised care is inherent to NHS digital health plans (30,31). In our local health economy, we have integrated data from acute hospital, community and primary care sources and deployed it into direct care to promote early identification for anticipatory care, captured required process for structured care, developed linked clinical systems, and ascertained prospective events to determine effectiveness. We have termed the specific EOL digital care pathway; The Proactive Risk-Based and Data-Driven Assessment of Patients at the End of Life (PRADA). Our preliminary evidence has shown the potential benefit (32). Now we report a prediction model in which we have not attempted to use data prediction to usurp clinical judgment but rather to provide enhanced intelligence in support of targeted direct care and improved outcomes. ## Methods ### The PRADA systems PRADA is an informatics driven digital health care system. It holds data, under full GDP compliance, of all patients registered to a Wolverhampton General Practice (GP), those resident in Wolverhampton but not registered to a local GP, and any who have had non-elective contact with our local hospital. Specific to the PRADA digital care pathway: risk escalators are applied to stratify the whole population followed by clinical stratification using the GSF Surprise Question which is captured at the point of care in hospital, community, and primary care settings. It ascertains the key EOL processes of EPaCCS, ReSPECT, EOL care plans, and EOL registration and key events including death, place of death, and non-elective urgent care. It has structured clinical governance functions and incorporates digital robotic processes in algorithms for clinical decision support such as highlighting individuals who may need to be on the EOL register. The PRADA clinical module is the deployment of that informatics resource to direct clinical care. It is accessed either through the electronic patient record (EPR) or as a standalone function. The system displays lists of patients in the specific arena of care that have been risk stratified as potentially requiring assessment for EOL care, so promoting proactive, anticipatory, and systematic care within non-referral pathways. It captures clinician data at the point of care, creates an EPaCC and a GSF prognosis specific EOL care plan, and automatically distributes finalised documentation electronically into the hospital EPR and to the GP patient record. At all times, the clinical module is linked to the core data system which then remains continuously up to date. ### Study design and setting This is an acute hospital cohort-controlled study. Over 2-years, all who were admitted acutely as a non-elective admission (NEA) was ascertained. To assure a tighter case mix, only patients from the medical block of wards were included. The index admission (IA) was either the first admission in that period, or the one in which a PRADA assessment was first made, thereby excluding all recurrent or multiple admissions in any given individual, whilst yielding a time anchor for retrospective and prospective events. ### Data The Wolverhampton Integrated Clinical Data Set links primary care, hospital, and community services data under GDPR. Demographic variables included age, gender, ethnicity, and the Index of Multiple Deprivation (IMD). Ethnicity data from all sources were recoded into White, South-Asian, Black, Mixed Ethnicity, Chinese or Unknown. The comorbidities utilized were the 16 commonest long-term conditions in the population. The variables chosen for digital risk stratification were based on common variables used in other risk prediction tools and assessed in our preliminary work to be linked to emergency activity and mortality. They represented urgent care and care complexity: >=3 Accident and Emergency (A&E) admissions over the prior 12 months (defined as emergency department attendances not leading to a hospital admission); >= 3 non-elective admissions (NEA) over the previous 12 months; >=3 co-morbidities, nursing home residency. The Rockwood score was used to assess frailty, or in its absence the Electronic Frailty Index, both categorised to moderate and severe frailty, whilst any missing vales coded to non-frail. Markers of end-of-life included end-of-life registration (EOL) or any recorded EOL process measure (EPaCC, ReSPECT or EOL care plans). The outcomes of the GSF Surprise question “Would you be surprised if this patient died within the next year?” was captured in the EPR together with the prognostic subcategories of “No” being Green (months), Amber (weeks) and Red (days). Any Specialist Palliative Care Team (SPCT) contact in the IA or in prior admissions was identified from routine inpatient coding. Mortality was determined from hospital mortality statistics and rolling NHS Strategic Tracing Service checks. ### Outcomes Prospectively, death up to 1 year, and up to 90 days death in hospital, NEA and the number of hospital bed days occupied. ### Statistical method All data were analysed on IBM SPPS version 29. When comparing independent groups, the Student’s t-test and the Chi-square test were used for the difference between means and proportions, respectively. Comparison of multiple group means was undertaken by one way analysis of variance (ANOVA). Variable taxonomy and collinearity were considered in principal components analysis (PCA). The association of independent factors with outcomes was achieved using binary logistic regression, presented with Odds Ratios (OR) and their 95% confidence intervals. Survival analysis was by Cox’s regression. Results are presented as the mean ± SD or as numbers with percentages. Statistical significance of all tests applied was taken at p<0.05. ### Research checklist Strobe ### Ethical Considerations The study did not involve randomisation or any intervention that ought not to have otherwise happened and so ethical approval was not deemed necessary, and this was confirmed by our authorised local governance group. ### Patient consent for publication Not applicable. ### Patient and Public Involvement None. ## Results ### Baseline characteristics There were 142,159 admissions in those aged >=18 years old, with 91,244 individual single hospital spells, of which 18,838 were medical cases. Of those, 1,826 (9.7%) died in the index admission in hospital, 17,012 (90.3%) were discharged alive but 3,217 (17.1%) then died within 1 year of that discharge, totalling 5,043 (26.8%) deaths. Clinical characteristics are shown in **Table 1**. Those who died were older, more likely of white ethnic background, less deprived, had more A/E attendances and NEAs in the preceding year, were more co-morbid, more likely to be a nursing home resident, though they were equally frail, and were more likely on the end-of-life register and an end-of-life process. However, many of the differences were marginal and only 2 factors had > 50% association with death: >=3 NEA and EOL registration (53.9% and 65.5% respectively). The outcome of GSFSQ overwhelmingly shows the high prevalence with which it was not ascertained. Subsidiarily, there was a lower prevalence of GSFSQ-Yes and a higher prevalence of GSFSQ-No in those who died. **Table 1** also shows the Green (days), Amber (weeks) and Red (days) sub-prognostic classifications of the GSFSQ-No category. Among those with a GSFSQ-No classification or a hospital SPCT contact, 73.4% and 82.1% respectively died within 1 year. The SPCT saw 894 (65.7%) of patients with GSFSQ-No status (n=1,361). View this table: [Table 1](http://medrxiv.org/content/early/2023/09/08/2023.09.07.23295196/T1) Table 1 The characteristics of the cohorts. Data are the mean ± SD (range) of numbers with percentage. Abbreviations are as defined in the text. ### Evaluation of variables The variables listed in Table 1 were entered into a principal component analysis and redundant variables removed. At an Eigen value of >1, 4 distinct domains emerged explaining 41% of the data variance. These represented: clinical complexity (>=3 co morbidities, frailty, age, and nursing home residence); EOL (GSFSQ, SPCT contact, EOL process marker, EOL registration); Urgent care (>=3 NEA, >=3 AE attendances); and the residual demographics of IMD and ethnicity. Gender did not associate with any domain whilst age sat within the clinical complexity component. These variables were utilised in constructing a mortality prediction model. ### A regression model for mortality prediction It is important to note that clinical judgment was used in the prediction model (any prior known recorded GSFSQ outcome) as well as being seen by the SPCT. The variables were entered into binary logistic regression with 1-year vital status as the dependent variable. The model was significant (*X2*=6650.2, p<0.001, r2 = 0.43), with a 46.6% prediction of those who died. It generated a predictive probability for death at 1 year which was ranked into quintiles. The 5th quintile (predicted probability 0.7 ±0.17 (0.50 – 1.0)) stood out by having a 70.9% mortality rate capturing 53.0% of all deaths and in its association with the GSFSQ category (GSFSQ Unknown, Yes, No of 0.29±0.25, 0.18±0.18, 0.81±0.20 respectively (F=4090.3, p<0.001)). ### Triaging the GSF SQ The 5th quintile probability prediction was used to reclassify the GSFSQ. **Figure 1** shows a flow schematic of that process. ![Figure 1](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2023/09/08/2023.09.07.23295196/F1.medium.gif) [Figure 1](http://medrxiv.org/content/early/2023/09/08/2023.09.07.23295196/F1) Figure 1 A flow diagram of the adjustments to the GSF SQ prognostic allocation made using the model’s probability estimates. ### Triage adjustment 1 Using the 5th quintile category, in the first adjustment, GSFSQ-Unknown was reclassified to GSFSQ-No, reallocating 2,213 (17.3%) of the 12,799 with an unknown prognostic classification. The adjusted GSFSQ-No then captured 2,536 (50.3%) of those that died vs 999 (19.8%) prior to adjustment, meaning an additional 1,537 deaths (30.5% of all deaths). Reallocating the known GSFSQ-Yes group by 5th quintile model predicted probability was of scant numerical value and was not incorporated. ![Figure 2](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2023/09/08/2023.09.07.23295196/F2.medium.gif) [Figure 2](http://medrxiv.org/content/early/2023/09/08/2023.09.07.23295196/F2) Figure 2 ROC curves assessing the GSFSQ-No category probability of death at 1 year either unadjusted (top) or adjusted by the described prediction model. ### Triage adjustment 2 The subsidiary prognosis for the GSFSQ-No was then further considered. In ANOVA (F=435.5, p<0.001), for the 5 baseline GSFSQ prognostic categories, GSFSQ-No Amber vs GSFSQ-No Red did not differ in probability scores in post hoc analysis, and they were merged as Amber-Red. The model probabilities for GSFSQ-N Green (0.76±0.21) and GSFSQ-No Amber-Red (0.86 ±0.18) were significantly different (p<0.001) with 95% CI for their means of 0.74 – 0.78 and 0.85 – 0.87, respectively. All reallocated GSFSQ-Unknows and GSFSQ-No Green were reclassified to Green or Amber-Red at a predicted mortality probability cut point of 0.85, the lower CI for the baseline GSFSQ-No Amber Red probability mean. As shown in **Table 2**, ultimately, predominately with re-classification of GSFSQ-Unknown, substantially more deaths were then captured by the finally adjusted GSFSQ-No classification (2,536 vs 999), with an increase in adjusted Green prognosis (988 vs 308) and a marked increase in adjusted Amber-Red (1,548 vs 691). Cox’s regression survival curves are shown in **Figure 3**, demonstrating that the residual post adjusted GSFSQ-Unknown group then approximates very closely to the original GSFSQ-Yes, with preservation of the distinction between Green and Amber-Red. ![Figure 3](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2023/09/08/2023.09.07.23295196/F3.medium.gif) [Figure 3](http://medrxiv.org/content/early/2023/09/08/2023.09.07.23295196/F3) Figure 3 Survival curves for the GSFSQ categories in the unadjusted and adjusted model. View this table: [Table 2](http://medrxiv.org/content/early/2023/09/08/2023.09.07.23295196/T2) Table 2 Model metrics with and without predicted probability adjustment. Odds ratios were determined in binary logistic regression and performance metrics were derived from ROC analysis. ### Triage adjustment 3 Whilst the final adjusted GSFSQ-Yes still retained negative predictive value compared to final residual GSFSQ-Unknown (OR 0.88 (0.80 - 0.97), p<0.01), the 2 could be merged, so that the final outcomes were essentially GSFSQ-Yes (15264 (81.0%)), GSFSQ-No Green (1771 (9.4%)) and GSFSQ-No Amber-Red (1803 (9.6%)) encapsulating 2,507 (49.7%), 988 (19.6%) and 1,548 (30.7%) of deaths, respectively. ### Model performance metrics These are shown in **Table 2** with ROC curves in **Figure 2**. The model adjusted GSFSQ No captured 50.3% of deaths, its OR for predicting death was stronger, the ROC statistic improved, as did accuracy and positive and negative predictive values. ### Workload considerations The purpose of this analytical strategy was not to predict mortality but drive accurate early identification of those at risk of dying, targeting them for systematically delivered assessment and interventions that might lead to benefit in better outcomes. Regarding workload, ***Table 3*** shows the association between GSFSQ status and palliative care team actual and potential input requirements. Regrading of the GSFSQ categories within the adjusted model, this demonstrates a truer reflection of the actual appropriateness of SPCT contact, which was focused in the highest risk categories and progressively less associated with the inaccurate baseline GSFSQ-Unknown status. As greater numbers are identified, the table shows how SPCT workload might change if the SPCT team then saw various categories of patients, including estimates of daily new caseload. In the adjusted model, seeing all identified GSFSQ-No patients would increase workload by 52%, restricting to only seeing those with GSFSQ-No Amber-Red status reduced it by 24% whilst seeing all GSFSQ-No Amber-Red additional to current caseload, increased it by only 10%. ### Estimations of benefit Estimating benefit in hard outcomes is difficult to determine and the analysis is hypothetical. Using the described recently deployed PRADA systematic care planning tool (32), where numbers remain small in the context of this 2-year cohort, we selected those discharged alive and in the highest propensity adjusted GSFSQ-No Amber-Red category. The associations with 90-day post discharge events are shown in **Table 3**. There were 1,102 such cases of whom 123 (11.2%) had a PRADA digitally supported EOL care planning contact with the SPCT. The PRADA contact group had a higher mortality rate but nevertheless had fewer hospital admissions, occupied less hospital bed days and were less likely to die in hospital (all p<0.001). The potential emergency bed day saving over 90-days was 3.53 days per non-PRADA contact patient. This equates to a total of £1,727,935 (979 x 3.53 x £500) or £1, 765 per individual. The combined adverse hard outcome event of 90-day NEA or hospital death was also significantly lower in the PRADA group (p<0.001). For PRADA contact the relative risk of this summative adverse event was 0.51 (0.35 – 0.73) (p<0.001) and the NNT for that benefit were 5.2 (3.5-9.7). View this table: [Table 3](http://medrxiv.org/content/early/2023/09/08/2023.09.07.23295196/T3) Table 3 The association between current and modelled GSFSQ status and the specialist palliative care team (SPCT) current and potential activity, and an estimate of potential benefit of contact when seen using the PRADA system. ## Discussion ### Main findings The model’s algorithm allowed end-of-life prognosis allocation, in line with the GSF SQ, amongst those who had not already been given a prognosis by clinicians, characterising the whole cohort, in manner that had strong predictive accuracy for 1 year mortality outcomes. Running this algorithm in live clinical care has the potential for anticipatory care by a SPCT to modify key outcomes leading to reduced rates of non-elective readmission, bed occupancy, and in hospital mortality. ### What this study adds The study defines how to take whole health economy clinical data and curate a limited data set to run an automated predictive model. The methodology for probability cut off points that characterise the intended prognostic groups are outlined, and the accuracy of their association with various relevant endpoints, most especially 1 year mortality, is demonstrated. ### Strengths and Weaknesses The study’s strengths are the use of said integrated clinical data set, the construction of a digital heath care pathway specific to the end-of-life, the use of it to run advanced analytics and the description of that process in addition to the ascertainment of the performance metrics of its association with 1 year mortality. The findings are likely generalisable to other such care settings given the large size of the cohort of a general medical case mix, not limited to small numbers, nor in niche clinical scenarios. Its weakness is that the predicted end-point benefits were theoretical and now require evidencing in real time care, although we have published our positive preliminary outcomes (32). Furthermore, the integrated clinical data between acute, community and primary care provision is not readily available in most health economies. Finally, as a first report, the findings require validation, in various care settings care, by independent research groups. The study relates to those with a hospital admission, and it is not yet known if the algorithm might apply to the non-admitted population. ### General discussion We describe the model not as predicting mortality but predicting a possible clinical decision in a form equivalent to the GSFSQ. This may be a moot point, they may be seen as the same thing, but our prediction methodology is intended to support clinical decision making and not to undermine or replace it in any way and it would be ethically very unsound to assume that the model predicated GSFSQ is the actual prognosis or for any care process to run automatically on that basis. Thus, in our model, established clinical judgment remains the prime decision, and the predicted output is simply an effective risk-stratification tool to drive further assessment by clinical teams for validation of the proposed digital allocation and for the planning of effective and appropriate care, ensuring it to be matched to the patient’s clinical needs, consent and wishes, which cannot be known to an automated prediction engine. Much is made of the intuitive, ad-hoc, “gut feeling” of global clinical judgment (GCJ) (33–36), but here we incorporate it into probability model in line with a Bayesian approach to clinical decision making (37). We have recently published the positive power of the use of GCJ in a similar prediction engine (38). In end-of-life care there is already an established format for capturing GCJ, namely the GSF SQ although it has been criticized in several ways (20,21). However, our view is that: the overwhelming problem resides with the question not being asked; when it is asked its performance should not be judged as a discreet variable (as it has been hitherto) which is bound to be poor, just as with each other single independent variable (e.g. comorbidity); it is strengthened by incorporating it alongside other key relevant variables; it is statistically a valuable parameter as determined by principal component analysis. Furthermore, the focus should not always be on predicting who might die but clarifying who is unlikely to, by valuing its negative predictive value, a notion that resonate with clinicians, who may find it much easier to rule something out as opposed to ruling it in as time bound event as crucial as death (39,40). Thus, the final GSFSQ-Yes category excluded 81% of the cohort who had a much lower 1-year mortality of 16.4% and risk escalated 19% to the GSF-No category with a 70.1% mortality rate and achieving this level of risk stratification or “prediction” was the intended prime outcome. It maybe, but unknown to us at this stage, that further clinical validation, assessments and events would have again modified the prediction probabilities, emphasising that this is intended to be a live and continuous process, not a single time-point cross-sectional snapshot, and so probability adjustment can become dynamic with trends that might be informative. Crucially, the models’ prognostic allocation is integrated in a digital care pathway and is used to drive anticipatory care by automated referral to the SPCT. The system will know if that SPCT had contact with the identified patients, whether required processes of care were undertaken, whether onwards care allocation occurred appropriately with effective communication to the relevant providers of care, especially community and primary care services. Prospective monitoring allows the ascertainment of the defined outcomes as well as an understanding end-of-life registration status, perhaps the most crucial event in the pathway. Thus, the next step of our work is to integrate the now defined analytical process into live routine care and to determine its impact across the whole PRADA digital pathway construct from identification to registration and the prevention of the adverse outcomes of escalating non-elective care and death not in the preferred place of death. Briefly, PRADA is distinctly different from many other mortality prediction tools (22–29). They share an aspiration for early identification and anticipatory care. They may utilise the GSFSQ as a preliminary identification step. They rely on a variety of individual functional assessments and specific clinical categories or markers of end organ failure and these generally require summation by an individual clinician case by case. Accordingly, contrary to their ambition, they may be deployed reactively late in this trajectory and not proactively. Though they may have viable performance metrics, these have generally been assessed in small and specific groups of patients. Their algorithms are not generally automated into clinical systems and thus are not easily made systematic, uniform, deployable at scale or amenable to continuous quality improvement. However, PRADA methodology, whilst differing in the data sets utilised, shares the common principles of digitised risk stratification, the capturing of validating clinical assessment, prognostic coding, care planning and registration with the innovative NHS EARLY toolkit which can run in a similarly automated manner, directly in primary care systems to highlight specific patient in these end-of-life cohorts (29). In summary, as part of a wider piece of work creating an end-of-life digital care pathway, PRADA, we have developed an informatics-based algorithm for prognosis prediction that maps to the GSFSQ categories and is validated to have performance accuracy against 1 year mortality. It can be run at scale in large cohorts of patients, in this case those who have had hospital admission. The next phase of our work is to demonstrate this can be automated live into direct care in our electronic clinical systems and demonstrate the hoped for consequent beneficial outcomes are achieved. Our work and methodology will have resonance not only in the developing UK NHS digital heath care ambition, but in many other health economies across the world faced with exactly similar imperatives. We would implore colleagues in the research community to consider and test our findings. ## Data Availability All data produced in the present study are available upon reasonable request to the authors ## Author contributions Principal author BMS; Authors EP, NK, AR, AV; Data systems VK; Statistical Analysis BMS; Manuscript Development All; PRADA system development and pilot AV, DM, GB, HJ, JS, KA MS, SR. ## Conflicts of Interest or Competing Interest None ## Funding None ## Data Sharing The corresponding author will share anonymised study data upon reasonable request. ## Copyright The corresponding author has the right to grant on behalf of all authors, the license to permit this article to be published. ## Footnotes * baldev.singh{at}nhs.net, nisha.kumari{at}nhs.net, a.ryder1{at}nhs.net, vijay.klaire{at}nhs.net, g.bennion{at}nhs.net, hannah.jennens{at}nhs.net, d.matthews10{at}nhs.net, sophie.rayner1{at}nhs.net, britzenthaler{at}nhs.net, jean.shears{at}nhs.net, kamahmed{at}nhs.net, mona.sidhu{at}nhs.net, ananthviswanath{at}nhs.net, kate.warren{at}nhs.net, e.parry{at}keele.ac.uk ## Abbreviations Used CI : confidence interval DIH : death in hospital EOL : end of life EPaCCS : Electronic Palliative Care Co-ordination Systems EPR : electronic patient record GDPR : General Data Protection Regulation GSF SQ : Gold Standard Framework Surprise Question IA : index admission IMD : Index of Multiple Deprivation. NEA : non-elective admission NHS : National Health Service OR : Odd’s Ratio PCA : Principal components analysis PRADA : Proactive Risk-Based and Data-Driven Assessment of Patients at the End of Life RCT : randomised controlled trial ReSPECT : Recommended Summary Plan for Emergency Care and Treatment ROC : receiver operating curve SD : standard deviation SPCT : specialist palliative care team * Received September 7, 2023. * Revision received September 7, 2023. * Accepted September 8, 2023. * © 2023, Posted by Cold Spring Harbor Laboratory This pre-print is available under a Creative Commons License (Attribution-NoDerivs 4.0 International), CC BY-ND 4.0, as described at [http://creativecommons.org/licenses/by-nd/4.0/](http://creativecommons.org/licenses/by-nd/4.0/) ## References 1. 1.NHS England. The NHS Long Term Plan. [https://www.longtermplan.nhs.uk/wp-content/uploads/2019/08/nhs-long-term-plan-version-1.2.pdf](https://www.longtermplan.nhs.uk/wp-content/uploads/2019/08/nhs-long-term-plan-version-1.2.pdf). Accessed 05.09.2023. 2. 2.Smith SM, Soubhi H, Fortin M, Hudon C, O’Dowd T. Managing patients with multimorbidity: systematic review of interventions in primary care and community settings BMJ 2012:345; e5205. doi:10.1136/bmj.e5205 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNDUvc2VwMDNfMS9lNTIwNSI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIzLzA5LzA4LzIwMjMuMDkuMDcuMjMyOTUxOTYuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 3. 3.Moore E, Munoz-Arroyo R, Schofield L, Radley A, Clark D, Isles C. Death within 1 year among emergency medical admissions to Scottish hospitals: incident cohort study. BMJ Open, Palliative Care Research 2017. doi:10.1136/bmjopen-2017-021432 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMToiOC82L2UwMjE0MzIiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMy8wOS8wOC8yMDIzLjA5LjA3LjIzMjk1MTk2LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 4. 4.Public Health England. Emergency admissions in the 3 months before death. 2020. [https://www.gov.uk/government/publications/emergency-admissions-in-the-3-months-before-death/emergency-admissions-in-the-3-months-before-death](https://www.gov.uk/government/publications/emergency-admissions-in-the-3-months-before-death/emergency-admissions-in-the-3-months-before-death). Accessed 05.09.2023. 5. 5.Smith SM, Wallace E, O’Dowd T, Fortin M. Interventions for improving outcomes in patients with multimorbidity in primary care and community settings. Cochrane Database Syst Rev. 2021: 1; CD006560. doi:10.1002/14651858.CD006560 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/14651858.CD006560.pub4&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F09%2F08%2F2023.09.07.23295196.atom) 6. 6.Murray S, Economist Intelligence Unit. The 2015 Quality of Death Index: ranking palliative care across the world. [https://impact.economist.com/perspectives/health/2015-quality-death-index](https://impact.economist.com/perspectives/health/2015-quality-death-index). Accessed 05.09.2023. 7. 7.Public Health England. Cost-effective commissioning of end-of-life care understanding the health economics of palliative and end of life care. [https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment\_data/file/612377/health-economics-palliative-end-of-life-care.pdf](https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/612377/health-economics-palliative-end-of-life-care.pdf). Accessed 05.09.2023. 8. 8.GSF National Gold Standards Framework (GSF) Centre in End of Life Care. [https://www.goldstandardsframework.org.uk/evidence](https://www.goldstandardsframework.org.uk/evidence). Accessed 05.09.2023. 9. 9.NICE guideline NG142. Evidence reviews: Barriers to accessing end of life care services. 2019. [https://www.nice.org.uk/guidance/ng142/evidence/c-barriers-to-accessing-end-of-life-care-services-pdf-6955526992](https://www.nice.org.uk/guidance/ng142/evidence/c-barriers-to-accessing-end-of-life-care-services-pdf-6955526992). Accessed 05.09.2023. 10. 10.NICE guideline NG142. Evidence reviews: What are the best service models to support the identification of people who may be entering the last year of life? 2019. [https://www.nice.org.uk/guidance/ng142/evidence/a-identifying-people-who-may-be-entering-the-last-year-of-life-pdf-6955526990](https://www.nice.org.uk/guidance/ng142/evidence/a-identifying-people-who-may-be-entering-the-last-year-of-life-pdf-6955526990). Accessed 05.09.2023. 11. 11.NICE. End of life care for adults: service delivery. 2019. [https://www.nice.org.uk/guidance/ng142/resources/end-of-life-care-for-adults-service-delivery-pdf-66141776457925](https://www.nice.org.uk/guidance/ng142/resources/end-of-life-care-for-adults-service-delivery-pdf-66141776457925). Accessed 05.09.2023. 12. 12.National Palliative and End of Life Care Partnership. Ambitions for Palliative and End of Life Care: A national framework for local action 2021-2026. 2021. [https://www.england.nhs.uk/wp-content/uploads/2022/02/ambitions-for-palliative-and-end-of-life-care-2nd-edition.pdf](https://www.england.nhs.uk/wp-content/uploads/2022/02/ambitions-for-palliative-and-end-of-life-care-2nd-edition.pdf). Accessed 05.09.2023. 13. 13.NHS England. Palliative and End of Life Care Statutory Guidance for Integrated Care Boards (ICBs). 2022. [https://www.england.nhs.uk/wp-content/uploads/2022/07/Palliative-and-End-of-Life-Care-Statutory-Guidance-for-Integrated-Care-Boards-ICBs-September-2022.pdf](https://www.england.nhs.uk/wp-content/uploads/2022/07/Palliative-and-End-of-Life-Care-Statutory-Guidance-for-Integrated-Care-Boards-ICBs-September-2022.pdf). Accessed 05.09.2023. 14. 14.WHO. Health and social care near the end of life: can policies reduce costs and improve outcomes? 2021. [https://eurohealthobservatory.who.int/publications/i/health-and-social-care-near-the-end-of-life-can-policies-reduce-costs-and-improve-outcomes](https://eurohealthobservatory.who.int/publications/i/health-and-social-care-near-the-end-of-life-can-policies-reduce-costs-and-improve-outcomes). Accessed 05.09.2023. 15. 15.Ohana S, Shaulov A, DeKeyser Ganz F. Acute palliative care models: scoping review. BMJ Supportive & Palliative Care Published Online First: 17 August 2023. doi: 10.1136/spcare-2022-004124 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiYm1qc3BjYXJlIjtzOjU6InJlc2lkIjtzOjIwOiJzcGNhcmUtMjAyMi0wMDQxMjR2MSI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIzLzA5LzA4LzIwMjMuMDkuMDcuMjMyOTUxOTYuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 16. 16.Higginson IJ, Finlay I, Goodwin DM, Cook AM, Hood K, Edwards AGK, Douglas JH, Norman CE. Do Hospital-Based Palliative Teams Improve Care for Patients or Families at the End of Life? J Pain and Symp Manag 2002: 23; 96–106. doi:10.1016/S0885-3924(01)00406-7 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0885-3924(01)00406-7&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=11844629&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F09%2F08%2F2023.09.07.23295196.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000173836200005&link_type=ISI) 17. 17.Bajwah S, Oluyase AO, Yi D, Gao W, Evans CJ, Grande G, Todd C, Costantini M, Murtagh FE, Higginson IJ. The effectiveness and cost-effectiveness of hospital-based specialist palliative care for adults with advanced illness and their caregivers. Cochrane Database of Systematic reviews. 2020. doi:10.1002/14651858.CD012780.pub2 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/14651858.CD012780.pub2&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F09%2F08%2F2023.09.07.23295196.atom) 18. 15.Shaver N, Lapenskie, Isenberg SR, Saunders S, Hsu AT, Tanuseputro P1,2, Does inpatient palliative care consultation impact outcomes following hospital discharge? A narrative systematic review. Palliative Medicine 2020: 34; 5–15 doi:10.1177/0269216319870649 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/0269216319870649&link_type=DOI) 19. 16.Petrova M, Wong G, Kuhn I, Wellwood I, Barclay S. Timely community palliative and end-of-life care: a realist synthesis. BMJ Supportive & Palliative Care 2021: ; 1–15 doi:10.1136/bmjspcare-2021-003066 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiYm1qc3BjYXJlIjtzOjU6InJlc2lkIjtzOjIzOiJibWpzcGNhcmUtMjAyMS0wMDMwNjZ2MyI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIzLzA5LzA4LzIwMjMuMDkuMDcuMjMyOTUxOTYuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 20. 18.White, et al. How accurate is the ‘surprise question’ at identifying patients at the end of life? a systematic review and meta-analysis. BMC Medicine 2017: 15; 139. doi:10.1186/s12916-017-0907 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s12916-017-0907&link_type=DOI) 21. 19.Chu C, White AN, Stone P. Prognostication in palliative care. Clinical Medicine 2019: 19; 306–10. doi:10.7861/clinmedicine.19-4-306 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTI6ImNsaW5tZWRpY2luZSI7czo1OiJyZXNpZCI7czo4OiIxOS80LzMwNiI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIzLzA5LzA4LzIwMjMuMDkuMDcuMjMyOTUxOTYuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 22. 20.Walsh RI, Mitchell G, Francis L, Van Driel ML. What Diagnostic Tools Exist for the Early Identification of Palliative Care Patients in General Practice? A systematic Review. J Pall Care 2015; 31; 118–123. doi:10.1177/082585971503100208 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/082585971503100208&link_type=DOI) 23. 21.Batsite G, Martinez-Munoz M, Blay C, Amblas J, Vila L, Costa X, Espaulella J, Villanueva A, Oller, R, Martoni JC, Constante C. Utility of the NECPAL CCOMS-ICO© tool and the Surprise Question as screening tools for early palliative care and to predict mortality in patients with advanced chronic conditions: A cohort study. Palliat Med 2016:31;754–763. doi:10.1177/0269216316676647 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/0269216316676647&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=27815556&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F09%2F08%2F2023.09.07.23295196.atom) 24. 22.Gold Standard Framework - Evidence that use of PIG improves early identification of patients in different settings and conditions (goldstandardsframework.org.uk) [https://www.goldstandardsframework.org.uk/](https://www.goldstandardsframework.org.uk/). Accessed 05.09.2023. 25. 23.O’Callaghan A, Laking G, Frey R, Robinson J, Gott M. Can we predict which hospitalised patients are in their last year of life? A prospective cross-sectional study of the gold standards framework prognostic Indicator guidance as a screening tool in the acute hospital setting. Palliat Med. 2014;28(8):1046–52. doi:10.1177/0269216314536089 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/0269216314536089&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24854032&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F09%2F08%2F2023.09.07.23295196.atom) 26. 24.De Bock R, Van Den Noortgate N, Piers R. Validation of the supportive and palliative care indicators tool in a geriatric population. J Palliat Med. 2018;21:220–4. doi:10.1089/jpm.2017.0205 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1089/jpm.2017.0205&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F09%2F08%2F2023.09.07.23295196.atom) 27. 25.Raubenheimer, P.J., Day, C., Abdullah, F. et al. The utility of a shortened palliative care screening tool to predict death within 12 months – a prospective observational study in two south African hospitals with a high HIV burden. BMC Palliat Care 18, 101 (2019). doi:10.1186/s12904-019-0487-5 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s12904-019-0487-5&link_type=DOI) 28. 26.Mitchell GK, Senior HE, Rhee JJ, et al. Using intuition or a formal palliative care needs assessment screening process in general practice to predict death within 12 months: A randomised controlled trial. Palliative Medicine. 2018;32(2):384–394. doi:10.1177/0269216317698621 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/0269216317698621&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=28452570&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F09%2F08%2F2023.09.07.23295196.atom) 29. 27.NHS England — North West:EARLY Identification in Primary Care. [https://www.england.nhs.uk/north-west/north-west-coast-strategic-clinical-networks/our-networks/palliative-and-end-of-life-care/for-professionals/early-toolkit-for-primary-care/?seq\_no=3](https://www.england.nhs.uk/north-west/north-west-coast-strategic-clinical-networks/our-networks/palliative-and-end-of-life-care/for-professionals/early-toolkit-for-primary-care/?seq_no=3). Accessed: 05.09.23 30. 28.NHS England. policy paper. A plan for digital health and social care. 2022. [https://www.gov.uk/government/publications/a-plan-for-digital-health-and-social-care/a-plan-for-digital-health-and-social-care](https://www.gov.uk/government/publications/a-plan-for-digital-health-and-social-care/a-plan-for-digital-health-and-social-care). Accessed: 05.09.2023. 31. 29.Thomas K, Muir. Population-based, person-centred end-of-life care: time for a rethink. Brit J Gen Pract, 2018: 68; 116–117. doi:10.3399/bjgp18x694925 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3399/bjgp18x694925&link_type=DOI) 32. 30.Singh BM, Kumari-Dewat N, Ryder A, Parry E, Klaire V, Matthews D, Bennion G, Jennens H, Ritzenthaler BME, Rayner S, Shears J, Ahmed K, Sidhu M, Viswanath A, Warren K. Digital health, and inpatient palliative care: a cohort-controlled study. l. BMJ Supportive & Palliative Care 2023; :1–4. doi:10.1136/spcare-2023-004474 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiYm1qc3BjYXJlIjtzOjU6InJlc2lkIjtzOjIwOiJzcGNhcmUtMjAyMy0wMDQ0NzR2MSI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIzLzA5LzA4LzIwMjMuMDkuMDcuMjMyOTUxOTYuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 33. 31.Montgomery K. How Doctors Think: Clinical Judgment and the Practice of Medicine. New York: Oxford University Press 2006 34. 32.Kienle GS, Kiene H. Clinical judgement and the medical profession. J Eval Clin Pract 2011; 17:621–7. doi:10.1111/j.1365-2753.2010.01560. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1365-2753.2010.01560.x&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=20973873&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F09%2F08%2F2023.09.07.23295196.atom) 35. 33.Sanders S, Doust J, Glasziou P. A Systematic Review of Studies Comparing Diagnostic Clinical Prediction Rules with Clinical Judgment. PLoS One 2015;10:e0128233. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pone.0128233&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=26039538&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F09%2F08%2F2023.09.07.23295196.atom) 36. 34.Pentzek M, Wagner M, Abholz H-H, et al. The value of the GP’s clinical judgement in predicting dementia: a multicentre prospective cohort study among patients in general practice. British Journal of General Practice 2019;69: e786–93. doi:10.3399/bjgp19X706037 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYmpncCI7czo1OiJyZXNpZCI7czoxMToiNjkvNjg4L2U3ODYiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMy8wOS8wOC8yMDIzLjA5LjA3LjIzMjk1MTk2LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 37. 35.Gill CJ, Sabin L, Schmid CH. Why clinicians are natural Bayesians. BMJ 2005; 330:1080–3. doi:10.1136/bmj.330.7499.1080 [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEzOiIzMzAvNzQ5OS8xMDgwIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjMvMDkvMDgvMjAyMy4wOS4wNy4yMzI5NTE5Ni5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 38. 36.Parry E, Ahmed K, Evans S, et al. General practitioner assessment of unmet need in a complex multimorbid population using a data driven and clinical triage system. BJGP Open 2023; BJGPO.2023.0078. doi:10.3399/BJGPO.2023.0078 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NjoiYmpncG9hIjtzOjU6InJlc2lkIjtzOjE3OiJCSkdQTy4yMDIzLjAwNzh2MSI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIzLzA5LzA4LzIwMjMuMDkuMDcuMjMyOTUxOTYuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 39. 37.Heneghan C, Glasziou P, Thompson M, et al. Diagnostic strategies used in primary care. BMJ 2009;338: b946–b946. doi:10.1136/bmj.b946 [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE2OiIzMzgvYXByMjBfMS9iOTQ2IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjMvMDkvMDgvMjAyMy4wOS4wNy4yMzI5NTE5Ni5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 40. 38.Body R, Cook G, Burrows G, et al. Can emergency physicians ‘rule in’ and ‘rule out’ acute myocardial infarction with clinical judgement? Emergency Medicine Journal 2014; 31:872–6. doi:10.1136/emermed-2014-203832 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiZW1lcm1lZCI7czo1OiJyZXNpZCI7czo5OiIzMS8xMS84NzIiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMy8wOS8wOC8yMDIzLjA5LjA3LjIzMjk1MTk2LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==)