Prognostic accuracy of qSOFA score, SIRS criteria, and EWSs for in-hospital mortality among adult patients presenting with suspected infection to the emergency department (PASSEM): protocol for an international multicentre prospective external validation cohort study =========================================================================================================================================================================================================================================================================== * Abdullah M. Algarni * Musa S. Alfaifi * Ali A. Al Bshabshe * Othman M. Omair * Mohammed A. Alsultan * Hasan M. Alzahrani * Hadi E. Alali * Abdullah A. Alsabaani * Ali M. Alqarni * Salah A. Alghanem * Nawaf M. Basulaib * Bandar S. Al Mufareh * Basem A. Al Gamdi * Ammar M. Al-Kashmiri * Ayesha M. Almemari * Abdulrahman A. Sindi * Ibrahim U. Ozturan * Abdullah A. Alhadhira * Asaad S. Shujaa * Abdullah H. Alotaibi * Moosa M. Awladthani * Mohammed M. Alghamdi * Abdullah A. Almarshed * AlHanouv M. AlQahtani * Tim R. Harris * Bader A. Alyahya * Saad Ali AlAsiri * Feras H. Abuzeyad * Sara N. Kazim * Abdulrahman A. Al-Fares * Naif H. Alshahrani * Faisal Y. Almazroua * Naima N. Al Hinai * Badr N. Alhamadi * Mohammed A. Al jumaan * Naif T. Marzook * Abdussalam A. Alshehri * Mujahid M. Alotaibi * Abdullah A. Basri * Abdulaziz S. Alalshaikh * Ahmed A. Alsaad * Yousef I. Alawad * Mohammed A. Asa’d * Moonis M. Farooq * Muneera M. Alasfour * Nidal H. Bokhary * Saeed F. Alqahtani * Wejdan S. Alzahrani * Abdulmajeed M. Omair * Faysal M. Alasmari * Sultan Y. Alfifi * Mohammed S. Al-Nujimi * Farid Foroutan ## ABSTRACT **Introduction** Early identification of a patient with infection who may develop sepsis is of utmost importance. Unfortunately, this remains elusive because no single clinical measure or test can reflect complex pathophysiological changes in patients with sepsis. However, multiple clinical and laboratory parameters indicate impending sepsis and organ dysfunction. Screening tools using these parameters can help identify the condition, such as SIRS, quick SOFA (qSOFA), National Early Warning Score (NEWS), or Modified Early Warning Score (MEWS). We aim to externally validate qSOFA, SIRS, and NEWS/NEWS2/MEWS for in-hospital mortality among adult patients with suspected infection who presenting to the emergency department. **Methods and analysis** PASSEM study is an international prospective external validation cohort study. For 9 months, each participating centre will recruit consecutive adult patients who visited the emergency departments with suspected infection and are planned for hospitalisation. We will collect patients’ demographics, vital signs measured in the triage, initial white blood cell count, and variables required to calculate Charlson Comorbidities Index; and follow patients for 90 days since their inclusion in the study. The primary outcome will be 30-days in-hospital mortality. The secondary outcome will be intensive care unit (ICU) admission, prolonged stay in the ICU (i.e., >72 hours), and 30-as well as 90-days all-cause mortality. The study started in December 2021 and planned to enrol 2851 patients to reach 200 in-hospital death. The sample size is adaptive and will be adjusted based on prespecified consecutive interim analyses. **Ethics and dissemination** The Aseer Regional Committee for Research Ethics in the General Directorate of Health Affairs-Aseer Region has approved this study. Informed consent is not required for this study due to its complete observational nature and absence of interventions or invasive procedures. We will publish study’s results in peer-reviewed journals and may present them at scientific conferences. **Study registration number** [ClinicalTrials.gov](http://ClinicalTrials.gov) identifier: [NCT05172479](http://medrxiv.org/lookup/external-ref?link_type=CLINTRIALGOV&access_num=NCT05172479&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F22%2F2022.03.19.22272537.atom) ## INTRODUCTION ### Background Over the past decade, there has been continued focus on sepsis as a prevalent condition that accounts for 10% of admissions to intensive care units (ICUs) and is associated with a 10–20% in-hospital mortality rate.1-5 Standardised protocols and physician awareness have significantly improved survival, but mortality rates remain high between 20% and 36%, with ∼270,000 deaths annually in the United States.6-8 It has been estimated that 80% of sepsis cases are identified and treated in the emergency department (ED), and the remainder develop sepsis during hospitalisation with other conditions.7 In 2016, the Society of Critical Care Medicine/European Society of Intensive Care Medicine (SCCM/ESICM) task force redefined sepsis based on organ dysfunction and mortality prediction.9-11 Sepsis now is defined as life-threatening organ dysfunction caused by dysregulated host response to infection. This definition emphasises the complexity of the disease that cannot be explained by infection or body response alone. Acute change in Sequential Organ Failure Assessment (SOFA) score ≥2 indicates sepsis-related organ dysfunction and is associated with in-hospital mortality. Systemic Inflammatory Response Syndrome (SIRS) and “severe sepsis” terms were omitted from the most recent definition. SIRS has been criticised for its poor specificity, while “severe sepsis” may underestimate sepsis’s seriousness. A subset of patients may develop septic shock with underlying profound organ dysfunction and excess mortality. Clinically, septic shock is defined as persistent hypotension requiring vasopressors to maintain mean arterial pressure (MAP) ≥ 65 mm Hg and serum lactate level ≥ 2 mmol/L (18 mg/dL) despite adequate volume resuscitation. Early identification of a patient with infection who may develop sepsis is of utmost importance.12 Unfortunately, this remains elusive because no single clinical measure or test can reflect the complex pathophysiological changes in patients with sepsis. However, multiple clinical and laboratory parameters indicate impending sepsis and organ dysfunction. Screening tools using these parameters can help identify the condition, such as SIRS, quick SOFA (qSOFA), National Early Warning Score (NEWS), or Modified Early Warning Score (MEWS) (Table 1 and 2).13 The 2016 SCCM/ESICM task force recommended using qSOFA,11 while the 2021 Surviving Sepsis Campaign strongly recommended against its use compared with SIRS, NEWS, or MEWS as a single screening tool for sepsis or septic shock.14 View this table: [Table 1.](http://medrxiv.org/content/early/2022/03/22/2022.03.19.22272537/T1) Table 1. Component of qSOFA score and SIRS criteria View this table: [Table-2.](http://medrxiv.org/content/early/2022/03/22/2022.03.19.22272537/T2) Table-2. Components of NEWS, NEWS2, and MEWS We hypothesised that qSOFA has greater prognostic accuracy than SIRS and EWSs (NEWS/NEWS2/MEWS). We developed a protocol for a prognostic study to determine whether qSOFA has a higher predictive performance for relevant clinical outcomes in adult patients with infection presenting to the ED. ### Rationale of the study Multiple studies have assessed qSOFA, SIRS, and EWSs validity in ED and showed conflicting results.15-21 One systemic review compared qSOFA and EWSs (NEWS/Modified EWS [MEWS]) for predicting ICU admission and mortality when applied in the ED.13 None of the eligible studies included NEWS2; and the authors of the review could not perform a meta-analysis due to marked heterogeneity in patient selection, outcomes, and settings. Moreover, studies have calculated the scores at different times. NEWS appeared more sensitive than qSOFA for predicting ICU admission and mortality at the commonly used thresholds (i.e., ≥2 for SIRS and qSOFA; ≥5 for NEWS, NEWS2, and MEWS), whereas qSOFA was more specific.13 This correlates with previous criticisms of qSOFA, which have low sensitivity for early risk assessment.18-21 We are unaware of any prospective studies that validate or specifically studied the predictive accuracy of qSOFA score, SIRS criteria, or EWSs in the ED in Middle East region in general and Arab countries in particular. Moreover, previous international studies have limitations that affect their validity.13 ### Aim The purpose of this study is to externally validate qSOFA score, SIRS criteria, and EWSs for in-hospital mortality among adult patients with suspected infection presenting to the ED in the Middle East region. In an ED setting, it is crucial to take a patient’s vital signs as early as possible to make decisions and predict the patient’s outcome. Hence, PASSEM study will use initial physiologic parameters the patient presented with, to the ED (triage vital signs) to calculate qSOFA, SIRS criteria, and EWSs, which might be more appropriate and pragmatic. The primary endpoint is in-hospital mortality within 30 days. The secondary endpoints are ICU admission, ICU length of stay, and all-cause mortality within 30- and 90-days following presentation to the ED. ## METHODS AND ANALYSIS ### Study design and setting This protocol describes a multicentre, prospective observational cohort study evaluating the prognostic accuracy of qSOFA score, SIRS criteria, and EWSs (NEWS/NEWS2/MEWS) for in-hospital mortality among adult patients presenting to the ED with suspected infection ([NCT05172479](http://medrxiv.org/lookup/external-ref?link_type=CLINTRIALGOV&access_num=NCT05172479&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F22%2F2022.03.19.22272537.atom)). The study’s outlines are shown in Figure 1. The study duration is 12 months per centre (9 months for recruitment and 3 months for follow-up). Recruiting centres and recruitment status are shown in Table 3 (19 March 2022). View this table: [Table 3.](http://medrxiv.org/content/early/2022/03/22/2022.03.19.22272537/T3) Table 3. PASSEM Study Centres ![Figure 1.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/03/22/2022.03.19.22272537/F1.medium.gif) [Figure 1.](http://medrxiv.org/content/early/2022/03/22/2022.03.19.22272537/F1) Figure 1. Study outlines ### Diagnosis of infection A presumptive diagnosis of infection will be judged based on the opinion of the ED physician upon the initial patient presentation. If required, two experts from each recruiting centre will ascertain the diagnosis of infection on the 30th day since inclusion to the study. Evidence of infection will be sought by analysing the patient’s clinical, microbiological, and radiological data. Evidence of infection would be determined by either positive culture, other microbiological techniques (e.g., serological, or molecular), or radiological findings. If all of these evidence measures were equivocal, clinical context will be used to confirm the presence of infection. In cases of disagreement, consensus will be sought between the two experts. In all cases, the diagnosis of infection will be blinded to the output of the prediction models and the outcomes of patients. ### Study population #### Inclusion criteria PSSEM study will enrol all consecutive adult patients (age ≥18 years) presenting to the ED with suspected infection who are planned for hospitalisation (Box 1). Box 1 **Eligibility criteria** **Inclusion criteria** * ▸ Adult patient (ages ≥18 years). * ▸ Suspected infection (based on the opinion of the emergency physician). * ▸ Planned for hospitalisation. * ▸ Willing to give oral informed consent (per centre policy). **Exclusion criteria** * ▸ Presentation to ED is not due to infection (e.g., autoimmune diseases, myocardial infarction, stroke, venous thromboembolism, trauma, intoxication … etc.). * ▸ Pregnancy. * ▸ Transferred from another hospitals. * ▸ Code status is “Do-Not-Resuscitate” (DNR). * ▸ Elective admission to the hospital (i.e., not through emergency department). * ▸ Initial diagnosis of infection in the ED was not confirmed a”er finishing of the recruitment and follow-up phase. #### Exclusion criteria We will exclude patients who present to the ED due to non-infectious causes (e.g., autoimmune diseases, myocardial infarction, trauma, …etc.), pregnant woman, those who are transferred from other hospitals, or with “Do-Not-Resuscitate” (DNR) code status. Patients whose initial diagnosis of infection in the ED was not confirmed after the recruitment and follow-up will also be excluded (Box 1). ### PASSEM study versus original derivation cohorts The key characteristics of PASSEM Study and the original derivation cohorts of qSOFA, SIRS, and EWSs that will be assessed are shown in Table 4. View this table: [Table 4.](http://medrxiv.org/content/early/2022/03/22/2022.03.19.22272537/T4) Table 4. Characteristics of PASSEM study and the original development cohorts of qSOFA score, SIRS criteria NEWS, NEWS2, and MEWS ### Study flow chart The study’s procedures and assessments are shown in Table 5. Patients will undergo 4 phases: screening (Time-1 [T-1], 1-2 days), enrolment (T), and in-hospital (T1, maximum 30 days after T), and out-hospital follow-up (T2, maximum 90 days after T). View this table: [Table 5](http://medrxiv.org/content/early/2022/03/22/2022.03.19.22272537/T5) Table 5 #### Screening and enrolment phases A staff member will screen patients for eligibility and check their measured vital signs once they arrive at the ED (triage) and the investigator will enrol potentially eligible patients (i.e., age ≥18 years, with suspected infection, and planned for hospitalisation). First, a web-based electronic data capture system (EDC) will assign each patient to a participant number in ascending order. Then, the investigator will collect and enter the patient’s initial data (demographics, contact information, Charlson Comorbidity Index (CCI) components, and variables required for qSOFA, SIRS, and EWS scores calculation) in an electronic case report form (eCRF) (see online supplementary materials). If the patient is not eligible, we will close the patient record in the EDC and clarify the cause of exclusion. #### In-hospital follow-up Once enrolment is completed, the in-hospital follow-up phase will start (T1, maximum 30 days after T) (Table 5). Study team will monitor hospitalised patients’ status (i.e., death, alive and either discharged, transferred to another hospital, or still hospitalised) by consulting their specific medical registration number (MRN) in the recruiting centre. #### Out-hospital follow-up This phase starts if the patient is discharged from the hospital or 30-days have passed since inclusion to the study (whenever earlier; T2, maximum 90 days after T) (Table 5). We will determine their status via telephone contact. We will also evaluate hospitalised patients’ situations by consulting their MRN in the recruiting centre. We will consider a patient lost to follow-up if we cannot reach them via telephone contact by the end of this phase. ### Study outcome The primary outcome of this study is 30-days in-hospital mortality. Secondary outcomes include ICU admission (within 30-days), ICU length of stay, and all-cause mortality within 30 and 90 days. ### Predictors Lead investigator in each centre will extract the demographics, components of CCI, vital signs, and blood investigations from the medical record of each potentially eligible patient. Study team will use the patient’s initial vital signs, level of consciousness (i.e., first measurement in triage), WBC count, and partial pressure of carbon dioxide (pCO2) to calculate qSOFA, SIRS, and EWSs. Blood pressure will be measured by using an electronic sphygmomanometer and results will be recorded in millimetres of mercury (mmHg). MAP will be calculated from SBP and diastolic blood pressure (DBP) using the following equation: ![Formula][1] Pulse rate (recorded as beats/min) and oxygen saturation (recorded as a percentage) will be measured using an electronic pulse oximetry device. We will report whether the oxygen saturation reading was in room air or while a patient is on oxygen therapy. Body temperature will be measured orally or (axillary if necessary) by electronic thermometer and recorded as degree Celsius. A new-onset Glasgow Coma Scale (GCS) score of <15 will be considered significant for qSOFA calculation. If it is unclear whether a patient’s confusion is ‘new’ or their usual state, we will assume the altered mental state/confusion is new until confirmed otherwise for all scores calculation. Initial WBCs count (recorded in ×109/μL) and pCO2 (recorded in mmHg) will be obtained from the patient’s medical record and entered into the eCRF. ### Sample size In the PASSEM study, we chose the method suggested by Collins et al.22 In this method, sample size calculation is based on the expected event rate (minimum of 100 events in all validation datasets). However, rules-of-thumb for sample size are problematic, as they are not specific to the model or validation setting. Indeed, Snell et al showed that the rule-of-thumb of having at least 100 events and 100 non-events does not always produce precise estimates of a model’s predictive performance measures.23 To overcome this limitation, we chose to target an event rate of ≥200. Previous work by Freund et al 15 showed that a sample size of 879 patients yielded 74 events when power was set at 90%. Therefore, if we target a minimum of 200 events and consider 20% of lost to follow-up and missing data, a sample size of 2851 should be included. We will conduct an interim analysis after recruitment of 25%, 50%, and 75% of the target sample size to re-evaluate our assumptions and correct the sample size accordingly. ### Statistical analysis Continuous data will be reported as mean (SD) or median (IQR) and compared using unpaired *t* tests or analysis of variance and Mann-Whitney or Kruskal-Wallis test. Categorical variables variables will be expressed as number (percentage) and compared using a χ2 test or a Fisher exact test. We will begin by calculating an overall area under the receiver operating characteristic curve (AUC of ROC curve) and generate calibration curves of the qSOFA, SIRS, and EWSs to predict the primary and secondary outcomes. Subsequent to assessing the model’s overall performance; sensitivity, specificity, positive and negative predictive values will be calculated with cross tables for predicting primary and secondary outcomes for a qSOFA score of ≥2, SIRS of ≥2, and EWSs of ≥5. We will use the Kaplan-Meier method to estimate in-hospital and 90-day all-cause mortality. A log-rank regressions will be used to assess groups’ differences. Odd ratios (ORs) for in-hospital death, ICU admission, and 90-days all-cause mortality of qSOFA, SIRS, and EWSs will be estimated with a logistic regression analysis after adjustment for the patients’ demographics, comorbidities, and CCI. The model fit will be assessed by the calculation of the log-likelihood, Akaike information criterion (AIC), AUC, Bayesian information criterion (BIC), and D-statistics. To compare the performance of qSOFA, SIRS and EWSs, we will use absolute net-reclassification index (NRI). The absolute NRI mathematically represents a net proportion of patients correctly reclassified by one model as compared to another.24 Net reclassification involves classifying patients in risk categories and determining how a new model reclassifies patients into various risk categories compared with a previous model. Risk differences are classified based on the actual outcome patients experienced (those who died vs those who did not). A priori subgroup analyses will be conducted based on status of the following: COVID-19 (present vs absent), febrile neutropenia (present vs absent), solid organs or hematological cancers (present vs absent), autoimmune diseases (present vs absent), and severe comorbidities (CCI ≥3 vs <3), and race of the patient (Asian vs Black vs South Asian vs White) as permitted by sample size. If missing data is minimal (<5%) we will conduct a complete case analysis, otherwise we will use multiple imputation. For all analyses, a 2-tailed *P* <0.05 will be considered statistically significant. Statistical analyses will be performed with STATA (StataCorp LLC). ### Data management We will use an encrypted, web-based EDC (Castor®) for this study.25 Lead investigators (or their delegates) will enter clinical data on an eCRF at each participating centre. They will make all entries, corrections, and alterations. The data manager of this study will provide all tools, instructions, and training necessary to complete the eCRF, and each user will be issued a unique username and password. The monitors will review the eCRFs, evaluate them for completeness and consistency, and compare them with the source documents to ensure no discrepancies. The Monitors cannot enter data in the eCRFs. Lead investigators must verify that all data entries in the eCRF are accurate and correct. If some assessments are not done, or specific information is unavailable, not applicable, or unknown, they must indicate this in the eCRF. Finally, lead investigators must electronically sign off all patients’ eCRF enrolled from their hospitals. Data manager will lock the final validated database so that no more change will be possible on the frozen data. Subsequently, the principal investigator will receive the patient data (eCRF data + audit trail) for archiving at the investigational site and transference in a secure way to the biostatistical team in STATA format. ### Patient and public involvement We will not involve patients or the public in the design, conduction, reporting, or dissemination of this study. ## ETHICS AND DISSEMINATION ### Ethics #### Informed consent Informed consent is not required of this study due to its complete observational nature and absence of interventions or invasive procedures. The study does not impose any change in the standard practice of sepsis at the site. The benefit/risk ratio of participation in the study is excellent. Moreover, we expect that PASSEM Study results may improve patient care in the recruiting centre by allowing a better understanding of ideal tools to identify patients with sepsis. #### Ethical approval This protocol complies with the principles laid down by the 59th World Medical Assembly and all applicable amendments laid down by the World Medical Assemblies, the applicable regulations per site, and any other relevant local requirement and laws. The Aseer Regional Committee for Research Ethics in the General Directorate of Health Affairs-Aseer Region has approved this study. The principal investigator will also work closely with the lead investigators of other hospitals to obtain all necessary ethical approval from the Local IRBs (LIRBs) as requested by the policies of these hospitals. Data manager of PASSEM study will not grant access to the EDC system or start the study until the principal investigator receives a copy of a written and dated approval/favourable signed opinion from each participating centre LIRB. We will present any change in this protocol as an amendment in written form to the protocol. The principal investigator and lead investigators will sign the protocol amendment and then submitted to the LIRBs. Following approval, we will send the amendment to all participating investigators. The amendment cannot be acted upon before the outcome of this decision. However, the study team will submit minor modifications (administrative modifications, including a new recruitment centre) to the LIRBs for information purposes only. ### Dissemination We will publish study’s results in peer-reviewed journals and may present them at scientific conferences. We will follow recommendations of Transparent Reporting of a Multivariable Prediction Model For Individual Prognosis or Diagnosis (TRIPOD) guidelines.26 The most significant results will be shared to the public through social networks. ### Patient confidentiality In order to maintain confidentiality, we will not collect any patient’s-identifying data (e.g., name, identification number, medical record number [MRN], etc.) in the eCRF. Instead, lead investigators will store such data in a separate list sheet specified for each participating centre. The lead investigator of each centre will maintain this list in strict confidence. ## Supporting information Case report form [[supplements/272537_file02.pdf]](pending:yes) ## Data Availability All data produced in the present study are available upon reasonable request to the authors * Received March 19, 2022. * Revision received March 19, 2022. * Accepted March 22, 2022. * © 2022, Posted by Cold Spring Harbor Laboratory This pre-print is available under a Creative Commons License (Attribution-NonCommercial-NoDerivs 4.0 International), CC BY-NC-ND 4.0, as described at [http://creativecommons.org/licenses/by-nc-nd/4.0/](http://creativecommons.org/licenses/by-nc-nd/4.0/) ## REFERENCES 1. 1.Angus DC, van der Poll T. Severe sepsis and septic shock. The New England journal of medicine 2013;369(9):840–51. doi: 10.1056/NEJMra1208623 [published Online First: 2013/08/30] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMra1208623&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=23984731&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F22%2F2022.03.19.22272537.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000323676100011&link_type=ISI) 2. 2.Cohen J, Vincent JL, Adhikari NK, et al. Sepsis: a roadmap for future research. Lancet Infect Dis 2015;15(5):581–614. doi: 10.1016/s1473-3099(15)70112-x [published Online First: 2015/05/02] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S1473-3099(15)70112-X&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25932591&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F22%2F2022.03.19.22272537.atom) 3. 3.Mouncey PR, Osborn TM, Power GS, et al. Trial of early, goal-directed resuscitation for septic shock. The New England journal of medicine 2015;372(14):1301–11. doi: 10.1056/NEJMoa1500896 [published Online First: 2015/03/18] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMoa1500896&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25776532&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F22%2F2022.03.19.22272537.atom) 4. 4.Peake SL, Delaney A, Bailey M, et al. Goal-directed resuscitation for patients with early septic shock. The New England journal of medicine 2014;371(16):1496–506. doi: 10.1056/NEJMoa1404380 [published Online First: 2014/10/02] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMoa1404380&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25272316&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F22%2F2022.03.19.22272537.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000342994700006&link_type=ISI) 5. 5.Yealy DM, Kellum JA, Huang DT, et al. A randomized trial of protocol-based care for early septic shock. The New England journal of medicine 2014;370(18):1683–93. doi: 10.1056/NEJMoa1401602 [published Online First: 2014/03/19] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMoa1401602&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24635773&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F22%2F2022.03.19.22272537.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000335405200006&link_type=ISI) 6. 6.Minasyan H. Sepsis and septic shock: Pathogenesis and treatment perspectives. J Crit Care 2017;40:229–42. doi: 10.1016/j.jcrc.2017.04.015 [published Online First: 2017/04/28] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jcrc.2017.04.015&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=28448952&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F22%2F2022.03.19.22272537.atom) 7. 7.Rhee C, Dantes R, Epstein L, et al. Incidence and Trends of Sepsis in US Hospitals Using Clinical vs Claims Data, 2009-2014. JAMA 2017;318(13):1241–49. doi: 10.1001/jama.2017.13836 [published Online First: 2017/09/14] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.2017.13836&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F22%2F2022.03.19.22272537.atom) 8. 8.Sherwin R, Winters ME, Vilke GM, et al. Does Early and Appropriate Antibiotic Administration Improve Mortality in Emergency Department Patients with Severe Sepsis or Septic Shock? The Journal of emergency medicine 2017;53(4):588–95. doi: 10.1016/j.jemermed.2016.12.009 [published Online First: 2017/09/17] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jemermed.2016.12.009&link_type=DOI) 9. 9.Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of Clinical Criteria for Sepsis: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA 2016;315(8):762–74. doi: 10.1001/jama.2016.0288 [published Online First: 2016/02/24] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.2016.0288&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=26903335&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F22%2F2022.03.19.22272537.atom) 10. 10.Shankar-Hari M, Phillips GS, Levy ML, et al. Developing a New Definition and Assessing New Clinical Criteria for Septic Shock: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA 2016;315(8):775–87. doi: 10.1001/jama.2016.0289 [published Online First: 2016/02/24] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.2016.0289&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=26903336&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F22%2F2022.03.19.22272537.atom) 11. 11.Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA 2016;315(8):801–10. doi: 10.1001/jama.2016.0287 [published Online First: 2016/02/24] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.2016.0287&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=26903338&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F22%2F2022.03.19.22272537.atom) 12. 12.Gauer R, Forbes D, Boyer N. Sepsis: Diagnosis and Management. American family physician 2020;101(7):409-18 [published Online First: 2020/04/02] 13. 13.Sabir L, Ramlakhan S, Goodacre S. Comparison of qSOFA and Hospital Early Warning Scores for prognosis in suspected sepsis in emergency department patients: a systematic review. Emerg Med J 2021 doi: 10.1136/emermed-2020-210416 [published Online First: 2021/08/19] [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiZW1lcm1lZCI7czo1OiJyZXNpZCI7czo4OiIzOS80LzI4NCI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIyLzAzLzIyLzIwMjIuMDMuMTkuMjIyNzI1MzcuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 14. 14.Evans L, Rhodes A, Alhazzani W, et al. Surviving sepsis campaign: international guidelines for management of sepsis and septic shock 2021. Intensive Care Med 2021;47(11):1181–247. doi: 10.1007/s00134-021-06506-y [published Online First: 2021/10/03] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/S00134-021-06506-Y&link_type=DOI) 15. 15.Freund Y, Lemachatti N, Krastinova E, et al. Prognostic Accuracy of Sepsis-3 Criteria for In-Hospital Mortality Among Patients With Suspected Infection Presenting to the Emergency Department. JAMA 2017;317(3):301–08. doi: 10.1001/jama.2016.20329 [published Online First: 2017/01/24] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.2016.20329&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=28114554&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F22%2F2022.03.19.22272537.atom) 16. 16.Haydar S, Spanier M, Weems P, et al. Comparison of QSOFA score and SIRS criteria as screening mechanisms for emergency department sepsis. The American journal of emergency medicine 2017;35(11):1730–33. doi: 10.1016/j.ajem.2017.07.001 [published Online First: 2017/07/18] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.ajem.2017.07.001&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F22%2F2022.03.19.22272537.atom) 17. 17.Hwang SY, Jo IJ, Lee SU, et al. Low Accuracy of Positive qSOFA Criteria for Predicting 28-Day Mortality in Critically Ill Septic Patients During the Early Period A”er Emergency Department Presentation. Annals of emergency medicine 2018;71(1):1-9.e2. doi: 10.1016/j.annemergmed.2017.05.022 [published Online First: 2017/07/04] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.annemergmed.2017.05.022&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F22%2F2022.03.19.22272537.atom) 18. 18.Churpek MM, Snyder A, Han X, et al. Quick Sepsis-related Organ Failure Assessment, Systemic Inflammatory Response Syndrome, and Early Warning Scores for Detecting Clinical Deterioration in Infected Patients outside the Intensive Care Unit. Am J Respir Crit Care Med 2017;195(7):906–11. doi: 10.1164/rccm.201604-0854OC [published Online First: 2016/09/21] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1164/rccm.201604-0854OC&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=27649072&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F22%2F2022.03.19.22272537.atom) 19. 19.Goulden R, Hoyle MC, Monis J, et al. qSOFA, SIRS and NEWS for predicting inhospital mortality and ICU admission in emergency admissions treated as sepsis. Emerg Med J 2018;35(6):345–49. doi: 10.1136/emermed-2017-207120 [published Online First: 2018/02/23] [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiZW1lcm1lZCI7czo1OiJyZXNpZCI7czo4OiIzNS82LzM0NSI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIyLzAzLzIyLzIwMjIuMDMuMTkuMjIyNzI1MzcuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 20. 20.Nieves Ortega R, Rosin C, Bingisser R, et al. Clinical Scores and Formal Triage for Screening of Sepsis and Adverse Outcomes on Arrival in an Emergency Department All-Comer Cohort. The Journal of emergency medicine 2019;57(4):453-60.e2. doi: 10.1016/j.jemermed.2019.06.036 [published Online First: 2019/09/11] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jemermed.2019.06.036&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F22%2F2022.03.19.22272537.atom) 21. 21.Usman OA, Usman AA, Ward MA. Comparison of SIRS, qSOFA, and NEWS for the early identification of sepsis in the Emergency Department. The American journal of emergency medicine 2019;37(8):1490–97. doi: 10.1016/j.ajem.2018.10.058 [published Online First: 2018/11/25] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.ajem.2018.10.058&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F22%2F2022.03.19.22272537.atom) 22. 22.Collins GS, Ogundimu EO, Altman DG. Sample size considerations for the external validation of a multivariable prognostic model: a resampling study. Stat Med 2016;35(2):214–26. doi: 10.1002/sim.6787 [published Online First: 2015/11/11] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/sim.6787&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=26553135&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F22%2F2022.03.19.22272537.atom) 23. 23.Archer L, Snell KIE, Ensor J, et al. Minimum sample size for external validation of a clinical prediction model with a continuous outcome. Stat Med 2021;40(1):133–46. doi: 10.1002/sim.8766 [published Online First: 2020/11/06] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/sim.8766&link_type=DOI) 24. 24.Alba A, Agoritsas T, Walsh M, et al. Discrimination and calibration of clinical prediction models: Users’ guides to the medical literature. JAMA 2017;318(14):1377–84. doi: 10.1001/jama.2017.12126 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.2017.12126&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=29049590&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F22%2F2022.03.19.22272537.atom) 25. 25.Castor EDC. Castor Electronic Data Capture Netherlands2021 [24 October 2021]. Available from: [https://castoredc.com](https://castoredc.com) accessed 24 October 2021. 26. 26.Collins GS, Reitsma JB, Altman DG, et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ : British Medical Journal 2015;350 doi: 10.1136/bmj.g7594 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNTAvamFuMDdfNC9nNzU5NCI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIyLzAzLzIyLzIwMjIuMDMuMTkuMjIyNzI1MzcuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) [1]: /embed/graphic-8.gif