Systematic review and meta-analysis of the accuracy of McIsaac and Centor score in patients presenting to secondary care with Pharyngitis

Study eligibility criteria

Cohort, case control studies, and nested case control studies that included study populations consisting of adults or children presenting to secondary care with pharyngitis, that used McIsaac or Centor scores to evaluate the risk of GAS pharyngitis and used throat culture and/or rapid antigen tests as reference standards were eligible for inclusion in our review. We considered RADTs sufficiently accurate for detecting GAS pharyngitis and so included studies that used RADT as the sole reference standard. All studies had to report sufficient data for McIsaac or Centor at scores 1, 2, 3, and 4 to make a crosstabulation of people with and without GAS against each score.

Data sources and study selection

MEDLINE, Embase and Web of Science were searched from inception until September 2022. An update search was performed in September 2022. Search terms for clinical prediction rules (CPR), including Centor and McIsaac, were combined with terms for pharyngitis and GAS (See Supplementary data for a detailed search strategy).

Two reviewers (AK and CE) independently screened titles and abstracts for relevance, followed by the detailed assessment of relevant full-text articles against the eligibility criteria, using COVIDENCE software (21). Any discrepancies were solved by discussion until a consensus was reached or by referral to a third reviewer (ME).

Data extraction

Data extraction forms were developed in COVIDENCE (21), tested on a small sample of studies and amended where necessary. We extracted information on study type, care setting, participant recruitment, participant age, total study population, type of reference standard, McIsaac or Centor scores (index tests) at all reported thresholds, and reference standard results. Non-GAS culture results were recorded as negative GAS cultures. Studies which omitted reporting number of cases with McIsaac score = 0 and McIsaac score = 5 were assumed to have no participants with these scores. If studies combined thresholds 0 and 1 and 4 and 5, we would consider these thresholds as 1 and 4, respectively, to allow comparison between Centor and McIsaac score for thresholds 1-4. If studies reported data for two reference standards separately i.e. RADT and throat culture, we extracted and used the throat culture data. Data extraction was performed by one reviewer (AK or CE) and checked by the other. Discrepancies were resolved through discussion.

Assessment of risk of bias

The Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool was used to assess the included studies’ quality which assesses 4 domains: patient selection, index test, reference standard and flow and timing (22). Low risk of bias was judged for: patient selection if participants were recruited consecutively, index test if it was interpreted without knowledge of reference standard results, and flow and timing if no data were missing and the clinical measures included in the prediction rule were measured before sample acquisition for RADT and/or throat culture. The reference standard domain was judged to be low risk in studies that used throat cultures as the reference standard and high risk if they used RADT as the only reference standard, given the limited accuracy of RADTs (23). We also judged this domain to be high risk if the reference standard was RADT followed by a confirmatory throat culture only in the RADT negatives as a proportion of patients would have only received an RADT reference standard. High risk of bias was determined for failure to meet these domain specific criteria. Domains were judged at unclear risk of bias if there was insufficient information for assessment. Studies which use convenience recruitment, inclusion/exclusion criteria specifically targeted at criteria from McIsaac or Centor score and not general pharyngitis symptoms were also judged to have applicability concerns due to potentially not recruiting a representative population of pharyngitis patients presenting to secondary care. Overall study level risk of bias was deemed low if all domains were at low risk, unclear if any domain had unclear risk of bias and no domains were at high risk, and high if any domain had high risk of bias. We presented risk of bias assessments in traffic light plots using the robvis R package (24).

Data synthesis and analysis

We first plotted study-level receiver operator characteristic (ROC) curves for each score. MetaDTA: Diagnostic Test Accuracy Meta-Analysis v2.0 (MetaDTA), a web application (25, 26), was then used to perform meta-analyses to pool sensitivity and specificity across studies at each score threshold. MetaDTA fits binomial bivariate random effects models (27, 28), which are generalized linear mixed effect models, using the glmer function from the lme4 R package (29). MetaDTA was used to estimate pooled sensitivity, specificity, positive likelihood ratio and negative likelihood ratio for each threshold. A separate meta-analysis was performed for each score at each threshold, with summary estimates of sensitivity and specificity plotted in ROC space, together with 95% confidence ellipses and 95% prediction ellipses. Summary ROC curves (showing the pooled results across thresholds) were also added to the plots showing study-level ROC curves. Positive predictive values (PPVs) and negative predictive values (NPVs) for each score threshold were calculated using a GAS prevalence of 24.1% in secondary care(6). We also performed sensitivity analyses, restricting the meta-analyses to studies that used throat culture as the reference standard.

Impact on antibiotic prescription

To aid interpretation of results, we considered a hypothetical cohort of 10,000 patients presenting at secondary care with pharyngitis. We assumed a prevalence of GAS of 24.1%, as has been reported for patients presenting to secondary care settings (6). We compared hypothetical scenarios in which a CPR (McIsaac or Centor score) is used as an initial triage test for RADT testing, to a baseline scenario in which all patients are given a RADT on presentation. In all scenarios, we assumed only individuals who received a positive RADT test were given antibiotics. In scenarios involving CPRs, we assumed that RADT tests were only performed if the initial CPR score was greater than or equal to each of thresholds 1, 2, 3 or 4. We assumed sensitivity of 88.4% and specificity of 91% for RADT (23) and based the sensitivity and specificity of the CPRs at each score on the results from meta-analyses restricted to studies using throat culture as the reference standard. RADT and CPR accuracy were assumed to be conditionally independent. We compared across scenarios (1) the number of GAS patients given antibiotics and (2) the number of non-GAS patients given antibiotics.

Study characteristics

Six studies have been conducted in the USA, two in Japan, and the rest in Egypt, Greece, Australia, Brazil, Croatia, Latvia, France and Pakistan (Table 1). Ten studies used both RADTs and throat cultures (30-35, 38, 39, 42, 43) as the reference standard: throat culture performed depending on RADT results (32, 35, 38, 43), or both tests performed (30, 31, 33, 34, 39, 42). Two studies (36, 41) only used RADT, and two studies (37, 40) only used throat cultures as the reference standard. Across Centor studies there were 7400 patients in total, of which Rich et al(43) and Rimoin and colleagues (44) contributed 3963 and 2472 patients respectively. A total of 4281 patients were included for the McIsaac score, with similar sample sizes across these eight studies.

Studies included both adults and children and also used different age ranges to classify children and adults – e.g. Hayakawa et al (40) defined adults as those aged 16 or above, while El-Ghany et al (30) defined adults as being over 18 years old.

Risk of bias assessment

Only two studies(44) explicitly mentioned the use of consecutive patient recruitment and were therefore assessed as low risk of bias for the patient selection domain. Seven out of eight McIsaac studies (Figure 2a) and three out of seven Centor studies (Figure 2b) were judged to be at unclear risk of bias overall due to patient recruitment method not being described. The study by DiMatteo and colleagues(38) was also judged at unclear risk of bias for using consecutively recruiting retrospective patients who received a RADT, as it was not described if all patients presenting to the hospital with pharyngitis received a RADT.

• Download figure
• Open in new tab

• Download figure
• Open in new tab

Figure 2a,b:

Risk of bias traffic light plots of the QUADAS-2 risk of bias assessment results for McIsaac studies (1a) and Centor studies (1b) produced using the robVis web application

Eight studies used at least two reference standards. These studies were considered low risk of bias for flow and timing because all patients had throat swabs taken at the same clinical encounter for both RADT and backup throat culture. Patient exclusion from analysis was only done for appropriate reasons, such as missing clinical or reference standard data, so flow and timing domain was mostly low risk. One study(42) was an exception and was of unclear risk of bias, because 4.8% of data were not included in the analysis due to lost laboratory data. We could not establish the exact nature of the patient data that was lost (e.g. whether all were from patients from a particular area/ socioeconomic background, etc.) and so we could not determine how this affected the risk of bias for the flow and timing domain.

Sensitivity and specificity of McIsaac and Centor scores

Sensitivity and specificity per study per threshold are shown for Centor score in Figure 3A and McIsaac score in Figure 3B. For McIsaac, Ezike et al also has two points lying just below the AUROC = 0.5 line, while Felsenstein et al has two points lying on this line

• Download figure
• Open in new tab

• Download figure
• Open in new tab

Figures 3a,b: Sensitivity versus 1-specificity (false positive rate) estimates per threshold per study plotted in receiver operating characteristic (ROC) space of the studies for Centor (3a) and McIsaac (3b) studies for all thresholds.

Studies are colour coded and different symbols are used to indicate different numerical thresholds. The solid black line represents the summary estimate from the meta-analysis. The black dot-dash line represents the gradient of a clinical prediction rule with 50% sensitivity and 50% specificity.

Centor and McIsaac score pooled sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) for a GAS prevalence of 24.1%, positive likelihood ratio (+LR) and negative likelihood ratio (-LR) are shown in Table 2a. Supplementary Figure 1A-D show results from each of the eight meta-analyses, including confidence and prediction ellipses. McIsaac score has higher summary sensitivity and lower summary specificity than the same Centor score thresholds (Table 2a). Visual comparison shows that the 95% confidence and prediction regions overlap across both scores (except for the 95% confidence region for threshold ≥3). The wide confidence intervals and correspondingly large confidence regions at most thresholds indicate high uncertainty in the summary estimates, and the much larger prediction regions indicate substantial heterogeneity across studies, for each threshold for both scores.

Results from sensitivity analyses restricting to studies that used throat culture as the reference standard are shown in Supplementary Table 1. Results were comparable between both analyses, with overlapping 95% Cis.

Impact on antibiotic prescription

The estimations of how many antibiotic prescriptions could be prevented with the use of McIsaac or Centor scores as a triage to RADT, compared with using RADT alone, to prescribe antibiotics and how many patients would not receive antibiotics who may need them are presented in Table 3, for McIsaac and Centor scores at each threshold.

The estimated number of false positive cases identified by the threshold step and the number of these who would be given antibiotics if a positive RADT is required is also shown, to appreciate the potential impact of using additional RADTs on unnecessary prescription.

Using CPRs as an initial triage step reduces the number of antibiotic prescriptions, but also reduces the number of GAS cases – who can benefit from the antibiotics – not receiving them. For example, in the baseline strategy of a RADT for everyone, 88% of GAS patients and 9% of non-GAS patients receive antibiotics. Following NICE treatment guidelines (Centor ≥3 as an initial triage for RADT testing), we estimate that only 51% of GAS patients and 4% of non-GAS patients would receive antibiotics. In our hypothetical cohort of 10,000 patients with 24.1% GAS prevalence, this translates to a reduction in number of antibiotic prescriptions from 2813 to 1569 (reduction of 1244), but we estimate that 893 of these people not prescribed would in fact have GAS pharyngitis. Similarly for the Danish (McIsaac ≥2) and German (McIsaac ≥3) treatment guidelines, we estimate that 84.8% and 72.4% of GAS pharyngitis patients and 7.9% and 5.3% of non-GAS patients would receive antibiotics, with a reduction in the number of prescriptions by 172 and 666 respectively, where 87 and 386 cases of these avoided prescriptions would have had GAS pharyngitis.

Main findings

McIsaac and Centor scores aim to accurately distinguish between cases of GAS and non-GAS pharyngitis in patients presenting with pharyngitis to reduce unnecessary use of antibiotics through better targeted prescription. After reviewing studies set in secondary care investigating the ability of McIsaac and Centor scores to distinguish between GAS pharyngitis and non-GAS pharyngitis, we note that there is little separating McIsaac and Centor scores once it is appreciated that McIsaac score of 0 or 1 is equivalent to Centor score of 0, McIsaac score of 2 is equivalent to Centor score of 1 and so on (Tables 2 & 3). We find that different treatment thresholds need to be used for these scores.

We estimate that Centor scores of ≥3 and 4 have sensitivities of 58% and 29%, respectively, compared to McIsaac score ≥2 and ≥3 which have sensitivities of 96% and 82%, respectively. Current NICE guideline recommendations therefore potentially misses a large cohort of GAS positive pharyngitis patients.

Using clinical scores either alone or as a triage to RADT testing leaves a sizable proportion of GAS positive patients missing out on antibiotics that could have benefited from them. RADT use on threshold positive patients significantly limits the incorrect treatment of false positive cases, particularly at low thresholds, and thereby prevents unnecessary use of antibiotics. However, the low sensitivity of CPR scores when implemented at higher cutoffs as a triage step leads to a significant proportion of GAS cases not receiving a RADT test and therefore not receiving treatment. Therefore, RADT follow up of patients with low CPRs is likely to have the most clinical value.

Strength and limitations

Our systematic review was carried out according to the best practice recommendations for review conduct. Our search strategy was comprehensive, study selection and data extraction were performed by one reviewer (AK or CE) and checked by the other, and the review was reported according to PRISMA-DTA guidelines.

However, we believe that there are significant limitations to the evidence base we examined in this review. Firstly, studies used heterogeneous age ranges for adults and children, preventing stratified analyses based on age groups. Secondly, several included studies solely used RADTs as the reference standard instead of more accurate throat cultures – although our sensitivity analyses restricting to studies using throat culture as the reference standard provided similar results. Thirdly, RADTs or throat cultures do not distinguish between GAS commensal carriage and GAS infection, meaning that identifying true positive GAS pharyngitis where the causative organism is GAS is difficult and the number of true positives and false positive is uncertain.

Comparisons with existing literature

We compared our findings to those from existing literature (Table 2b). Compared to the original Centor study(10) conducted on adults presenting to an urban emergency room and the original McIsaac study(11) conducted on patients aged 3 to 76 presenting to a primary care centre, we report higher PPVs for thresholds Centor 0-3 and lower PPV for Centor = 4. For McIsaac score we report higher PPVs for thresholds 0-3, and lower PPV for threshold ≥4.

Fischer Walker and colleagues(45) compared multiple CPRs (Breese score, Wald score, Centor score, McIsaac score, Abu Reesh score, Steinhoff score, WHO score) on the same cohort of children presenting to the hospital with sore throat and pharyngeal erythema to predict streptococcal pharyngitis. Their results show lower McIsaac score sensitivity, both higher and lower Centor score sensitivity and higher specificity for both scores across equivalent thresholds, potentially due the study population being children exclusively.

Willis and colleagues evaluated Centor and McIsaac score in primary care (17); their results were comparable to this review’s findings (Table 2b): The 95% confidence intervals for McIsaac scores 0-4 and Centor scores 1-4 overlap between this review and Willis and colleagues’ review. They concluded that for both tests, “a score of ≤0 may be sufficient to rule out infection” but that higher scores could result in overuse of antibiotics and therefore justifies the implementation of additional point of care tests to confirm the diagnosis.

The Primary care Streptococcal Management (PRISM) study by Little and colleagues(46) evaluated the use of clinical scores, RADTs and delayed prescriptions in the context of pharyngitis. They found that the tested RADTs were 100% specific for GAS, but sensitivity ranged from 62% to 95%. They developed and tested the FeverPAIN score, which is based on fever, purulence, attend rapidly (within 3 days), severe inflammation and no cough or coryza. In the two primary care study cohorts included in PRISM with prevalence of 34% and 40% of A, C or G beta-haemolytic streptococci, FeverPAIN had a higher bootstrapped estimated AUROC (0.73 and 0.71) compared to the estimates from Willis and colleagues for Centor score, 0.6888 (95% CI 0.653 - 0.724), and McIsaac score, 0.7052 (95% CI 0.624 - 0.778), in primary care populations with a prevalence of 26.4% (range: 4.7%–42.0%) across Centor score studies, and 23.0% (range: 12.7%–44.8%) for McIsaac score studies (17). We did not identify any studies that have used FeverPAIN in a secondary care population. Little et al also found that symptom management and antibiotic use wise there were no reason to justify use of RADT over clinical score alone from a time and economic cost perspective and that both interventions reduced antibiotic use.

Recommendations for practice

Risks of treating false negative and false positive cases must be considered when setting a treatment threshold: the former could lead to patients developing complications while the latter could lead to over-prescription of antibiotics and eventually increased anti-microbial resistance. The diagnostic pathway must be both highly sensitive and specific for GAS pharyngitis to treat true positive cases and minimise unnecessary antibiotic prescriptions and medicalisation of relative minor symptoms. We cannot recommend the use of one score over the other to triage patients presenting to hospitals with pharyngitis.

Adequate safety netting by clinicians would ensure that patients return should their symptoms worsen. This may allow the use of higher, less sensitive score thresholds, which exclude more GAS pharyngitis patients from receiving antibiotics at the time primary presentation while minimising potential harm to patients who miss out on receiving antibiotics, thus decreasing the number of false positives treated with antibiotics and limits medicalisation of relative minor symptoms. We cannot recommend a sufficiently inclusive threshold for both these scores to decide whether to treat patients with antibiotics or not when we regard the impact on antimicrobial stewardship with the data analysed in this review. Thus, we recommend that current antibiotic prescription guidelines are followed and clinicians safety net patients to return on worsen of/ non-resolution of symptoms.

Due to insufficient clinical effectiveness and economic evidence on the effect of using RADT after a low McIsaac or Centor on antibiotic prescriptions, we cannot make a strong recommendation for this strategy. However, our analysis is promising and so we recommend further research. Both scores have low sensitivities at high thresholds, hence we recommend that if these scores are used as a triage to RADT testing then a low threshold should be implemented for RADT testing.

Conclusions

Centor and McIsaac scores are equally ineffective at triaging patients who need antibiotics presenting with pharyngitis at hospitals. At high thresholds too many true positive cases are missed while at low thresholds too many false positives are identified and treated leading to over-prescription of antibiotics. The former may be compensated by proper safety netting to seek help if symptoms worsen and/or delayed prescriptions by clinicians. As shown here, additional testing of positive threshold cases by RADT decreases false positives at low thresholds but at an economic cost and at the cost of further decreasing the number of true positive cases treated at high thresholds. For now, current guidelines should be followed but we recommend further study of the clinical and economic impact of RADT testing of patients with low CPR scores to determine antibiotic prescription and similar evaluation of using FeverPAIN in populations of patients presenting to hospitals. Better CPRs may be developed by combining signs and symptoms with test results, such as RADT or inflammation markers. However, the cost-effectiveness of these strategies needs to be considered carefully because they could lead to an increase in testing.