ABSTRACT
Background The specificity of molecular methods for the detection of Streptococcus pneumoniae carriage is under debate. We propose a procedure that increases the accuracy of molecular detection of live pneumococci in polymicrobial respiratory samples.
Methods Culture and qPCR methods were applied to detect S. pneumoniae and pneumococcal serotypes in 1549 nasopharyngeal samples collected in the Netherlands (n=972) and England (n=577) from 946 toddlers and 603 adults, and in paired oropharyngeal samples collected exclusively from 319 Dutch adults. Samples with no live pneumococci isolated at primary diagnostic culture yet generating pneumococcus-specific signal in qPCRs were re-examined with a second, qPCR-guided culture. Optimal Cq cut-offs for positivity in qPCRs were determined via receiver operating characteristic (ROC) curve analysis using isolation of live pneumococci from the primary and qPCR-guided cultures as reference.
Results Detection of S. pneumoniae and pneumococcal serotypes with qPCRs in cultured (culture-enriched) nasopharyngeal samples exhibited near-perfect agreement with conventional culture (Cohen’s kappa: 0.95). Molecular methods also displayed increased sensitivity of detection for multiple serotype carriage. Among paired samples from adults, the sensitivity of S. pneumoniae detection in primary nasopharyngeal plus oropharyngeal cultures was significantly lower compared with molecular detection in both culture-enriched samples together (p<0.0001) and also in culture-enriched oropharyngeal samples alone (p<0.05).
Conclusions The sensitivity of S. pneumoniae carriage surveillance can be greatly improved by complementing conventional culture with qPCR and vice versa. The specificity of molecular methods for the detection of live pneumococci can be enhanced by incorporating statistical procedures based on ROC curve analysis. The procedure we propose improves detection of S. pneumoniae carriage in adults in particular and enhances specificity of serotype carriage detection.
INTRODUCTION
Streptococcus pneumoniae (pneumococcus) is the most common aetiological agent of invasive bacterial disease [1] and of community-acquired pneumonia of bacterial aetiology [2]. Despite being vaccine-preventable, pneumococcal disease remains among the leading causes of death in childhood [3] as available vaccines target only a various subsets of only twenty-four from ca. 100 of the known serotypes. Due to high pneumococcal carriage rates [4, 5] children are considered the primary reservoir of pneumococcus and the main drivers of transmission and infections in any population [6–8]. Children are also the primary group targeted with pneumococcal vaccination [9].
With pneumococcal vaccines protecting not only against disease but also against colonization, carriage is now an accepted endpoint in vaccination studies [10]. Following the introduction of pneumococcal conjugate vaccines (PCV), of which ten-valent (PHiD-CV, GSK) and thirteen-valent (PCV13, Pfizer) are currently marketed, epidemiological surveillance of carriage became an essential tool for monitoring the direct and indirect (herd protection) effects of vaccination. Surveillance of carriage is also used to detect the emergence and track expansion of non-vaccine serotypes, the phenomenon described as vaccine-induced serotype replacement [11]. Finally, carriage studies are instrumental in monitoring serotype-associated invasiveness [12].
To understand the trends in pneumococcal epidemiology and guide strategies for PCV use, it is critical to establish methodology for S. pneumoniae carriage detection that is sufficiently sensitive across all ages [13, 14]. However, conventional culture, which is considered to be the gold standard method in carriage detection, is not suitable for detection of multiple serotype carriage [15] and lacks the sensitivity when used to detect S. pneumoniae in age groups other than children [16]. There is also evidence that testing solely a single site within the upper airways reduces sensitivity of carriage detection [17–19].
Molecular methods have largely improved the sensitivity of S. pneumoniae detection and have made multiple serotype carriage detection feasible [4, 20]. Using assays developed by our groups and others, we have tested the serotype composition of respiratory samples from children and adults and have demonstrated an under-detection of S. pneumoniae and of individual pneumococcal serotypes by culture when compared with molecular methods [7, 18, 21–24]. However, some caution that molecular methods exhibit poor specificity due to the presence of pneumococcal genes among commensal streptococci [25–27]. It could also be argued that molecular detection is unable to discriminate between live bacteria and presence of relic DNA [28].
Here, we outline the protocol that combines conventional culturing with pneumococcus-specific qPCRs and employs statistical procedures to interpret the molecular results and enhances the specificity of the molecular methods. We show that molecular methods applied to nasopharyngeal samples demonstrate near-perfect agreement with primary culture and yet the sensitivity of S. pneumoniae carriage surveillance can be greatly enhanced by complementing conventional culture with qPCRs and vice versa. We also show that testing nasopharyngeal samples alone leads to underestimation of pneumococcal carriage in adults.
MATERIALS AND METHODS
Study Design and Ethics Statement
Pneumococcal carriage was investigated in two cross-sectional prospective observational studies conducted in 2015/2016 in community-dwelling individuals in the Netherlands [29] and in England [30]. The study conducted in the Netherlands was approved by the Medical Ethics Committee Noord Holland (NTR5405 on http://www.trialregister.nl). The study conducted in England was approved by the NHS Health Research Authority and the London Fulham Research Ethics Committee (reference 15/LO/0458) and was registered on clinicaltrials.gov (reference NCT02522546). Both studies were conducted in accordance with Good Clinical Practice and the Declaration of Helsinki.
Sample Collection and Laboratory Processing
Respiratory samples were collected in the Netherlands between October 2015 and March 2016 in the study on carriage of respiratory bacterial pathogens coordinated by National Institute of Public Health and the Environment. Nasopharyngeal samples were collected in children aged 24 months (+1 month) vaccinated with PHiD-CV according to “2 primary + 1 booster” (2p+1b) dose schedule, children aged 44 to 49 months vaccinated with PHiD-CV in 3p+1b schedule, and parents of the 24-month-old children (one parent per child). Oropharyngeal swabs were also collected from Dutch adults [29, 31]. The English study took place between July 2015 and June 2016 and was conducted by the National Vaccine Evaluation Consortium which included Public Health England [30]. In England, nasopharyngeal samples were collected in children aged 1-5 years vaccinated with PCV13 according to a 2p+1b schedule and in their household contacts (adults aged >20 years).
All samples were obtained according to World Health Organization standard procedures [4]. Immediately after sampling, swabs in the Netherlands were placed in liquid Amies transport medium (Copan, Brescia, Italy) and within 8 hours transported to the diagnostic laboratory. In England swabs were placed in skim-milk, tryptone, glucose and glycerol (STGG) broth and delivered to a diagnostic laboratory within 48 hours (Table S1).
Methodological differences between Streptococcus pneumoniae carriage studies conducted in the Netherlands and in England.
Conventional Culture
Approximately 50 µl of specimen was used as an inoculum in detection of pneumococci by conventional culture (primary culture). In the Netherlands samples were cultured on SB7-Gent agar selective for streptococci (BA-GENT, Oxoid, Badhoeve Dorp, The Netherlands) and on non-selective Columbia blood agar (CBA, Oxoid). In England samples were cultured on Streptococcus-selective Blood Agar (COBA, Oxoid, Basinstoke, United Kingdom) and CBA (Oxoid). Remaining of transport media were stored frozen at −70°C. In the Netherlands, samples were aliquoted into 200 μl volumes, and a single aliquot was supplemented with glycerol (10% v/v final concentration) prior to freezing.
After overnight incubation at 37°C and 5% CO2, cultures were screened for pneumococcus-like colonies to be re-cultured. Once screened, in the Netherlands all colony growth was harvested from BA-GENT plate into 10% glycerol in brain heart infusion broth (Oxoid) [24]. In England, all colony growth was harvested from any plates containing alpha-haemolytic colonies into PBS, centrifuged, the supernatant removed, and any growth stored as a pellet [30]. These samples were considered culture-enriched (CE) for pneumococci and stored at −80°C. Cultured strains (one per sample, but more if distinct pneumococcal morphotypes were apparent) were serotyped by Quellung method in the Netherlands [29] and with the previously described PneumoCaT bioinformatic pipeline in England that was supplemented with slide agglutination when required [30, 32].
Molecular Detection of Streptococcus pneumoniae
Molecular detection of S. pneumoniae in respiratory samples was conducted as described previously [24]. For this, Amies medium, STGG medium, and culture-enriched samples were shipped on dry ice from primary diagnostic sites to the study central laboratory in the Netherlands. There, pellets of culture-enriched samples collected in England were reconstituted to the original volume. Next, nucleic acids were extracted from 100 µl of medium using the DNeasy Blood and Tissue Kit (Qiagen) and eluted into 200 µl. These extractions represented minimally processed samples. For culture-enriched samples, 100 µl of a bacterial growth harvest was centrifuged for 2 min at 14,000 ×g, the pellet was resuspended with 90 µl of TE buffer (20 mM Tris-HCl [pH 8.0], 2 mM EDTA) and incubated for 15 min at 95°C. Next, 90 µl of lysis buffer (20 mM Tris-HCl [pH 8.0], 2 mM EDTA, 2.4% Triton X-100 and 40 mg/ml lysozyme) was added, and the samples were processed as above. Pneumococcal DNA was detected via single plex qPCR using primers and probe targeting sequences within genes coding for the pneumococcal iron uptake ABC transporter lipoprotein PiaB [18], and for the major pneumococcal autolysin LytA [33] by testing 5.5 μl DNA from minimally processed or 1.0 μl of culture-enriched samples.
Molecular Detection of Pneumococcal Serotypes
DNA extracted from culture-enriched harvests was used to determine the serotype composition of nasopharyngeal and oropharyngeal samples using a panel of primers and probes [34–36], targeting all serotypes covered by pneumococcal vaccines available either in the Netherlands or in England at the time of sample collection (PHiD-CV, PCV13 and 23-valent polysaccharide vaccine, PPV23, Merc Sharp Dohme) except for serotype 2. The panel also covered selected non-vaccine serotypes (serotypes 6C, 6D, 7A, 10B, 12A, 12B, 15A, 15C, 15F, 16F, 21, 22A, 23A, 23B, 33A, 34, 35B, 35F, 37 and 38). However, with certain assays targeting more than one serotype it was not possible to distinguish between serotypes 6A and 6B; 6C and 6D; 7A and 7F; 9A, 9L, 9N and 9V; 10A and 10B; 11A and 11D; 12A, 12B and 12F; 15A, 15B, 15C and 15F; 33A, 33F and 37; 35B and 35C when detected with qPCR. A sample pooling strategy was employed when testing for serotypes in order to reduce the number of serotype-specific qPCRs. For this, samples generating any signal below Cq 40 for either piaB or lytA were pooled in groups of five to be tested [7] and all remaining samples were pooled in groups of ten. Samples negative for piaB and lytA were tested to assess specificity of serotype/serogroup-specific assays [7]. Samples from pools generating a signal for a particular serotype/serogroup were tested individually. In the Netherlands, the PCRs were performed on the LightCycler 2 platform and in England they were performed on the QuantStudio 7 platform, using identical PCR conditions.
qPCR-guided Recovery of Live S. pneumoniae Strains
Culture-enriched samples classified as negative in primary diagnostic culture but positive by qPCR were revisited with culture in a second attempt to isolate live pneumococci. For this, CBA plates were inoculated with 100 µl of 10-1-10-3 and 10-3-10-5 dilution of culture-enriched nasopharyngeal and culture-enriched oropharyngeal samples, respectively, and incubated at 37°C and 5% CO2. Pneumococcus-like colonies were individually tested in qPCR for piaB and lytA and confirmed to be S. pneumoniae based on susceptibility to optochin.
Statistical Analysis
Data analysis was performed in GraphPad Prism software version 9.1.0 and R version 4.1.0. Receiver operating characteristic (ROC) curve analysis was performed using the “cutpointr” R package to validate qPCR results with culture results (primary culture plus qPCR-guided culture). Maximum Youden index values, the sum of sensitivity and specificity minus one, were estimated via bootstrapping (n=1000) on piaB and lytA qPCR data from respiratory samples to determine optimal cut-off values for qPCR detection [37]. Two-way mixed effects intraclass correlations (ICCs) [38] and Bland-Altman plots [39] were used to evaluate agreement for quantitative results using the “ïrr” and “blandr” R packages, respectively. Carriage rates were compared with McNemar’s test unless otherwise stated. Estimates for accuracy of diagnostic tests between methods (subgroups) were compared with a test of interaction [40]. A p value of <0.05 was considered significant. Diagnostic test parameters (predictive values, sensitivity, and specificity) were calculated using an in-house made R function (https://github.com/wmiellet/test-comparison-R), and 95% confidence intervals were calculated with Wilson-Brown score. Cohen’s kappa, a measure of chance-corrected agreement, and its 95% confidence interval was calculated as described by McHugh [41] and ratios of ≤ 0, 0.01-0.20, 0.21-0.40, 0.41-0.60, 0.61-0.80 and >0.81 were interpreted as exhibiting poor, slight, fair, moderate, substantial, and near-perfect agreement, respectively [42]. For comparison between serotyping by culture and by qPCR, analysis was limited to qPCR-targeted serotypes. For serogroup-specific qPCR assays a result was considered congruent when a serogroup was detected in qPCR that matched the serogroup of the serotype detected by culture.
Assessment of method’s inter-laboratory reproducibility
One-hundred seventy-six nasopharyngeal samples collected in England and n=59 oropharyngeal samples collected in the Netherlands have been selected to evaluate the inter-laboratory reproducibility of molecular methods. For this, an aliquot of minimally processed STGG nasopharyngeal sample, a fresh preparation of culture enriched nasopharyngeal sample from a subset of the English study samples and culture-enriched oropharyngeal samples from a subset of the Dutch study samples were tested at the study site in England, as described above. Results for paired samples were compared between centres by calculating the percent agreement and Cohen’s kappa. Quantitative results of both laboratories were also compared by calculating an intraclass correlation coefficient (ICC) and by comparing results in Bland-Altman plots. Carriage rates between both laboratories were compared using Cohen’s kappa.
RESULTS
We evaluated the performance of conventional and molecular methods applied to detect S. pneumoniae and pneumococcal serotypes in 1549 nasopharyngeal samples collected from 946 children aged 1-5 years (n=653 in the Netherlands and n=293 in England) and from 603 adults (n=319 in the Netherlands and n=284 in England), and on oropharyngeal samples from 319 adults in the Netherlands. All PCR results shown were generated by testing the Dutch and English samples in the Dutch laboratory, except were stated otherwise.
Detection of Streptococcus pneumoniae
When analysing culturing results without the addition of PCR-guided culturing, live S. pneumoniae was isolated from 29% (445/1549) of primary cultures of nasopharyngeal swabs (Fig. 1A), with significantly higher fraction of samples from children (45% or 422/946) compared with adults (4% or 23/603) positive for pneumococcus (Pearson’s chi-square, p<0.0001). Among 319 adults sampled in the Netherlands (Fig. 1B), live pneumococcus was significantly more often cultured from nasopharyngeal (n=15 or 5%) compared with oropharyngeal (n=3 or 1%) samples (p<0.01).
(A) Matrix layout for observed intersections of the Streptococcus pneumoniae detection procedures applied to n=1549 nasopharyngeal samples, sorted by size. (B) Matrix layout for observed intersections of the S. pneumoniae detection procedures applied to nasopharyngeal and oropharyngeal samples from n=319 adults sampled in the Netherlands, sorted by size. Circles in the matrix indicate sets that are part of the intersection, and circles are coloured by sample type with nasopharyngeal and oropharyngeal samples coloured in green and blue, respectively. The bar diagram displaying the intersection size is coloured using the same scheme, with orange indicating an intersection that represents positivity in both sample types.
Next, we conducted qPCRs on minimally processed and culture-enriched samples. To enhance the specificity of detection we used a “two-to-tango” approach by quantifying piaB and lytA genes and considering a sample to be positive for S. pneumoniae when both targets were detected. When applying an arbitrary quantification cycle (ACq) of <40 Cq as criterium for positivity altogether 583 nasopharyngeal swabs (38% of 1549) were identified as positive for pneumococcus (Fig. 2A and 2B). The fraction of positive samples was significantly higher among minimally processed compared with culture-enriched swabs (561 or 36% versus 479 or 31% of 1549; p<0.0001). In line with results of primary diagnostic culture, here too the proportion of positive samples was higher in children compared with adults (Pearson’s chi-square, p<0.0001) whenever nasopharyngeal swabs were tested minimally processed (56% or 528/946 versus 5% or 33/603) or culture-enriched (48% or 454/946 versus 4% or 25/603). However, contrary to results of primary diagnostic culture, among adults significantly larger proportions of oropharyngeal compared with nasopharyngeal samples (p<0.0001) have been classified as positive for pneumococcus with qPCR whenever tested minimally processed (26% or 83/319 versus 6% or 19/319) or culture-enriched (18% or 58/319 versus 6% or 18/319).
Scatter plots (A, B, E and F) displaying correlation between cycle threshold (Cq) from real-time PCR assays targeting the Streptococcus pneumoniae genes piaB and lytA for minimally processed (A and E) and culture-enriched (B and F) nasopharyngeal (A and B) and oropharyngeal (E and F) samples followed by the Bland-Altman plots (C, D, G and H) displaying agreement between piaB and lytA for samples positive with minimally processed (C and G) and culture-enriched (D and H) ROCdCq criteria in nasopharyngeal (C and D) and oropharyngeal (G and H) samples. The shaded grey areas mark the 95% confidence interval of the upper limit of agreement, bias and lower limit of agreement. The Bland-Altman plots show the absence of systematic bias between piaB and lytA because the line of equality (grey solid line) is within the limits of agreement. Bias between piaB and lytA is more often observed in high Cq samples. Primary culture-positive and qPCR-guided culture-positive samples are indicated with red and green dots, respectively. Culture-negative samples are indicated with blue dots. In panels A, B, E and F black dashed lines indicate the arbitrary Cq criterium (<40 Cq), green dashed lines indicate ROCdCq criteria derived by validating qPCR results with primary and qPCR-guided cultures results, and red dashed lines in panels A and B indicate ROCdCq criteria derived by validating qPCR results with primary culture alone.
Next, samples negative for pneumococcus in primary diagnostic cultures yet generating signal in qPCRs for piaB and lytA were cultured again. This qPCR-guided culture effort increased the number of samples from which live pneumococci was cultured, in relatively small albeit significant increase of 3% (from 445 vs. 460 of 1549; p<0.0001) for nasopharyngeal (Fig. 1A, 2A, and 2B) and 666% increase (from 3 to 23 of 319; p<0.0001) for oropharyngeal samples (Fig. 1B, 2E, and 2F).
To validate qPCR results using culture and to further enhance the specificity of qPCR methods for the detection of live pneumococci we performed receiver operating characteristic (ROC) curve analysis and identified Cq cut-off values that yielded maximum Youden indices (Table 1). Reliability of positive qPCR results was assessed with Bland-Altman plots (Fig. 2C and 2D and Fig. 2G and 2H and supplementary information Table S2) and by calculating the intraclass correlation coefficient between piaB and lytA. Samples with Cq values below the ROC-derived Cq criterium (ROCdCq) demonstrated excellent agreement between piaB and lytA while samples with a Cq value above the ROCdCq threshold displayed poor agreement (Table S2). Using the ROCdCq criterium we classified 33% (512/1549) of minimally processed nasopharyngeal samples as positive for pneumococcus (Fig. 1A). It included 52% (489/946) samples from children and 4% (23/603) from adults. For culture-enriched samples application of ROCdCq resulted in 30% (469/1549) classified as positive including 47% (447/946) samples from children and 4% (22/603) from adults (Fig. 1A). With ROCdCq, the adults in the Netherlands were again significantly more often positive for pneumococcus in oropharyngeal compared to nasopharyngeal samples whenever tested minimally processed (19% or 59/319 versus 5% or 15/319; p<0.0001) or culture-enriched (12% or 39/319 versus 5% or 15/319; p<0.01) (Fig. 1B).
Results of an intraclass correlation coefficient (ICC) reliability analysis for ROCdCq-based criteria in a subset of samples classified as positive for S. pneumoniae according to the ACq criterium of <40.
Optimal qPCR cycle threshold Cq and corresponding parameters for Streptococcus pneumoniae carriage detection in nasopharyngeal (n=1549) and oropharyngeal (n=319) samples. Results from qPCR were validated in a receiver operating characteristic curve analysis with culture as reference.
Comparison of Molecular Methods to Culturing Live S. pneumoniae
We compared the diagnostic accuracy of detection methods using the combined results of primary diagnostic and qPCR-guided culturing as reference and applying ROCdCq criteria for positivity in qPCRs (Fig. 1A). For nasopharyngeal samples alone (Table 2), molecular detection of pneumococcus displayed near-perfect, and substantial agreement to the reference for culture-enriched and minimally processed samples, respectively. This represented significantly reduced agreement for minimally processed compared with culture-enriched samples (p<0.0001). Molecular detection in minimally processed samples identified significantly more samples positive for pneumococcus compared with culture-enriched samples from children (p<0.0001) but not adults (p=1), the difference we attributed to low number of positive nasopharyngeal samples among collected from adults. Exclusively for children, molecular detection of pneumococcus in culture-enriched samples demonstrated significantly increased sensitivity (p<0.001) and specificity (p<0.0001) compared with molecular detection in minimally processed samples.
The accuracy of Streptococcus pneumoniae detection in n=1549 nasopharyngeal samples from children (n=946) and adults (n=603) tested using conventional diagnostic culture and molecular methods applied to DNA extracted from minimally processed and culture-enriched samples and applying ROCdCq thresholds for a sample positivity in qPCRs. Measures of diagnostic accuracy were calculated by comparing the number of detected samples positive per method with the number of (n=460 for all individuals, and n=437 and n=23 for nasopharyngeal samples from children and adults, respectively) individuals positive for S. pneumoniae based on isolation of live pneumococcus either from the primary diagnostic or qPCR-guided culture.
For paired nasopharyngeal and oropharyngeal samples from adults, the isolation of S. pneumoniae in primary or qPCR-guided culture in either nasopharyngeal or oropharyngeal swab was used as the reference in method accuracy analysis (Table 3). Although the sensitivity of S. pneumoniae detection with primary nasopharyngeal culture was significantly higher compared with corresponding values for primary oropharyngeal cultures (p<0.001), every detection method when applied to a single sample type displayed only slight to moderate agreement to the reference. Moreover, the sensitivity of S. pneumoniae detection with either nasopharyngeal and oropharyngeal primary cultures was significantly lower compared with molecular detection in culture-enriched oropharyngeal samples (p<0.05) and also when compared with molecular detection with either culture-enriched sample of either type (p<0.0001). To this extent, molecular detection in culture-enriched oropharyngeal plus nasopharyngeal samples displayed the highest agreement to the reference out of all other evaluated approaches (Table 3).
The accuracy of Streptococcus pneumoniae detection in paired nasopharyngeal and oropharyngeal samples from n=319 adults tested using molecular methods applied to DNA extracted from minimally processed and culture-enriched samples and applying ROCdCq thresholds for a sample positivity in qPCRs. Measures of diagnostic accuracy were calculated by comparing the number of detected samples positive per method with the overall number of n=37 individuals identified as carriers of S. pneumoniae based on isolation of live pneumococcus either at the primary diagnostic or qPCR-guided culture and either from nasopharyngeal or oropharyngeal sample.
Comparison of Serotype Carriage Detection Methods
With conventional culture (without PCR-guided additional culture), 29% (445/1549) of nasopharyngeal samples including 45% (422/946) from children and 4% (23/603) from adults were positive for a serotype (nontypeable pneumococci excluded), as already described above. Next, we assessed the accuracy of molecular methods when applied to detect carriage of pneumococcal serotypes. Figure S1 depicts results of serotype detection in culture-enriched samples tested with serotype-specific qPCR assays. None of the samples generated any signal in qPCRs targeting serotype 1, and 23F and serogroup 18, nor was positive for any of these serotypes by culture. The assays targeting serotypes 4, 5 showed a lack of specificity and the assay targeting serotype 17F showed lack of sensitivity when applied to both, nasopharyngeal and to oropharyngeal samples. Results of these three assays were excluded from analysis.
Four-hundred and twenty-three nasopharyngeal samples were positive for one or more serotypes targeted in qPCRs either by culture (n=393 samples) or with molecular methods (n=411 samples) and applying ROCdCq criterium for positivity for a serotype. It included 42% (395/946) samples from children and 3% (16/603) from adults. Altogether, there were n=479 serotypes carriage events detected by testing nasopharyngeal samples with either conventional culture or molecular methods (Table S3). For serotypes targeted by qPCRs detected in nasopharyngeal samples the results of molecular detection displayed excellent agreement (ICC 0.93, 95% CI 0.92-0.94) with piaB and lytA Cqs (Fig. 3) and, with exception of serotype 3, almost perfect agreement (Cohen’s kappa >0.81) with isolation of live strain of a particular serotype from nasopharyngeal swab (Fig. 4). Also, results of serotype detection in oropharyngeal samples from adults displayed excellent agreement with piaB and lytA Cqs (ICC 0.96, 95% CI 0.92-0.98) (Fig. S2). Finally, there was near-perfect agreement between overall serotypes carriage events detected by qPCR compared with detected by culture (Table 4).
Results of molecular method based quantification of serogroups/serotypes-specific sequences in DNA from culture-enriched nasopharyngeal and oropharyngeal samples. Individual scatter plots depict results of a single serotype-specific or serogroup-specific qPCR assay as labelled above the panel. Each symbol represents an individual sample: pale-red dots represent nasopharyngeal samples from children; blue triangles and green triangles represent nasopharyngeal and oropharyngeal samples from adults, respectively. Sample was classified as positive for a serotype/serogroup with molecular method when the signal detected by qPCR for piaB (X-axis) and serotype/serogroup (Y-axis) were both below the ROCdCq cut-off threshold of 30 Cq. Only samples that generated a signal of Cq<40 in a particular serogroup/serotype-specific qPCR are depicted. Symbols of Cq >40 for piaB depict individual samples or pools of samples negative for pneumococcus by qPCR. None of the samples generated any signal in qPCRs targeting serotype 1, serotype 23F and serogroup 18. In addition, no samples were identified as positive for serogroup 12 according to study criteria.
Scatter plot displaying correlation between cycle threshold (Cq) from real-time PCR (qPCR) assays targeting the Streptococcus pneumoniae piaB and serotype/serogroup specific signal detected with qPCR for culture-enriched (CE) oropharyngeal samples from adults classified positive for S. pneumoniae according to ROCdCq criterium (green dashed lines). Dots depicts serotypes detected with qPCR. Dots are color-coded according to serotypes/serogroup targeted in an assay (see legend).
Number of serotypes/serogroups detected with any method.
Scatter plots (A-C) displaying correlation between cycle threshold (Cq) from real-time PCR (qPCR) assays targeting the Streptococcus pneumoniae piaB and serotype/serogroup specific signal detected with qPCR for nasopharyngeal samples classified positive for S. pneumoniae according to ROCdCq criterium (green dashed lines). Bland-Altman plot (D) displaying agreement between piaB and dominant serotype detected by qPCR. Each dot represents an individual serotype carriage event detected by qPCR in culture-enriched nasopharyngeal sample (A-C). Dots in panel A depicts serotypes detected with qPCR. Dots are color-coded according to serotypes/serogroup targeted in an assay (see legend). Colour dots in panel B depict subdominant serotypes detected qPCR while grey dots mark dominant serotypes. Red dots in panel A and B depict samples not generating any signal in serotype/serogroup-specific qPCRs or with the signal of Cq higher than the ROCdCq threshold, hence classified as negative for a serotype. In panels C and D green dots mark samples with congruent serotype between culture and molecular methods and orange dots mark samples with noncongruent result. In panel D shaded grey areas mark the 95% confidence interval of the upper limit of agreement, bias and lower limit of agreement. The solid line grey line marks the line of equality.
Forest plot displaying the point estimate and 95% confidence of intervals sensitivity (A), specificity (B), concordance (C) and Cohen’s kappa (D) for molecular diagnostic tests applied to culture-enriched samples when compared to isolation of S. pneumoniae strains of particular serotype/serogroup from nasopharyngeal swabs collected in the study. Graphs displayed results for serotypes/serogroups that have been cultured from >5 nasopharyngeal samples.
The accuracy of Streptococcus pneumoniae serotype detection in n=1549 nasopharyngeal samples from children (n=946) and adults (n=603) tested using molecular methods applied to DNA extracted culture-enriched samples. Measures of diagnostic accuracy were calculated by comparing the numbers of serotype carriage events detected with molecular methods with n=393 samples from which serotypes were cultured that were targeted by serotype-specific qPCR assays.
Multiple-serotype carriage events were significantly more often detected using molecular methods compared with culture (p<0.0001). Furthermore, despite a limited number of different serotypes tested by qPCR, observed multiple-serotype carriage frequencies were not significantly different from expected frequencies based on molecular detection on culture-enriched NP samples (11% versus 9%, respectively; one proportion Z-test, p=0.2669) unlike detection of multiple-serotype carriage by culture which significantly underestimated expected frequencies (2% versus 9%, respectively; one-proportion Z-test, p<0.0001).
Interlaboratory Comparison of Methods
To evaluate the reproducibility of molecular methods and to assess the agreement in laboratory results between the Netherlands and England we processed a subset of culture-enriched samples from both countries in both laboratories. Results for piaB and lytA qPCRs demonstrated good reliability between both laboratories (Table S4). We observed near-perfect agreement identifying culture-enriched samples as positive for pneumococcus with molecular methods, and substantial agreement for minimally processed nasopharyngeal samples (Table S5). For culture-enriched nasopharyngeal samples we evaluated agreement between both laboratories for serotype carriage detection by qPCR. Overall, near-perfect agreement was observed (Cohen’s kappa 0.82, 95% CI 0.74 – 0.90).
Optimal qPCR cycle threshold Cq and corresponding parameters for Streptococcus pneumoniae carriage detection in a subset of nasopharyngeal (n=176) and oropharyngeal (n=100) samples processed independently in England and in the Netherlands. Results from qPCR were validated in a receiver operating characteristic curve analysis with culture as reference*.
The accuracy of Streptococcus pneumoniae detection in n=176 nasopharyngeal samples and n=59 oropharyngeal tested independently in both countries using molecular methods applied to DNA extracted from minimally processed and culture-enriched samples and applying ROCdCq thresholds for a sample positivity in qPCRs. Measures of diagnostic accuracy were calculated by comparing the number of detected nasopharyngeal and oropharyngeal samples positive per method with the number of individuals positive for S. pneumoniae based on isolation of live pneumococcus either from the primary diagnostic or qPCR-guided culture (n=101 of nasopharyngeal samples culture-positive in UK and n=19 oropharyngeal samples culture-positive in the Netherlands).
DISCUSSION
In the current study we have demonstrated that molecular methods exhibit near-perfect agreement to conventional culture in the detection and serotyping of S. pneumoniae in children and adults when a nasopharyngeal swab is the only sample tested. Furthermore, we have observed increased sensitivity of S. pneumoniae carriage detection among adults by testing oropharyngeal samples with molecular methods and conducting qPCR-guided culturing. We highlight several statistical procedures that can be used to evaluate the reliability of molecular results and enhance the specificity of molecular methods for the detection of live S. pneumoniae.
The current gold standard method for carriage detection is the isolation of live pneumococci from cultures of deep trans-nasal nasopharyngeal swab, in adults complemented with culture of a swab collected trans-orally (OP) [4]. However, the gold standard lacks sensitivity in case of low-density carriage or when applied to poly-microbial samples in which S. pneumoniae is not a dominant bacterium and it does not allow the detection of co-carriage of pneumococcal strains [7, 18, 19, 21, 43]. Since the density of pneumococcal carriage episodes in adults is much lower than in children [18], sampling multiple sites increases sensitivity of carriage detection [7, 16, 18]. However, carriage surveillance based exclusively on primary diagnostic cultures of nasopharyngeal and oropharyngeal samples often provides low quality data in adults. This limitation of the gold standard method is of particular concern for surveillance of carriage in older adults [19, 24], the age group with the largest incidence and burden of pneumococcal pneumonia and invasive pneumococcal disease [2, 44] as carriage is often reported to be virtually absent when the gold standard method is applied [16, 45].
To overcome these limitations, we complement conventional culture with molecular methods to improve the overall sensitivity of S. pneumoniae carriage detection [7, 18, 21–24]. This approach is particularly effective when applied to highly poly-microbial samples from the oral niche. To validate this method, we have compared the performance of molecular methods to the gold standard. While demonstrating near-perfect agreement with primary nasopharyngeal cultures from children, application of qPCR-based methods still significantly increased the number of carriers detected. Near perfect agreement was also observed in adults when analysis focused explicitly on nasopharyngeal swabs. In adults, application of qPCR-based methods to nasopharyngeal samples did not increase sensitivity of carriage detection.
Importantly, in adults testing oropharyngeal samples changed the results dramatically. By revisiting samples negative in primary culture yet positive by qPCR with culture qPCR-guided culturing we have significantly increased the number of adult carriers from whom viable pneumococci were isolated. It demonstrates that molecular methods can improve the overall sensitivity of S. pneumoniae detection, a result in line with previous reports by us and others [7, 16, 18, 22, 46]. It also highlights that the gold standard method applied to oropharyngeal cultures severely underestimates presence of S. pneumoniae in adults as qPCR-guided culturing increased by 7.7-fold the number of oropharyngeal samples from which viable pneumococci were isolated. Interestingly, there is evidence that testing oropharyngeal in addition to nasopharyngeal samples substantially enhances sensitivity of carriage detection also in children [17].
Concerns have been raised that the molecular methods are overly sensitive and lack specificity for the detection of live bacteria as “relic DNA” (DNA from non-intact cells) could be detected as well [28]. Indeed, live pathogens are less likely to be cultured from samples displaying weak positivity by qPCR [21, 47]. While this may reflect limitations in current culturing techniques, it could also indicate presence of relic DNA that may reflect recent exposure to pneumococcus instead of colonization proper [28]. The addition of the culture-enrichment procedure prior to molecular detection reduces the risk of misclassifying these events as carriage, thus improving the specificity of detection for live pneumococci. Culture-enrichment also increases the sensitivity of carriage surveillance, in particular for poly-microbial samples [7, 18, 21–24, 46–48]. Indeed, we observed that S. pneumoniae detection in culture-enriched samples displayed significantly increased, sensitivity, specificity, and agreement with culture and qPCR-guided culture when compared with detection in minimally processed samples.
To further improve the specificity of molecular methods for live pneumococci we performed ROC curve analysis to estimate with the Youden index Cq cut-off values that display an optimal combination of sensitivity and specificity for samples positive for S. pneumoniae by culture in primary cultures or qPCR-guided cultures [37]. This approach excluded samples that exhibited minimal positivity for targeted genes by qPCR or displayed poor agreement between piaB and lytA genes as shown in Bland-Altman graphs and with the intra-class correlation coefficient. The use of the Youden index in a ROC curve analysis to identify optimal Cq cut-off values enhances the specificity of molecular methods for detection of live pneumococcus, and the ROCdCq criteria can be used to direct qPCR-guided culturing. The success of this approach is dependent on culturing methods, therefore care should be taken to employ sensitive culturing techniques, such as the use selective culture plates [4]. Furthermore, for molecular methods to be informative for culturing efforts DNA extraction should be performed irrespective of the identification of pneumococcal growth in cultures. ROC curve analysis with the Youden index may yield overly stringent Cq cut-off values if qPCR-guided culturing efforts are not conducted, which could result in classifying samples containing live pneumococci as negative by qPCR. Recent advances in culturing techniques may further enhance the specificity of molecular methods for the detection of live S. pneumoniae [49].
Some studies have cautioned against the use of molecular methods due to the presence of pneumococcal genes among commensal streptococci [25–27]. This phenomenon is likely to be common among bacterial species co-existing in a shared niche [50–52] and of particular concern for highly poly-microbial samples, hence careful selection of targeted genes is important [53]. As previously described by our group [18, 22, 23] and others [54] a “two-to-tango” approach, quantifying both piaB and lytA genes with molecular methods enhances the specificity of S. pneumoniae detection in poly-microbial samples. This approach allows for measurements to be evaluated in a reliability analysis on a per sample basis [39] and for bias between measurements to be identified. While systemic biases between quantified genes could reflect dissimilarities between qPCR assay efficiencies, non-systemic biases can arise due to the presence of a gene or closely related sequence in DNA from other bacterial species. In case of multiple serotype carriage, serotype-specific abundance of the dominant serotype should display high agreement to targeted genes used to detect pneumococcus (e.g., piaB and lytA) while serotype-specific abundances of nondominant serotypes may display reduced agreement. Serotypes that exhibit greater abundances than piaB or lytA, or appear to be present in samples classified as negative for S. pneumoniae are likely to be due to commensal streptococci harbouring genes involved in the biosynthesis of the pneumococcal polysaccharide capsule [55, 56]. As such, certain serotype-specific assays may be nonreliable in poly-microbial sample types.
Insights into the co-occurrence of multiple serotypes in carriage is critical for understanding the dynamics of the serotypes during colonization, host-to-host transmission, and carriage progression into disease. Detection of secondary strains present in co-carriage is also important when distinguishing between unmasking and serotype replacement in assessment of pneumococcal vaccines impact [57]. However, as demonstrated in our study, and described previously the gold standard method does not readily allow detection of multiple serotypes, an event that is likely to occur often in carriage [57]. Furthermore, using our methodology we have observed no significant difference in observed and expected frequencies of multiple-serotype carriage despite what has been reported previously by others whom exclusively used data from conventional culture as method of detection [58].
An important strength of our study is the application of the procedure to two different carriage studies conducted in two different countries, and the evaluation of reproducibility in an interlaboratory comparison. Another strength of our study is that the procedure described is flexible and can be readily adapted to the carriage surveillance of other bacteria, such as Neisseria meningitidis [47].
Our study had a number of limitations, the impact of testing oropharyngeal samples was only evaluated for Dutch adults and not adults sampled in England nor for any children. Since oropharyngeal samples tested were collected explicitly from Dutch adults, our findings could be unique for that demographic group and geographic location. In the protocol described we only used piaB and lytA to detect S. pneumoniae with molecular methods and we did not evaluate the procedure with alternative targets described by others [54, 59]. Furthermore, we tested for a limited number of serotypes by qPCR. Finally, not all serotypes were shown to be equally reliably detected with molecular methods and for a number of qPCR assays we were not able to identify the serotype within a serogroup.
In summary, we argue that accurate detection of pneumococcal carriage using qPCR requires concordant quantification of two genes (‘two-targets-to-tango’) to classify a sample as positive for pneumococcus. Similarly, qPCR-based detection requires concordance between pneumococcal and serotype-specific quantification to assure specificity of the method (two-targets-to-tango). We provide evidence that accurate detection of pneumococcal carriage in adults requires at least testing of both, nasopharyngeal and oropharyngeal samples (‘two-samples-to-tango’) and requires molecular detection to be intertwined with culture (‘two methods-to-tango’). Finally, we advise revisiting samples for qPCR-guided culturing (‘two-cultures-to-tango’) when positive by qPCR but negative at primary diagnostic culture. The use of qPCR-guided culturing is of utmost importance for oropharyngeal swabs.
We have outlined the procedure that enhances the specificity of molecular methods for the surveillance of pneumococci and of pneumococcal serotypes in nasopharyngeal and oropharyngeal samples. Our results demonstrate near-perfect agreement between conventional culture and molecular methods when applied to nasopharyngeal samples from children. We have shown that the sensitivity of S. pneumoniae carriage surveillance can be greatly enhanced by complementing conventional culture with qPCRs. In adults, testing oropharyngeal on the top of nasopharyngeal samples was of paramount importance for accuracy of pneumococcal carriage detection. Studies investigating impact of testing oropharyngeal samples on detection of pneumococcal carriage in children are needed.
Authors’ contribution
E.A.M.S. and K.T. had an idea and initiated the study. N.R., E.M., N.K.F., E.A.M.S. and K.T. secured financial support. N.F. and K.T. led the project. A.J.W-M., P.B., C.S., M.v.H., N.R., E.M., and N.K.F. conducted carriage studies, collected the data and provided study materials. W.R.M., J.v.V., and K.T. developed and validated laboratory methods, and wrote the laboratory protocol. W.R.M., J.v.V., D.L., P.B., T.N., S.M., R.T., S.E., and C.S. analysed samples and collected the data. W.R.M., R.M., and K.T. contributed analytical tools. W.R.M., J.v.V., and D.L. curated the data. W.R.M., D.L., N.K.F., and K.T. managed the study. W.R.M., and K.T. performed formal analysis of study data, visualized presentation of the results and drafted the manuscript. All authors amended, critically reviewed, and commented on the final manuscript.
Financial support
Funding for this study was provided to UMCU and PHE by GlaxoSmithKline Biologicals SA. GlaxoSmithKline Biologicals SA was provided the opportunity to review a preliminary version of this paper for factual accuracy, but the authors are solely responsible for final content and interpretation. The authors received no financial support or other form of compensation related to the development of the paper.
Potential Conflicts of Interest
K.T. received funds for an unrestricted research grant from GlaxoSmithKline Biologicals SA, consultation fees, fees for participation in advisory boards, speaking fees and funds for unrestricted research grants from Pfizer, and fees for participating in advisory boards from Merck Sharp & Dohme, all paid directly to his home institution. Except for the funds from GlaxoSmithKline Biologicals SA none was received in the relation to the work reported here. Public Health England (PHE) has provided vaccine manufacturers (GSK, Pfizer, Sanofi) with post-marketing surveillance reports on pneumococcal infection which the companies are required to submit to the UK Licensing authority in compliance with their Risk Management Strategy. A cost recovery charge is made for these reports. PHE’s Respiratory and Vaccine Preventable Bacteria Reference Unit has received unrestricted research grants from Pfizer to participate in pneumococcal surveillance projects.
Data Availability
All data produced in the present work are contained in the manuscript.
REFERENCES
