Metagenomic next-generation sequencing of cerebrospinal fluid for diagnosis of central nervous system infections: 7-year performance of a clinically validated test

UCSF mNGS test cohort description

Overall, 4,828 CSF mNGS tests were performed by the UCSF clinical microbiology laboratory from June 2016 to April 2023, including testing of 1,130 UCSF patients (Table 1). The number of tests performed increased by nearly 10-fold from 116 in 2016 to 1,067 in 2022 (Figure 1). 55.7% of patients were male; the mean age was 41.5 years old with 24.2% of the cohort younger than 18 years old. Most CSF mNGS tests were performed for US patients representing 46 different states (n=4,075), with 31 tests for patients from other countries (Canada was the most frequent). 722 samples were sent from national or regional reference laboratories (Figure 1). Median turnaround times for UCSF and non-UCSF patients were 8.2 and 11.4 days (p < 0.0001), respectively, from sample collection to result, and 3.6 and 3.8 days, respectively, from start of sample processing in the laboratory to result (p < 0.0001). The longer turnaround times from sample collection to test results for non-UCSF patients may be explained by the delays between the lumbar puncture and decision to send the sample for mNGS testing, delays from shipping of samples, and delays from batching of samples prior to testing.

• Download figure
• Open in new tab

Figure 1. Distribution of tests ordered by year and by location.

Distribution of tests ordered by State, internationally, and from reference laboratories. A total of 4075 tests were from the United States, California being the most frequent State of origin (n = 2420 samples). Reference laboratories such as ARUP, Labcorp, and Mayo Clinic Laboratories (n=722) receive tests from multiple States, so individual samples couldn’t be retraced and, therefore, excluded from the figure. 14.8 % (n = 715) of samples were sent from hospitals dedicated to the care of pediatric patients. * Data are up to April 2023.

We also evaluated QC metrics and microorganism detection yields associated with CSF mNGS testing. High host background associated with presumptive reduced assay sensitivity¹⁰ was more frequently seen in DNA libraries (12.2%) than in RNA libraries (1.6%). This result illustrates the interference of host DNA background and the higher efficiency of DNase treatment (RNA libraries) compared to antibody-based methylated DNA removal (DNA libraries) in reducing host DNA background. On the other hand, RNA libraries were more difficult to amplify due to low read counts which were more frequent for RNA libraries (5.3%) than for DNA libraries (2.6%). QC failure due to inadvertent errors in sample processing that required additional repeat testing was seen rarely (<~1%).

Out of 4,828 samples, 697 samples (14.4%) were positive for any microorganism, excluding possible or likely contaminants (Table 1, Supplementary Dataset 1). The annual positivity rate varied from 11.4% to 17,4%. Notably, for 2016, the positivity rate was 27%, which corresponds to the year the prospective Precision Diagnosis of Acute Infectious Diseases (PDAID) study was conducted using enrollment criteria to select patients for mNGS testing⁶. DNA viruses were most frequently identified (7% of all samples), followed by RNA viruses (4.3%), bacteria (2.7%), fungi (1.4%) and parasites (0.5%). Multiple bacterial and/or fungal taxa were detected in 7.2% of samples (348/4828) and single organisms corresponding to a commensal and/or environmental species in 3.4% of samples (164/4828). These results categories were reported more frequently for patients outside of UCSF (7.8% vs. 5.4 %, p=0.007 and 3.8% vs. 2%, p=0.004, respectively) and can be attributed to differences in CSF collection or transportation methods. These results were classified as negative since they are clinically considered likely to represent sample contamination rather than detection of etiology, as indicated in the clinical report⁶. The positivity rate in UCSF (16.2%) was higher than in non-UCSF (13.9%) patients (p=0.05). This difference can be explained in part by the higher rate of results for UCSF patients that were subthreshold (2.9% vs. 1.8%, p=0.03) (the number of reads was lower than threshold values specified in the QC metrics), and were classified as positives, based on review of patient charts, clinical presentation, and discussion with the UCSF clinical team. Overall, 2% (98/4828) positive results were reported as subthreshold. 92.3% (12/13) of Mycobacterium tuberculosis and 93.4 % (15/16) Coccidioides spp. results were reported as subthreshold. Other overrepresented organisms in this category were Balamuthia mandrillaris, Histoplasma capsulatum, West Nile virus, and Powassan virus, with 66% (2/3), 50% (2/4), 28.6% (8/28), 21.4% (3/14) of their results being reported as subthreshold, respectively (Supplementary dataset 1).

The pathogens detected with the UCSF mNGS test represented a large diversity of organisms (Figure 2A). A total of 59 unique bacterial species were detected among the 132 positive results for bacteria (Figure 2B). The assay detected 86 non-fastidious bacterial pathogens with 35 unique species, corresponding to bacteria that traditional culture methods can detect. 46 uncommon and/or difficult-to-diagnose pathogens from 24 unique species were detected such as Mycobacterium tuberculosis, Nocardia farcinia, Leptospira borgpetersenii, Borrelia miyamotoi, and Tropheryma whipplei, corresponding to rare but definitive CNS pathogens, that are fastidious or not often detected by culture (Supplementary table 1). The most common RNA viruses detected were HIV (92), vector-borne viruses (57), and enteroviruses (16) (Figure 2C). Uncommon arboviruses (Figure 2D) such as St-Louis encephalitis virus, La Crosse virus, Cache Valley virus, and even a novel pathogen not previously described in humans, Potosi virus ¹¹, were detected. Various serotypes of enteroviruses could also be detected, including A71 and D68, associated with acute flaccid myelitis (Figure 2E). Other interesting pathogens detected include Coccidioides immitis and posadasi (16), Cryptococcus neoformans and gattii (13), Histoplasma capsulatum (4), and Fusarium spp. (3) for fungi (Figure 2G), and Toxoplasma gondii (10), Balamuthia mandrillaris (3), Angiostrongylus cantonensis (2), and Naegleria fowleri (1) for parasites (Figure 2H). Some results were also related to the initiation of outbreak investigations. For example, a vaccine strain of Yellow fever virus in CSF from a transplant recipient with encephalitis was reported - this case prompted an investigation by the US Centers for Disease Control of a nationwide transplant-associated outbreak¹². Also, the UCSF mNGS test uncovered an outbreak of fungal meningitis with Fusarium solani in the US associated with surgical procedures performed under epidural anesthesia in Matamoros, Mexico¹³.

• Download figure
• Open in new tab

Figure 2. Summary of positive results by mNGS (n=4828).

(A) Summary of positive UCSF mNGS test results by microorganism type. One test can be positive for more than one target. (B) A total of 59 unique species with a total of 132 cases were detected by the assay. 88 pathogens that can be detected by traditional culture methods corresponding to 35 unique species were detected. 46 uncommon and/or difficult-to-diagnose bacteria, corresponding to rare but definitive CNS pathogens, that are fastidious or not detected by culture. The list of bacterial pathogens detected in available in supplementary table 1. (C) The most common RNA viruses detected were HIV (92), vector-borne viruses (57), and enteroviruses (16). Other RNA viruses detected include Lymphocytic Choriomeningitis virus (3), Astrovirus (2), Calicivirus (2), Coronavirus 229E (2), SARS-CoV-2 (1), HTLV-2 (1), Human parechovirus (1), Measles virus (1). (D) Vector-borne viruses, a diverse and clinically important group, are typically challenging to diagnose by traditional methods because they require specific NAAT or serologies for each virus, whereas mNGS can detect a broad range of pathogens in a single assay. Orthobunyaviruses detected included Cache Valley virus (3), Jamestown Canyon virus (1) La Crosse virus (1), Potosi virus (1). Of interest, the Potosi virus is a novel species in this genus, not reported before as causing diseases in humans ¹¹. (E) One of the strengths of the assay is its ability to discriminate between species and serotypes, unlike the typical testing method of NAAT. For instance, the assay detected various subtypes of enteroviruses and five cases of A71 and D68 associated with acute flaccid myelitis. (F) Other DNA viruses detected included Human parvovirus 4 and B-19 (4) and HHV-8 (1). (G) Other fungi detected included Alternaria spp. (2), Mucorales spp. (2)., Epicoccum spp. (1), and Sporothrix schenkii (1). (H) The assay detected challenging parasitic infections to diagnose, such as Toxoplasma gondii (10), Balamuthia mandrillaris (3), Angiostrongylus cantonensis (2), and Naegleria fowleri (1). HCV: Hepatitis C virus, HEV: Hepatitis E virus, HIV: Human immunodeficiency virus, EV: Enterovirus, EBV: Epstein-Barr virus, CMV: Cytomegalovirus, HHV-6: Human herpes virus 6, HHV-7: Human herpes virus 7, VZV: Varicella-zoster virus, HSV: Herpes simplex virus, NAAT: Nucleic acid amplification test.

UCSF patient cohort

Results from 1,164 CSF samples tested by mNGS from June 2016 to June 2023 were analyzed along with a retrospective chart review of the corresponding 1052 patients (some patients had multiple CSF mNGS tests performed). The UCSF cohort consisted of 223 (19.2%) cases of microbiologically confirmed CNS infection, 431 (37%) cases of autoimmune disease or another presumably non-infectious condition, 1 case (0.1%) of prion disease, and 509 (43.7%) cases of meningitis/encephalitis with unknown etiology despite all diagnostic testing including mNGS testing and longitudinal follow-up (Table 2). Among all UCSF patients who received the UCSF mNGS test, 425 (36.5%) were immunocompromised, 1021 (87.7%) were hospitalized, 450 (38.7%) had an intensive care unit (ICU) stay with a with a mean length of stay of 21.6 days, and 119 (10.2%) died within 60 days of admission. The average number of diagnostic tests performed was 6 from CSF and 20.2 from all samples (including CSF), consisting of 28.2% (average of 5.7) culture, 24.8% (5) NAAT, 13.4% (2.7) antigen, and 33.7% (6.8) serology-based assays.

For each case, we established a composite diagnosis that incorporated clinical adjudication with the results of all microbiologic testing (Supplementary dataset 2). There were 184 (15.8%) cases of positive detection of a microorganism by CSF mNGS, of which 36 (3.1%) were considered as subthreshold positive results (Table 2). 77 samples (6.6%) included incidental findings (for example, HIV detection in an HIV-infected patient with an acute neurologic illness not explained by HIV) and were excluded from further analysis. 5 samples (0.4%) included results of unclear significance and were also excluded (1 sample with HSV-1, 2 with CMV, 1 with LCMV, and 1 with Candida albicans). Among results with a comment about possible contamination and multiple bacterial/fungal taxa, only 2/22 (9.1%) and 1/65 (1.5%) were considered positive upon clinical review, respectively, as they were clearly related to the final adjudicated diagnosis (Supplementary table 5 and 6). Among the included mNGS positive results, 130 were true positive, where a target relevant to the adjudicated diagnosis was detected, and 11 were false positive, detecting a target not relevant to the final diagnosis (Table 3). For the false positive results, none of the final diagnoses were due to an infectious etiology, and none of the patients received treatment for the organism detected. Most of the false positive results were bacteria (75%) (Figure 4, Supplementary table 4). 86% (31/36) of subthreshold results were considered true positive. Among the mNGS negative results, 930 were true negative and 100 were false negative. Overall, CSF mNGS testing exhibited a sensitivity of 56.5%, specificity of 98.8%, accuracy of 90.5%, positive predictive value (PPV) of 92.2% and negative predictive value (NPV) of 90.3% for CNS infections (Table 3). The sensitivity of CSF mNGS (56.5%) was statistically higher than all other diagnostic modalities, including direct detection tests from CSF (44.8%, p=0.004), direct detection tests from samples other than CSF (16.2%, p<0.001), and indirect serologic detection from blood (or blood components including serum and plasma) or CSF (30.3%, p<0.001) (Table 3). A proportional Venn diagram revealed that 46 infectious diagnoses were made exclusively using the UCSF mNGS test (20% of all diagnosis), with the overlap between mNGS and other testing modalities ranging from 6 to 63 diagnoses (Figure 3). The UCSF mNGS test was the most sensitive diagnostic modality for RNA viruses, DNA viruses, bacteria, and parasites, but not for fungi, which was detected more often by culture and antigen detection methods. When only considering diagnosis made by direct detection of pathogen on CSF, positive percentage agreement (PPA) of mNGS was 80.7%.

• Download figure
• Open in new tab

Figure 3. Comparison of each diagnostic modality in obtaining a final diagnosis and analysis of positive mNGS results.

(A) The proportional Venn diagram represents the overlap between diagnostic modalities, with CSF mNGS and direct detection on CSF having the biggest overlap due to the similarity in the analytic principle (detection of pathogen). A substantial number of diagnoses were made only by indirect serologic methods (44), and direct detection on samples other than CSF (24). The total number of diagnoses is 230 (Six cases were attributed to more than one causative organism and were considered individually for this analysis). Diagnoses made by mNGS that were negative by direct detection methods (n=58) included common (Cytomegalovirus (CMV), Coxsackieviruses, Streptococcus pneumoniae) and rare pathogens (Lymphocytic Choriomeningitis virus, Ureaplasma parvum, Balamuthia mandrillaris). When the diagnosis was made by both mNGS and direct detection methods on CSF, 16/72 (22%) were made faster by mNGS, implying that most of these diagnoses were, in fact, made by mNGS and confirmed by a secondary analysis. These results are available in supplementary table 2. (B) CSF mNGS detected the most cases of RNA viruses, DNA viruses, bacteria, and parasites, but not for fungi, which was detected more often by culture and antigen detection methods. (C) The most common discrepant factor to explain an mNGS positive result and a negative direct detection method on CSF was failure to specifically test for the pathogen when a test was readily available (33%), inability to culture a fastidious micro-organism (21%), failure to specifically test for the pathogen when there is no readily available test (17%), prior antimicrobial therapy with negative culture at time of CSF collection (15%), and inferior sensitivity of conventional tests compared to mNGS (14%).

• Download figure
• Open in new tab

Figure 4. Analysis of false negative and false positive mNGS results.

(A) 31 samples were positive by direct detection methods on CSF and negative by mNGS. 64% of the false negative results were attributed to either high DNA host background (45%), a known limitation of mNGS approaches^7,10, or persistent antigen positivity from fungal infection after onset of treatment (19%). The list of mNGS false negative results is available in supplementary table 3. (B) False positive results were infrequent (n=11), with a majority of bacteria detected (75%). None of these mNGS results were acted upon, and all pathogens detected by mNGS were not compatible with the clinical presentation of the patient. The list of false positive results is available in supplementary table 4.

As mNGS is most comparable to combined direct detection tests from CSF in the analytic principle, we analyzed discrepant results between the two diagnostic modalities. 58 infectious diagnoses were positive by CSF mNGS and not detected by another direct detection modality (Supplementary table 2). 33% (19/58) were not detected because no test was ordered that specifically targeted the pathogen despite the availability of such test, 17% (10/58) were not detected because a test to detect the pathogen was not readily available (e.g., Lymphocytic Choriomeningitis Virus), 14% (8/58) were not detected because the test had lower sensitivity compared to mNGS, 21% (12/58) were not detected because of fastidious microorganisms like mycobacteria and fungi that were not detectable by culture, and 15% (9/58) were not detected because antimicrobial treatment before CSF collection led to falsely negative culture (Figure 3). Among the 72 cases for which both the UCSF mNGS test and direct detection testing were positive, the UCSF mNGS test yielded the first reported result in 16 (22%) cases. Since most direct detection tests have a lower turnaround time, this result strongly implies that these results were first discovered by UCSF mNGS test and then confirmed by a targeted direct detection test (Supplementary table 2). Among 31 samples with negative mNGS results compared to positive direct detection tests, 14 (45%) were associated with high host background reducing assay sensitivity¹⁰, 6 (19%) were from patients who had received effective antimicrobial treatment prior to CSF collection, 7 (23%) was from a low titer sample under or near the limit of detection, and 1 (3%) was due to mNGS contamination (Figure 4). Notably, in cases of prior effective antimicrobial treatment, all patients were antigen-positive for either Cryptococcus sp. or Coccidioides sp. following initiation of treatment for cryptococcal or coccidioidal meningitis, respectively. No clear explanation for discrepant results was found in 3 (10%) false-negative cases.

CSF mNGS testing

The UCSF mNGS test was developed to facilitate the broad identification of pathogens in diagnostically challenging CNS infections. The previously reported assay workflow⁹ consisted of (i) nucleic acid extraction, (ii) microbial enrichment using antibody-based removal of methylated host DNA for DNA libraries and DNase treatment for RNA libraries, (iii) library preparation and pooling in equimolar concentrations, and (iv) sequencing on an Illumina (San Diego, California) instrument targeting 5 – 20 million sequences per library. Raw sequences were analyzed using SURPI+, a bioinformatic analysis pipeline for pathogen identification that was modified for clinical use²⁶. Automatically generated results from SURPI+ included heat maps of raw and normalized read counts per million (RPM) for each sample and an mNGS results summary in an Excel spreadsheet. A laboratory director (CC or SM) interpreted and reviewed the results before reporting the findings.

Quality control (QC) metrics for the assay included a minimum of 5 million preprocessed reads per sample, >75% of data with quality score >30 (Q>30), and successful detection of all seven representative organisms in the positive control and the internal spiked T1 and MS2 phage controls. A threshold criterion of ≥3 non-overlapping viral reads aligning to the target viral genome was considered a positive detection for virus identification. An RPM ratio threshold of 10 was considered positive for bacteria, fungi, and parasite detection⁹. Following the completion of a prospective study to evaluate clinical utility and diagnostic yield⁶, the reporting algorithm was modified to include additional comments regarding subthreshold detections of likely clinically relevant organisms (for example, Mycobacterium tuberculosis and Coccidioides immitis) at the discretion of the laboratory director. If the T1 (DNA library) and/or MS2 (RNA library) spiked internal control RPM is <100, indicating high host background, or the number of preprocessed reads is below the established cutoff of 5 million, negative CSF mNGS results are reported with an added comment regarding the potential for decreased sensitivity for organisms with a genome of that library type.

Patient Cohorts

The UCSF mNGS test results analyzed in this study (n=4,828) included those from samples received from June 2016 to April 2023; samples came from both within and outside the UCSF Health network. Patient demographic information that was available included age, sex, and geographic location by state. Clinical information was available for a subset of cases – those for which the detection of an unusual microorganism prompted a clinical microbial sequencing board (CMSB) discussion with the treating provider and those which were from UCSF patients (“UCSF cohort”). QC metrics and sequencing results were evaluated, along with overall turnaround time (sample collection to result) and laboratory turnaround time (start of processing to result).

The UCSF cohort (n=1,164) from June 2016 to June 2023 was assessed by retrospective chart review to determine the performance and utility in diagnosing CNS infections compared to other assays. All requests for the UCSF mNGS test were reviewed by the laboratory director, and testing was approved if patients met one of more of the following criteria: (i) CSF pleocytosis, (ii) brain biopsy that suggested a likely infectious etiology, (iii) immunocompromised patient with strong suspicion for infection based on clinical presentation, laboratory testing, and/or radiographic imaging, and (iv) recommendation for mNGS testing after neurology and/or infectious diseases consultation. A final composite diagnosis for the UCSF cohort was established through clinical adjudication and review of all diagnostic tests (mNGS and other conventional microbiological testing); this composite diagnosis was performed independently by three infectious disease physicians (PB, MLT and CYC) and categorized results as bacterial, viral, fungal, parasitic, non-infectious/autoimmune, prion-associated, or unknown. Any discrepancies in the final diagnosis were resolved by mutual agreement after re-review and discussion. A case was considered infectious when at least one test result (mNGS or conventional) was positive. All tests relevant to the final diagnosis were recorded for each patient and assigned to 1 of 4 different categories: (1) mNGS; (2) direct detection testing on CSF (culture, antigen detection, and NAAT); (3) direct detection testing sample types other than CSF, such as brain biopsy tissue and/or plasma (culture, antigen detection, and NAAT); and (4) indirect serologic testing from CSF or blood. For each positive test, the turnaround time from sample collection to test result was also recorded. mNGS results were further classified as (i) positive, (ii) single organism with possible contamination, (iii) multiple bacterial/fungal taxa (probable contamination), (iv) incidental findings (organisms detected that are not linked to the CNS infection pathogenesis were not considered a false positive, e.g. detection of HIV detection in an HIV-infected individual), or (v) unclear significance (organisms that were detected and were possibly associated with the infection without any possible confirmation). For the performance analysis, possible and probable contaminants were considered as negative, unless clearly related to the final adjudicated diagnosis. Incidental and unclear significance results were excluded from analysis. A complete review of the adjudication process is available as a supplementary file.

Lead Contact

Further information and requests for resources and reagents should be directed to and will be fulfilled by the Lead Contact, Charles Chiu (charles.chiu{at}ucsf.edu).

Materials Availability

This study did not generate any new reagents.

Data and Code Availability

The data that support the findings of this study are available on request from the corresponding author C.Y.C. The sequencing data are not publicly available because they contain information that could compromise research participants’ privacy/consent.