Abstract
Metagenomic sequencing is increasingly being used in clinical settings for difficult to diagnose cases. The performance of viral metagenomic protocols relies to a large extent on the bioinformatic analysis. In this study, the European Society for Clinical Virology (ESCV) Network on NGS (ENNGS) initiated a benchmark of metagenomic pipelines currently used in clinical virological laboratories.
Methods Metagenomic datasets from 13 clinical samples from patients with encephalitis or viral respiratory infections characterized by PCR were selected. The datasets were analysed with 13 different pipelines currently used in virological diagnostic laboratories of participating ENNGS members. The pipelines and classification tools were: Centrifuge, DAMIAN, DIAMOND, DNASTAR, FEVIR, Genome Detective, Jovian, MetaMIC, MetaMix, One Codex, RIEMS, VirMet, and Taxonomer. Performance, characteristics, clinical use, and user-friendliness of these pipelines were analysed.
Results Overall, viral pathogens with high loads were detected by all the evaluated metagenomic pipelines. In contrast, lower abundance pathogens and mixed infections were only detected by 3/13 pipelines, namely DNASTAR, FEVIR, and MetaMix. Overall sensitivity ranged from 80% (10/13) to 100% (13/13 datasets). Overall positive predictive value ranged from 71-100%. The majority of the pipelines classified sequences based on nucleotide similarity (8/13), only a minority used amino acid similarity, and 6 of the 13 pipelines assembled sequences de novo. No clear differences in performance were detected that correlated with these classification approaches. Read counts of target viruses varied between the pipelines over a range of 2-3 log, indicating differences in limit of detection.
Conclusion A wide variety of viral metagenomic pipelines is currently used in the participating clinical diagnostic laboratories. Detection of low abundant viral pathogens and mixed infections remains a challenge, implicating the need for standardization and validation of metagenomic analysis for clinical diagnostic use. Future studies should address the selective effects due to the choice of different reference viral databases.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
MH was supported by the Clinical Research Priority Program 'Comprehensive Genomic Pathogen Detection' of the University of Zurich FXL receives funding from Instituto de Salud Carlos III, Spain (Grant numbers PI18/01824 and PI18/01759 and CIBEResp).
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
This is a comparison of pipelines using datasets. No clinical intervention was made.
All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Footnotes
jjcdevries{at}lumc.nl, I.A.Sidorov{at}lumc.nl, A.C.M.Kroes{at}lumc.nl, E.C.J.Claas{at}lumc.nl, julianne.brown{at}gosh.nhs.uk, breuej{at}gosh.nhs.uk, sofia.morfopoulou.10{at}ucl.ac.uk, nfischer{at}uke.de, j.huang{at}uke.de, b.oudemunnink{at}erasmusmc.nl, s.vanboheemen{at}erasmusmc.nl, arzu.sayiner{at}deu.edu.tr, alihanbulgurcu{at}gmail.com, christophe.rodriguez{at}aphp.fr, guillaume.gricourt{at}aphp.fr, els.keyaerts{at}kuleuven.be, leen.beller{at}kuleuven.be, claudia.bachofen{at}uzh.ch, jakub.kubacki{at}uzh.ch, Samuel.Cordey{at}hcuge.ch, Florian.Laubscher{at}hcuge.ch, Dennis.Schmitz{at}RIVM.nl, martin.beer{at}fli.de, dirk.hoeper{at}fli.de, huber.michael{at}virology.uzh.ch, kufner.verena{at}virology.uzh.ch, zaheri.maryam{at}virology.uzh.ch, aitana.lebrand{at}sib.swiss, annap{at}auth.gr, F.Xavier.Lopez{at}uv.es
Data Availability
The FASTQ datasets were and remain publicly available for user-friendly downloading at https://veb.lumc.nl/CliniMG (hosted by the dept. MM, LUMC, Leiden), and part of the datasets were additionally accessible via a COMPARE Data Hub at http://www.ebi.ac.uk/ena/pathogens (hosted by the European Bioinformatics Institute, EMBL-EBI)