PT - JOURNAL ARTICLE AU - Miranda de Paiva, Bruno Barbosa AU - Delfino-Pereira, Polianna AU - de Andrade, Claudio Moisés Valiense AU - Gomes, Virginia Mara Reis AU - Lima, Maria Clara Pontello Barbosa AU - Souza-Silva, Maira Viana Rego AU - Carneiro, Marcelo AU - Martins, Karina Paula Medeiros Prado AU - Sales, Thaís Lorenna Souza AU - de Carvalho, Rafael Lima Rodrigues AU - Pires, Magda C. AU - F. Ramos, Lucas Emanuel AU - Silva, Rafael T. AU - Bezerra, Adriana Falangola Benjamin AU - Schwarzbold, Alexandre Vargas AU - Nunes, Aline Gabrielle Sousa AU - Maurílio, Amanda de Oliveira AU - Scotton, Ana Luiza Bahia Alves AU - Costa, André Soares de Moura AU - Castro, Andriele Abreu AU - Farace, Bárbara Lopes AU - Cimini, Christiane Corrêa Rodrigues AU - De Carvalho, Cíntia Alcantara AU - Silveira, Daniel Vitório AU - Ponce, Daniela AU - Pereira, Elayne Crestani AU - Manenti, Euler Roberto Fernandes AU - Cenci, Evelin Paola de Almeida AU - Lucas, Fernanda Barbosa AU - Rodrigues, Fernanda D’Athayde AU - Anschau, Fernando AU - Botoni, Fernando Antonio AU - Graça Aranha, Fernando AU - Bartolazzi, Frederico AU - Bastos, Gisele Alsina Nader AU - Vietta, Giovanna Grunewald AU - Nascimento, Guilherme Fagundes AU - Noal, Helena Carolina AU - Duani, Helena AU - Vianna, Heloisa Reniers AU - Guimarães, Henrique Cerqueira AU - Gomes, Isabela Moraes AU - Salles Martins Costa, Jamille Hemétrio AU - da Fonseca, Jéssica Rayane Corrêa Silva AU - Guimarães, Júlia Di Sabatino Santos AU - de Morais, Júlia Drumond Parreiras AU - Rugolo, Juliana Machado AU - Batista, Joanna D’arc Lyra AU - de Alvarenga, Joice Coutinho AU - Chatkin, José Miguel AU - Ruschel, Karen Brasil AU - Moreira, Leila Beltrami AU - de Oliveira, Leonardo Seixas AU - Zandoná, Liege Barella AU - Pinheiro, Lílian Santos AU - Monteiro, Luanna da Silva AU - Sousa, Lucas de Deus AU - Kopittke, Luciane AU - Viana, Luciano de Souza AU - de Castro, Luis César AU - Assis, Luisa Argolo AU - Santos, Luisa Elem Almeid AU - Cabral, Máderson Alvares de Souza AU - Raposo, Magda Cesar AU - Floriani, Maiara Anschau AU - Ferreira, Maria Angélica Pires AU - Bicalho, Maria Aparecida Camargos AU - de Godoy, Mariana Frizzo AU - Nogueira, Matheus Carvalho Alves AU - de Figueiredo, Meire Pereira AU - Guimarães-Júnior, Milton Henriques AU - De Sordi, Mônica Aparecida de Paula AU - Sampaio, Natália da Cunha Severino AU - de Oliveira, Neimy Ramos AU - Assaf, Pedro Ledic AU - Lutkmeier, Raquel AU - Valacio, Reginaldo Aparecido AU - Finger, Renan Goulart AU - Senger, Roberta AU - Menezes, Rochele Mosmann AU - Silva, Rufino de Freitas AU - Francisco, Saionara Cristina AU - Guimarães, Silvana Mangeon Mereilles AU - Araújo, Silvia Ferreira AU - Oliveira, Talita Fischer AU - Kurtz, Tatiana AU - Fereguetti, Tatiani Oliveira AU - de Oliveira, Thainara Conceição AU - Diniz, Thulio Henrique Oliveira AU - Ribeiro, Yara Cristina Neves Marques Barbosa AU - Ramires, Yuri Carlotto AU - Gonçalves, Marcos André AU - Marcolino, Milena Soriano TI - Effectiveness, Explainability and Reliability of Machine Meta-Learning Methods for Predicting Mortality in Patients with COVID-19: Results of the Brazilian COVID-19 Registry AID - 10.1101/2021.11.01.21265527 DP - 2021 Jan 01 TA - medRxiv PG - 2021.11.01.21265527 4099 - http://medrxiv.org/content/early/2021/11/02/2021.11.01.21265527.short 4100 - http://medrxiv.org/content/early/2021/11/02/2021.11.01.21265527.full AB - Objective To provide a thorough comparative study among state-of-the-art machine learning methods and statistical methods for determining in-hospital mortality in COVID-19 patients using data upon hospital admission; to study the reliability of the predictions of the most effective methods by correlating the probability of the outcome and the accuracy of the methods; to investigate how explainable are the predictions produced by the most effective methods.Materials and Methods De-identified data were obtained from COVID-19 positive patients in 36 participating hospitals, from March 1 to September 30, 2020. Demographic, comorbidity, clinical presentation and laboratory data were used as training data to develop COVID-19 mortality prediction models. Multiple machine learning and traditional statistics models were trained on this prediction task using a folded cross-validation procedure, from which we assessed performance and interpretability metrics.Results The Stacking of machine learning models improved over the previous state-of-the-art results by more than 26% in predicting the class of interest (death), achieving 87.1% of AUROC and macro F1 of 73.9%. We also show that some machine learning models can be very interpretable and reliable, yielding more accurate predictions while providing a good explanation for the ‘why’.Conclusion The best results were obtained using the meta-learning ensemble model – Stacking. State-of the art explainability techniques such as SHAP-values can be used to draw useful insights into the patterns learned by machine-learning algorithms. Machine-learning models can be more explainable than traditional statistics models while also yielding highly reliable predictions.Competing Interest StatementThe authors have declared no competing interest.Funding StatementThis study was funded by CAPES and FAPEMIG Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:The study protocol was approved by the Brazilian National Commission for Research Ethics (CAAE 30350820.5.1001.0008). Individual informed consent was waived due to the severity of the situation and the use of deidentified data, based on medical chart review only.I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.YesAll data produced in the present study are available upon reasonable request to the authors.