Abstract
An “efficiency index” (EI) for the evaluation of binary classifiers was recently characterised, where EI is the ratio of classifier accuracy to inaccuracy. The purpose of this study was to further develop EI by substituting balanced accuracy and unbiased accuracy in place of accuracy, and their respective complements in place of inaccuracy, to construct balanced EI and unbiased EI measures. Additional investigations, using the dataset of a prospective pragmatic test accuracy study of a cognitive screening instrument, explored use of the log method to calculate confidence intervals for the various EI formulations; the dependence of EI formulations on prevalence; and comparison of EI formulations with analogous formulations based on the Identification Index (II), a previously described metric which is also based on accuracy and inaccuracy, where II is accuracy minus inaccuracy. EI formulations are shown to have advantages over II formulations, in particular their boundary values (0 and ∞) mean that negative values never occur, unlike the case for II, and the inflection point of 1 demarcates likelihood of correct versus incorrect classification.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
This study did not receive any funding
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
Ethics committee of Walton Centre for Neurology and Neurosurgery gave ethical approval for this work (N 310).
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
The Chan Zuckerberg Initiative, Cold Spring Harbor Laboratory, the Sergey Brin Family Foundation, California Institute of Technology, Centre National de la Recherche Scientifique, Fred Hutchinson Cancer Center, Imperial College London, Massachusetts Institute of Technology, Stanford University, University of Washington, and Vrije Universiteit Amsterdam.
Subject Area
Reviews and Context
0
Comment
0
TRIP Peer Reviews
0
Community Reviews
0
Automated Services
0
Blogs/Media
Author Videos