PT  - JOURNAL ARTICLE
AU  - Franceschiello, Benedetta
AU  - Noto, Tommaso Di
AU  - Bourgeois, Alexia
AU  - Murray, Micah M.
AU  - Minier, Astrid
AU  - Pouget, Pierre
AU  - Richiardi, Jonas
AU  - Bartolomeo, Paolo
AU  - Anselmi, Fabio
TI  - Convolutional neural networks on eye tracking trajectories classify patients with spatial neglect
AID  - 10.1101/2020.07.02.20143941
DP  - 2021 Jan 01
TA  - medRxiv
PG  - 2020.07.02.20143941
4099  - http://medrxiv.org/content/early/2021/12/03/2020.07.02.20143941.short
4100  - http://medrxiv.org/content/early/2021/12/03/2020.07.02.20143941.full
AB  - Background and Objective Eye-movement trajectories are rich behavioral data, providing a window on how the brain processes information. We address the challenge of characterizing signs of visuo-spatial neglect from saccadic eye trajectories recorded in brain-damaged patients with spatial neglect as well as in healthy controls during a visual search task.Methods We establish a standardized preprocessing pipeline adaptable to other task-based eye-tracker measurements. We use a deep convolutional network, a very successful type of neural network architecture in many computer vision applications, including medical diagnosis systems, to automatically analyze eye trajectories.Results Our algorithm can classify brain-damaged patients vs. healthy individuals with an accuracy of 86±5%. Moreover, the algorithm scores correlate with the degree of severity of neglect signs estimated with standardized paper-and-pencil test and with white matter tracts impairment via Diffusion Tensor Imaging (DTI). Interestingly, the latter showed a clear correlation with the third branch of the superior longitudinal fasciculus (SLF), especially damaged in neglect.Conclusions The study introduces a new classification method to analyze eyes trajectories in patients with neglect syndrome. The method can likely be applied to other types of neurological diseases opening to the possibility of new computer-aided, precise, sensitive and non-invasive diagnosing tools.HighlightsWe identify signs of visuo-spatial neglect through an automated analysis of saccadic eye trajectories using deep convolutional neural networks (CNNs).We provide a standardized pre-processing pipeline adaptable to other task-based eye-tracker measurements.Patient-wise, we benchmark the algorithm prediction with standardized paper-and-pencil test results.We evaluate white matter tracts by using Diffusion Tensor Imaging (DTI) and find a clear correlation with the microstructure of the third branch of the superior longitudinal fasciculus.Deep CNNs can efficiently and non-invasively characterize left spatial neglect.Competing Interest StatementThe authors have declared no competing interest.Funding StatementB.F., A.M., received financial support for this work from the Fondation Asile des Aveugles (grant #232933 to M.M.M.), a grantor advised by Carigest SA (#232920 to M.M.M.), as well as the Swiss National Science Foundation (grants #169206 to M.M.M.). A.B. acknowledges the support of Swiss National Science Foundation to Armin Schnider under grant 32003B-175472 and to Radek Ptak under grant 32003B-184702. The work of P.B. and P.P. is supported by Agence Nationale de la Recherche through ANR-16-CE37-0005 and ANR-10-IAIHU-06. F.A. acknowledges the Center for Brains, Minds and Machines (CBMM), funded by NSF STC award CCF-1231216 and the financial support of the AFOSR projects FA9550-17-1-0390 and BAA-AFRL-AFOSR-2016-0007 (European Office of Aerospace Research and Development), and the EU H2020-MSCA-RISE project NoMADS - DLV-777826. Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:The dataset used in this study was reviewed by the INSERM ethical committee and received the approval of an Institutional Review Board (CPP Ile de France 1) https://www.sciencedirect.com/science/article/abs/pii/S0028393215001591I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.YesThe dataset is detailed, analysed and discussed in https://www.sciencedirect.com/science/article/abs/pii/S0028393215001591 In the current work we only used the dateset for supplementary automatized analyses. https://www.sciencedirect.com/science/article/abs/pii/S0028393215001591