Abstract
Artificial Intelligence (A.I.) solutions are increasingly considered for telemedicine. For these methods to adapt to the field of behavioral pediatrics, serving children and their families in home settings, it will be crucial to ensure the privacy of the child and parent subjects in the videos. To address this challenge in A.I. for healthcare, we explore the potential for global image transformations to provide privacy while preserving behavioral annotation quality. Crowd workers have previously been shown to reliably annotate behavioral features in unstructured home videos, allowing machine learning classifiers to detect autism using the annotations as input. We evaluate this method with videos altered via pixelation, dense optical flow, and Gaussian blurring. On a balanced test set of 30 videos of children with autism and 30 neurotypical controls, we find that the visual privacy alterations do not drastically alter any individual behavioral annotation at the item level. The AUROC on the evaluation set was 90.0% +/- 7.5% for the unaltered condition, 85.0% +/- 9.0% for pixelation, 85.0% +/- 9.0% for optical flow, and 83.3% +/- 9.3% for blurring, demonstrating that an aggregation of small changes across multiple behavioral questions can collectively result in increased misdiagnosis rates. We also compare crowd answers against clinicians who provided the same annotations on the same videos and find that clinicians are more sensitive to autism-related symptoms. We also find that there is a linear correlation (r=0.75, p<0.0001) between the mean Clinical Global Impression (CGI) score provided by professional clinicians and the corresponding classifier score emitted by the logistic regression classifier with crowd inputs, indicating that the classifier’s output probability is a reliable estimate of clinical impression of autism from home videos. A significant correlation is maintained with privacy alterations, indicating that crowd annotations can approximate clinician-provided autism impression from home videos in a privacy-preserved manner.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
These studies were supported by awards to DPW by the National Institutes of Health (1R21HD091500-01 and 1R01EB025025-01). Additionally, we acknowledge the support of grants to DPW from The Hartwell Foundation, the David and Lucile Packard Foundation Special Projects Grant, Beckman Center for Molecular and Genetic Medicine, Coulter Endowment Translational Research Grant, Berry Fellowship, Spectrum Pilot Program, Stanford's Precision Health and Integrated Diagnostics Center (PHIND), Wu Tsai Neurosciences Institute Neuroscience: Translate Program, Spark Program in Translational Research, Stanford's Institute of Human Centered Artificial Intelligence, the Weston Havens Foundation, as well as philanthropic support from Peter Sullivan. PW would like to acknowledge support from the Stanford Interdisciplinary Graduate Fellowship (SIGF).
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institution (approved by the Stanford University Institutional Review Board) and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Data Availability
Data can be provided at the request to the authors.
