Abstract
Background Few metrics exist to describe phenotypic diversity within ophthalmic imaging datasets, with researchers often using ethnicity as an inappropriate marker for biological variability.
Methods We derived a continuous, measured metric, the retinal pigment score (RPS), that quantifies the degree of pigmentation from a colour fundus photograph of the eye. RPS was validated using two large epidemiological studies with demographic and genetic data (UK Biobank and EPIC-Norfolk Study).
Findings A genome-wide association study (GWAS) of RPS from UK Biobank identified 20 loci with known associations with skin, iris and hair pigmentation, of which 8 were replicated in the EPIC-Norfolk cohort. There was a strong association between RPS and ethnicity, however, there was substantial overlap between each ethnicity and the respective distributions of RPS scores.
Interpretation RPS serves to decouple traditional demographic variables, such as ethnicity, from clinical imaging characteristics. RPS may serve as a useful metric to quantify the diversity of the training, validation, and testing datasets used in the development of AI algorithms to ensure adequate inclusion and explainability of the model performance, critical in evaluating all currently deployed AI models. The code to derive RPS is publicly available at: https://github.com/uw-biomedical-ml/retinal-pigmentation-score.
Funding The authors did not receive support from any organisation for the submitted work.
Evidence before this study Vision loss due to retinal disease is a global problem as populations age and diabetes becomes increasingly prevalent. AI algorithms developed for efficient diagnosis of diabetic retinopathy and age-related macular degeneration rely on large imaging datasets collected from clinical practice. A substantial proportion (more than 80%) of publicly available retinal imaging datasets lack data on participant demographic characteristics. Some ethnic groups are noticeably underrepresented in medical research.
Previous findings in dermatology suggest that AI algorithms can show reduced performance on darker skin tones. Similar biases may exist in retinal imaging, where retinal colour has been shown to affect disease detection.
Added value of this study We introduce the Retinal Pigment Score (RPS), a measure of retinal pigmentation from digital fundus photographs. This score showed strong, reproducible associations with genetic variants related to skin, eye, and hair colour. Additionally, we identify three genetic loci potentially unique to retinal pigmentation, which warrant further investigation. The RPS provides an accurate and objective metric to describe the biological variability of the retina directly derived from an image.
Implications of all the available evidence The RPS method represents a valuable metric with importance to harness the detailed information of ophthalmic fundus imaging. Its application implies potential benefits, such as improved accuracy and inclusivity, over human-created sociodemographic classifications used in dataset compilation and in the processes of developing and validating models. The RPS could decouple the distinct social and political categorical constructs of race and ethnicity from image analysis. It is poised to both accurately describe the diversity of a population study dataset or an algorithm training dataset, and for investigate algorithmic bias by assessing outcomes.
Further work is needed to characterise RPS across different populations, considering individual ocular factors and different camera types. The development of standard reporting practices using RPS for studies employing colour fundus photography is also critical.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
Funding: The authors did not receive support from any organisation for the submitted work.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
The source data are openly available human data that were were orginally located at: https://www.ukbiobank.ac.uk/ https://www.epic-norfolk.org.uk/
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Data Availability
The code is available on github: https://github.com/uw-biomedical-ml/retinal-pigmentation-score
https://github.com/uw-biomedical-ml/retinal-pigmentation-score
