Predicting the risk of diabetes complications using machine learning and social administrative data in a country with ethnic inequities in health: Aotearoa New Zealand

Background In the age of big data, linked social and administrative health data in combination with machine learning (ML) is being increasingly used to improve prediction in cardiovascular diseases (CVD). We aimed to apply ML methods on extensive national-level health and social administrative datasets to predict future diabetes complications by ethnicity.

Methods Five ML models were used to predict CVD events among all people with known diabetes in the population of New Zealand, utilizing national-level administrative data at the individual level.

Results The Xgboost ML model had the best predictive power for predicting CVD events three years into the future among the population with diabetes. The optimization procedure also found limited improvement in AUC by ethnicity. The results indicated no trade-off between model predictive performance and equity gap of prediction by ethnicity. The list of variables of importance was different among different models/ethnic groups, for examples: age, deprivation, having had a hospitalization event, and the number of years living with diabetes.

Discussion and conclusions We provide further evidence that ML with administrative health data can be used for meaningful future prediction of health outcomes. As such it could be utilized to inform health planning and healthcare resource allocation for diabetes management and the prevention of CVD events. Our results may suggest limited scope for developing prediction models by ethnic group and that the major ways to reduce inequitable health outcomes is probably via improved delivery of prevention and management to those groups with diabetes at highest need.