Deep Learning for Rare Disease: A Scoping Review

Literature search

We systematically searched for candidate articles in four databases: 1) PubMed; 2) the Association of Computing Machinery (ACM) Digital Library; 3) the Institute of Electrical and Electronics Engineers (IEEE) Xplore; and 4) DBLP. The following search criteria were used: (1) written in English; (2) published or publicly available (e.g., conference proceeding) between January 1, 2010 and December 31, 2021; 3) topic of interest includes deep learning and rare disease.

The general search string was constructed by combining terms representing deep learning (“machine learning” or “deep learning”) and “rare disease”. For DBLP, we only used the term “rare disease” due to the limitation of advanced search features of the database. For PubMed that indexes a broad range of scientific literature on life sciences and biomedical topics, we used 7,138 individual rare disease names with the terms “deep learning” and “machine learning”. Those 7,138 rare disease names were obtained from Orphanet[16], one of the largest knowledge databases for rare diseases. 7,138 individual rare disease names are available in Supplementary Table 1. The initial search was conducted in June 2021. An additional search to collect articles published in 2021 was conducted in May 2022. Table 1 shows the search strings for each database.

Literature selection

After candidate articles were obtained, we conducted abstract and full-text screening. Exclusion criteria were: 1) not published in a peer-reviewed journal or conference proceeding (e.g., preprint); 2) not original research (e.g., editorial); 3) reviews; 4) not human subject research; and 5) topic of interest does not include deep learning. We excluded studies that solely used multilayer perceptron (MLP) as their deep learning architectures since MLP alone does not represent the state-of-the-art deep learning architectures; and 6) topic of interest is not on rare diseases. All abstracts and full-texts were initially screened by one reviewer (JL), followed by the verifications done by three independent reviewers (JK, ZC and YS). Disagreements were discussed among the reviewers until consensus was reached. Covidence[17] was used for citation and screening management.

Data extraction and synthesis

One reviewer (JL) developed and extracted metadata for each article. Three reviewers (JK, ZC and YS) verified and refined the metadata for completeness. The resulting metadata include 1) publication year; 2) analytic task; 3) data type; 4) data size; 5) purpose of using deep learning; 6) deep learning architecture; 7) rare disease; 8) rare disease group; and 9) challenges in using deep learning for rare diseases. Table 2 summarizes the metadata elements.

Yearly publication trend

Table 3 shows the number of yearly publications in deep learning for rare disease, deep learning for any disease, and deep learning in general. The number of deep learning for any disease publications was estimated by search using the keyword “deep learning” and “disease” in Google Scholar. The number of deep learning publications was estimated by search using the keyword “deep learning” in Google Scholar. The number of publications in deep learning for rare disease was zero before 2017, but increased sharply from 10 article in 2017 to 145 articles in 2021. The number of publications in deep learning for rare disease and deep learning for any disease both showed greater growth trend than the number of publications in deep learning.

Analytic task for rare disease

20 articles focused on multiple analytic tasks and thus qualified for multiple categories. Of the 332 articles, 263 articles (79.2%) focused on diagnosis, 43 (13.0%) on prognosis, 29 (8.7%) on treatment, and 8 on characterization (2.4%) (Figure 2A). Diagnosis was the dominant focus of rare disease research using deep learning.

• Download figure
• Open in new tab

Figure 2.

The number of articles identified for each (A) analytic task for rare disease, (B) data type, (C) purpose of using deep learning, and (D) deep learning architecture.

Data types and sizes

As shown in Figure 2B, image data were the predominant data type used in 286 (86.1%) articles, while measurement data, encounter data, and literature data were used in 36 (10.8%), 8 (2.4%), and 2 (0.6%) articles, respectively. Specifically, MRI (85 articles) and CT (70 articles) were frequently used among image data (Supplementary Table 2).

We measured the data size for each data type based on the number of instances before any data augmentation was applied. An instance is defined as a single input for the deep learning model in the study. For example, we measured the number of patches instead of the number of images if the input of a deep learning model was patches of an image. The entire data size was measured before any split was conducted (e.g., split into training, validation, and test set). If multiple datasets were used for separate experiments independently, we measured the size of the largest dataset. Articles that did not clarify the size of data were excluded from the analysis. The median data sizes for image, encounter, and measurement data are 2,762 (interquartile range [IQR]: 790.5 – 12,817), 193,676 (IQR: 49,727.5 – 2,090,819), and 1,358 (IQR: 498 – 10,148) (Figure 3). We did not calculate median and IQR of the data size for literature data since it was used by only 2 articles.

• Download figure
• Open in new tab

Figure 3.

Box plots of data size for all data types.

We found that publicly available datasets were frequently used. Commonly used datasets included Brain Tumor Segmentation Challenge (BraTS) dataset[18], MIMIC-III[19] that contains various encounters from electronic health records obtained from the patients in critical care, and The Cancer Genome Atlas (TCGA) data portal[20] where genomic data of various cancers are available.

Purpose of using deep learning

We classified the reviewed articles based on the purpose of using deep learning: why the study used deep learning over other traditional statistical or machine learning methods? Figure 2C shows the number of articles identified for each purpose of using deep learning. 13 articles had two purposes and hence were included in both categories. 176 articles (53.0%) simply used deep learning based on the successful application in other domains or in similar tasks with other rare disease studies without specifying the purpose. The second most common purpose of using deep learning was to improve the performance of the analytic task (74 articles [22.3%]). Deep learning was used in 64 articles (19.3%) to reduce manual effort for their target analytic tasks, specifically to reduce manual feature engineering effort or automation of target analytic tasks. 31 articles (9.3%) utilized deep learning to mitigate lack of data issue.

Deep learning architecture

We recognized a deep learning models used in the articles include combinations of different architectures. Figure 2D shows the number of articles using each deep learning architecture to build their deep learning models: convolutional neural networks (CNN; 307 articles [92.5%]); autoencoders (AE; 93 articles [28.0%]); recurrent neural networks (RNN; 21 articles [6.3%]); generative adversarial networks (GAN; 18 articles [5.4%]); and self-attention-based architecture (4 articles [1.2%]). Figure 4 visually illustrates the deep learning architectures. 100 articles (30.1%) used more than one deep learning architectures. We discussed the uses of each deep learning architecture with selected representative articles in the following subsections. Details of the architecture for all reviewed articles are available in Supplementary Table 2.

• Download figure
• Open in new tab

Figure 4.

Visual illustrations of deep learning architectures. (A) Autoencoder (AE). (B) Convolutional neural networks (CNN). (C) Generative adversarial networks (GAN). (D) Recurrent neural networks (RNN). (E) Transformer encoder and BERT.

Autoencoders (AE)

Autoencoder (AE) is a neural network architecture that is trained to reconstruct a given input with internal hidden layers consisting of two parts: encoder and decoder[21]. Encoder transforms an input data into a representation in a distributed space and decoder reconstructs the input from the transformed representation (Figure 4A), which learns useful information of the input data in an efficient way, generally having low dimensionality. Image data (83 articles) were frequently used with AEs (Figure 5A). The most common analytic task is diagnosis (Figure 5B).

• Download figure
• Open in new tab

Figure 5.

(A) A heatmap of deep learning architecture vs. data type and (B) deep learning architecture vs. analytic task. The number in each cell in the heatmap represents the number of articles that used the corresponding deep learning architecture and data type in (A) and the number of articles that used the corresponding deep learning architecture and analytic task in (B). The color of each cell in the heatmap represents the relative prevalence ratio (RPR) based on deep learning architecture. RPR for each cell was calculated as dividing the prevalence of the metadata element pair based on deep learning architecture (the number in a cell / row sum) by the baseline prevalence of the deep learning architecture. The baseline prevalence of each deep learning architecture is calculated as the number of articles identified with each deep learning architecture divided by the total number of articles. RPR was clipped by maximum value=3.0 for visualization.

The transformed representation by using AEs was often directly used for target analytic tasks with supervision[22-26] or used for clustering of the data[27-29]. For example, [22] used transformed representation by encoder to predict survival of Neuroblastoma patients. In [27, 28], omics data were encoded by AEs and then transformed representations were used to cluster tuberculosis patient and subtypes of neuroblastoma respectively. U-Net[30] and its variants (e.g., V-Net[31]), which have encoder-decoder architecture combined with convolutional neural network, are predominantly chosen architecture in the most of image segmentation tasks for rare disease. For example, [32] used U-Net for segmentation of cardiac magnetic resonance imaging to predict prognosis of patients with tetralogy of Fallot.

Variational autoencoder (VAE)[33] has characteristics of generative models and autoencoders and was used to learn efficient representation of input since the encoding of the input is restricted by prior distribution. For example, [34] used VAE to learn efficient encoding of input data for status prediction of glioblastoma patients and [35] used VAE with convolutional neural networks to cluster atomistic molecular dynamics for Gaucher disease research.

Convolutional neural networks (CNN)

CNNs are neural networks designed for data having grid-like topological features (Figure 4B). CNNs’ impressive capability to learn local conjunctions of features has shown significant success in many practical applications[5]. Image data (283 articles) were predominantly used with CNNs, followed by measurement data (21 articles) (Figure 5A).

Various state-of-the-art CNN architectures were used in the articles. Commonly used CNN architectures includes but not limited to VGGNet[36], GoogLeNet[37], ResNet[38], Inception-V3[39], and DenseNet[40]. A few studies proposed novel architectures based on the existing CNN architectures for their target rare diseases and analytic tasks. For example, [41] proposed a novel architecture for tuberculosis detection based on chest X-ray using ResNet and attention; [42] proposed a novel architecture for retinitis pigmentosa detection based on optical coherence tomography using the idea of ResNet and DenseNet.

DeepMedic[43], a CNN architecture especially developed for 3D brain MRI, is one of the widely used CNN architectures for image segmentation tasks for rare brain tumors. For example, DeepMedic was used to diagnose glioblastoma[44-46], primary central nervous system lymphoma[47], and meningioma[48, 49]. As mentioned above, state-of-the-art CNN architectures combined with encoder-decoder architecture (e.g., U-Net[30]) are also widely used in image segmentation tasks for rare disease.

Transfer learning was a common strategy to solve insufficient data problem for CNNs. In the studies that used CNNs, CNNs were often pre-trained with publicly available datasets such as Imagenet[50] or MS-COCO[51], then pre-trained CNNs were transferred to be used for target analytic tasks. In several studies, datasets that are more closely related to their target analytic tasks were used. For example, [45, 47, 48, 52] used brain MRI dataset to pre-train CNNs for their brain MRI segmentation task. [53] reported improved performance by using CNNs pre-trained with task-related datasets than CNNs pre-trained with Imagenet.

CNNs were used with other data types by utilizing CNN’s capacity to learn patterns in data such as genomic data[54, 55], EMG signals[56], electroencephalography[57], protein-protein molecular dynamics[35], and published rare disease literature[58]. For example, [54] applied CNN to SNP maps to predict occurrences of Down syndrome. [56] converted EMG signals to 2-dimensional time-frequency representation and applied CNN to classify amyotrophic lateral sclerosis. [58] used CNN to find the relationship between disabilities and rare diseases in published rare disease literature.

CNNs were also used to learn representations of the nodes in graph data. For example, [59] used graph convolutional networks (GCN)[60] to learn the representations of rare disease phenotypes from disease ontology graph where phenotypes are connected nodes; [61] applied GCN to antiplasmodial hit compounds dataset for discovery of antimalarial drugs; and [62] used GCN to learn representations of diseases using disease knowledge graph in generating synthetic radiology report for rare disease patients.

Generative adversarial networks (GAN)

GANs are generative models that can generate synthetic data through an adversarial training of generator and discriminator in the models[63]. The generator is trained to generate realistic input-like data, while the discriminator is trained to distinguish between the generated data and real input data (Figure 4C). GANs were only used with Image data (14 articles) and encounter data (4 articles) (Figure 5A). While 50% of the studies used GANs aimed to diagnosis tasks, 6 articles, which are classified as others, purely focused on synthetic data generation without specifying analytic task (Figure 5B).

GANs were often used to augment data for training. This is especially useful when there was lack of data or supervision (i.e., labels) for training a model, which is important for rare disease research that often suffers from lack of data. For example, [62, 64] used GAN to generate synthetic X-ray images and encounter visits of rare disease patients. In [65-67], GANs were used to learn efficient representation from unlabeled data in semi-supervised setting (i.e., lack of supervision) to detect rare disease patients. [68] aimed to generate realistic synthetic CT images based on magnetic resonance images using CNN-based GAN. Besides utilizing GANs for generating synthetic data, the adversarial training scheme has also been used to improve the performance of the model for an analytic task[69-71].

Recurrent neural networks (RNN)

RNNs are a family of neural networks for modeling sequential data [21]. RNNs consist of RNN cell that is specially designed to process input sequence at each sequential timestamp (Figure 4D). Among many RNN cell variants, Long Short-Term Memory (LSTM)[72] was the most frequently used in the articles that used RNNs. Measurement data and encounter data were more frequently used with RNNs (Figure 5A). While the most common analytic task is diagnosis, prognosis and characterization were more frequently involved with RNNs (Figure 5B).

Accumulation of medical history for accurate diagnosis of rare diseases often takes several years. RNNs are capable of learning long-term dependencies and can be suitable architectures for diagnosis and prognosis of rare diseases[64, 66, 67, 73]. For example, [73] used RNN to learn from longitudinal billing codes for mortality prediction of rare disease. Likewise, longitudinal measurement was also found to be a promising data type to utilize the RNNs[74-76]. Sequential images of a rare disease patient (e.g., MRI) were applied to RNNs for prognosis of the disease[77, 78]. Multiple slices or pixels of a single image of a rare disease patient were applied to RNNs to detect the region affected by the disease[71, 79].

Bidirectional RNNs often outperform unidirectional RNNs by utilizing two directional RNNs instead of the single direction of left-to-right RNNs[80]. There were a few studies that used bidirectional RNNs: [78] used bidirectional RNN for the prognosis of glioblastoma patients; [58] used bidirectional RNN to identify the relationships between disabilities and diseases from rare disease corpus; and [81] used CNNs and bidirectional RNN to classify medulloblastoma.

Self-attention-based architectures

Self-attention-based architectures utilize attention-weighted features where attention weights are calculated based on the relevance to the downstream task and have gained popularity due to its impressive performance in many application domains. Transformer[82] and BERT[83], the two most widely used self-attention-based architectures, were used in the articles (Figure 4E). The articles used self-attention-based architectures only focused on diagnosis task (Figure 5A). Encounter data were predominantly used with self-attention-based architectures (Figure 5B).

[67] used Transformer to learn representations of conditions and drugs recorded in visits of patients for identifying rare disease patients. [84] used Transformer to generate radiology report for rare diseases in data-scarce setting (i.e., few shot learning). [85, 86] used BERT to identify rare disease patients with encounter data.

Rare diseases and rare disease groups

83 unique rare diseases were identified from the reviewed articles. Table 4 shows the 10 most frequently studied rare diseases. Multiple rare diseases were often studied in one article. We classified the rare diseases identified from the reviewed articles into rare disease groups using the hierarchical structure defined by Orphanet. A rare disease can qualify for multiple groups and resulted in the 83 unique rare diseases being classified into 30 rare disease groups. Figure 6A shows the 10 most frequently studied rare disease groups in the reviewed articles, with the three most common being rare neoplastic diseases (75.3%), rare genetic diseases (51.2%) and rare neurological diseases (38.3%), respectively. Figure 6B shows the number of articles identified for each challenge. The full list of rare disease groups with the count of articles for each rare disease group is available in Supplementary Table 3.

• Download figure
• Open in new tab

Figure 6.

(A) The 10 most frequently studied rare disease groups in the reviewed articles. (B) The number of articles identified for each challenge.

Challenges in Using Deep Learning for Rare Diseases

We classified the challenges in using deep learning for rare disease research reported by the authors into eight categories. Insufficient data is the most common challenge in the reviewed articles and were reported by 144 articles (43.4%). The size of the data used in the studies was relatively small compared to studies in other domains. For instance, the median size of image data used in the studies was 3,843 while the size of commonly used datasets for image classification task was from tens of thousands to tens of millions[50, 87]. 104 articles (31.3%) claimed that enhancing data quality can improve the performance of analytic task but come at the expense of manual examination and annotation of the data by experts (e.g., addressing selection bias or class imbalance of data), which is difficult to scale.

45 articles (13.6%) reported the lack of robust evaluation, primarily due to small size of data for evaluation and lack of clinically meaningful evaluation. Many studies attempted to overcome this challenge by using data augmentation and publicly available datasets. The increasingly available shared data can potentially mitigate this challenge. The need for external validation of a deep learning model was also frequently noted (41 articles [12.3%]). External validation of a deep learning model is often based on publicly available benchmark dataset. However, constrained by patient privacy concerns, developing publicly sharable healthcare datasets is difficult, unlike other domains to which deep learning is commonly applied such as natural language processing or computer vision. These restrictions limit evaluation of studies using external data. 38 articles (11.4%) pointed out that the studies can improve by including more state-of-the-art deep learning architectures. 29 articles (8.7%) reported difficulties in training deep learning models. 23 articles (6.9%) were concerned with models’ limited generalizability to other rare diseases.

Improving interpretability is a pressing need in using deep learning to healthcare and medicine[88, 89]. While many studies achieved good performance on their target analytic tasks, 24 articles (7.2%) claimed that the studies need better interpretability of the model. Few studies attempted to improve interpretability using various methods such as Grad-CAM [90] and SHAP[91] but through understanding of how the model(s) are converting inputs to outputs in order to understand how they are achieving their analytic task is still restricted[92-98].

“Architecture-data type” conjugates

Our analysis showed that CNNs were the most frequently used deep learning architecture, and image data were the most frequently used data type. We observed that most of the studies used CNNs with image data, which shows rare disease research using deep learning has been driven by the conjugate of CNNs-image data. This is primarily because CNNs have gained more attention than other deep learning architectures resulting in sufficient available data and readily accessible source codes to implement the deep learning architectures. Image data in healthcare (e.g., CT and MRI) are often generated with a standardized protocol and can be de-identified, thus making data available for rare disease research using deep learning. As a result, some rare diseases where medical imaging is more commonly used for diagnosis or prognosis have been more frequently investigated using deep learning: tuberculosis research using chest X-ray and malaria research using blood smear image. This “deep learning architecture-data type” conjugate was also observed in other deep learning architectures. AEs were often used with measurement data, particularly with omics data. AEs were used to transform high-dimensional omics data into low-dimensional representations containing useful information. Encounter data were generally used with RNNs due to their longitudinal nature.

To overcome difficulties underlying rare disease research using deep learning

Most studies applied the off-the-shelf deep learning architectures to their analytic tasks without modifications of the architectures in consideration of the characteristics of the target analytic task or data type. We found that more than half of the articles used deep learning as proof-of-concept or without clarifying a purpose. This perhaps indicates that current rare disease research using deep learning has been driven by simply adopting deep learning architectures that showed successful performance in other application domains such as computer vision or natural language processing.

Deep learning was used to reduce manual effort for the target analytic task and to address lack of data issue in 65 and 31 articles respectively. These two purposes are worth emphasis since they are closely related to the underlying difficulties of rare disease research. Reducing manual effort for rare disease analytic task is important since rare disease research and its clinical application often suffers from the high cost of obtaining resources. For example, timely and accurate diagnosis of a rare disease in low-resource environments such as developing countries is difficult since large amount of time and resources of experts is required.

Lack of data is one of the biggest hurdles for rare disease research using deep learning, as emphasized in the analysis of challenges in using deep learning for rare diseases. In the articles with the purposes to mitigate the lack of data issue, diverse approaches with deep learning were used to achieve this goal. Several articles attempted to develop novel deep learning architecture for their analytic tasks under the lack of data condition[57, 73, 99-105]. One of the frequently used strategies for addressing the scarce data problem is data augmentation, by either modifying the original data[92, 106-110], or generating synthetic data[62, 64-67, 111]. Transfer learning after pre-training of the deep learning models on other large datasets was also used to mitigate the lack of data issue[112-116].

Future directions

First, the amount of data is often insufficient to fully explore the potential of deep learning for rare diseases. While insufficient data was the most frequently identified challenge in the reviewed articles, only about 10% of the entire articles sought to resolve the challenge in their studies. Additionally, only a few studies among those articles leveraged deep learning to tackle the challenge of insufficient data rather than simple data augmentation using the existing data. We expect to see more studies tackle this challenge using deep learning, which will be beneficial for improving rare disease research.

Second, the current evaluation of deep learning architectures on analytic tasks for rare diseases is limited. More robust evaluations should be conducted with external validation and the broad inclusion of deep learning architectures. However, external validation requires data sharing. One approach to overcome barriers in data sharing is to use a common data model (CDM). While there are several available CDMs (e.g., Observational Medical Outcomes Partnership CDM[117]), existing CDMs were developed for common diseases, thus their compatibility with rare diseases needs to be investigated. Inclusion of more deep learning architectures for evaluation requires additional efforts. A benchmark study that includes comparison of various deep learning architectures on a specific analytic task will be beneficial for providing robust evaluation.

Third, current deep learning models that have been used in rare disease research lack interpretability. Improving interpretability has been considered particularly important in using deep learning to research in healthcare and medicine[88, 89] and has been noted in the deep learning community. Since lack of interpretability makes it difficult for clinicians and healthcare providers to understand and adopt deep learning, the lack of interpretability should be addressed.

While application of deep learning to rare diseases in the real-world clinical setting is an important advance, none of the reviewed articles sought to apply deep learning in the real-world clinical setting. This is perhaps because of additional challenges to the aforementioned gaps to utilize deep learning for rare disease research. For example, a single institution or hospital only can see a small number of rare disease patients, which makes it challenging to apply deep learning for rare diseases. Another challenge is that deep learning models often require large amount of time for training and for conducting the analytic task. This can be a critical challenge in the real-world clinical application, where obtaining results of the analytic task in a timely and swift manner is important. Although there exist many challenges, we believe the field will benefit from studies that attempt to perform real-world application of deep learning for rare diseases.

Limitations

There are some limitations to our study. First, identification and classification of rare diseases was solely based on Orphanet, thus there could be discrepancies based on other rare disease knowledge sources even though Orphanet provided monthly-updated rare disease definitions based on published scientific literature and expert reviews. For example, infectious diseases such as tuberculosis and malaria are often not recognized as rare diseases but we included them based on the Orphanet. Second, since our analysis did not evaluate the performance of deep learning architectures on their analytic tasks, our findings cannot be used to evaluate or compare different deep learning architectures on a specific analytic task.