Computational Image Analysis Techniques, Programming Languages and Software Platforms Used in Cancer Research: A Scoping Review

1 Introduction

Cancer has dominated the medical literature. At the time of writing of this paper (June 2022), there were 4,633,885 Cancer-related entries from the total 34,216,925, which represented 13% of all entries in PubMed (https://www.ncbi.nlm.nih.gov/pubmed), the search engine of the United States National Library of Medicine MEDLINE. These entries have risen from 6% of all yearly entries recorded in PubMed in the 1950s, to more than 16% seventy years later [19]. Among many technological innovations that have had an impact in medicine in general, and in Cancer in particular, computational data analysis techniques have been important, especially as new imaging technologies provide more and more data every year. Diagnostic images acquired with Magnetic Resonance Imaging [1], Computed Tomography [27], Ultrasound [8], and pathology staining [12, 25] are routinely obtained and analysed, traditionally by experts that trained for many years to provide accurate diagnosis. In addition, computational techniques have been used to support decision-making [28], preoperative planning [23] and predict survival based on the analysis of the images acquired [7].

In recent years, the computational techniques included under the umbrella term of Artificial Intelligence have grown at a very fast rate, but before that, a rich body of techniques developed under the areas of Image Processing and Computer Vision were applied to analyse data from a variety of Cancer data sets. Techniques like Fourier analysis [4], mathematical morphology [22], and fractals [13] among many others have been applied with success in the analysis of Cancer treatments or drug development [15, 16, 18]. The question that arises is then, are the artificial intelligence techniques displacing all the previous image processing and computer vision techniques as applied to the analysis of Cancer images?

In this paper, a review of the data analysis techniques that have been applied to Cancer data analysis is presented. Since the objective of the review is to map the literature, assess the volume of work, as reflected by the number of entries in PubMed, rather than compare outcomes or perform a precise systematic review and synthesis of the evidence [17, 26], a scoping review, rather than a systematic review was performed. The review utilised data mining techniques to extract the entries related to different techniques that have been applied to the analysis of Cancer images. A series of keywords were used in combination to select different techniques, and their yearly entries in PubMed. In addition, the tools through which these techniques were applied, i.e. the software platforms or programming languages were also reviewed.

2 Materials and Methods

The scoping review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [11, 14] and the extension for Scoping Reviews (PRISMAScR) guidelines [17, 26] and included the following stages: (1) Identify the research questions, (2) Identify data source and the combination of keywords that would retrieve the relevant entries, (3) Mine the number of entries per year of publication, (4) Display the trends graphically, (5) Interpret the findings. One notable difference of this review is that the screening and eligibility steps were performed automatically by sequentially adding a series of specific keywords to a basic search instead of excluding entries manually or with criteria other than the presence of keywords. The inclusion was solely based on the specific keywords being present in the query as described below.

3 Results and Discussion

During the initial period of study, 1990-2022, the analysis techniques that had the higher number of entries were those identified by the keywords: machine learning, tracking and deep learning with more than 1,500 entries in PubMed. These were followed by five keywords with under 1,000 entries: linear regression, convolutional neural, Fourier, texture analysis and clustering. On the other side, the keywords which returned the lowest number of entries corresponded to: mutual information, region growing, multiresolution, self-organizing, hessian and scale space (Fig. 2).

• Download figure
• Open in new tab

Fig. 1.

PRISMA statement flowchart illustrating the details of the scoping review. Notably, the inclusion was given by the presence of keywords on PubMed for individual searches. Studies were not read individually.

• Download figure
• Open in new tab

Fig. 2.

Number of entries in PubMed corresponding to data analysis techniques as determined by specific keywords. The queries were formed with the combination of the following terms (keyword) AND (pathology) AND (cancer) AND ((image) OR (imaging)) AND (1990:2022[dp]). This last term corresponds to the date of publication.

The year-on-year analysis of these same keywords showed some interesting trends and a rather different story (Fig. 3). Whilst it would be expected that the number of entries would increase for all keywords, as would be expected since the number of entries in PubMed have grown year on year, from 410,894 entries in 1990 to 1,763,742 entries in 2021 (https://pubmed.ncbi.nlm.nih.gov/?term=(1990[dp]), https://pubmed.ncbi.nlm.nih.gov/?term=(2021[dp])), some cases seem to grow faster. It is clear that some techniques, namely machine learning, deep learning and convolutional neural, have accumulated their entries quite recently as opposed to tracking, linear regression and Fourier. Clustering, texture analysis and feature extraction also seem to show an increase, but far less pronounced and difficult to appreciate in this graph. For this reason it is important to observe the trends in relative terms. The ratio of entries of every specific keyword divided by the sum of the entries of all the keywords in every year is shown in Fig. 4. This graph shows the relative contribution of each keyword against the rest for a particular year and emphasises the relative decrease of certain techniques, like Fourier and linear regression. It should be noticed that ratios of these two techniques were close to 30% in the early 90s. Tracking accumulated the majority of its entries in the 2000s but has shown a steep decrease lately.

• Download figure
• Open in new tab

Fig. 3.

Number of entries in PubMed corresponding to the specific keywords and the date of publication. Colours have been allocated to the ribbons and the names to aid visual discrimination of the computational techniques.

• Download figure
• Open in new tab

Fig. 4.

Relative number of entries in PubMed per keyword and date of publication. The number of entries was divided by the sum of all the entries corresponding to the keywords of a given year and thus emphasises relative decrease of certain techniques. Colours have been allocated to the ribbons and the names to aid visual discrimination of the computational technique.

The number of entries for machine learning specific keywords is illustrated in Fig. 5. The first point to notice is that the vertical scale is logarithmic, as the number of entries for Regression and Classification was much higher than the rest, and this would be expected as these two terms encompass other terms, for instance, classification with random forests. However, these are useful to give perspective to the following entries. Deep learning is one order of magnitude above the next keywords; Feature Selection, Random Forest, Discriminant Analysis and Cluster Analysis. On the lower side, the keywords with fewer entries were Feature Engineering, Reinforcement learning, Anomaly detection and Hidden Markov Model. It is important to highlight at this stage, that the queries only returned what is indexed in PubMed. If the work of a certain entry did use a technique, say, feature engineering, but this appeared in the methods of the paper, but not in the fields of the PubMed entry, that entry would not be counted in the query. As an example of the data that was mined with the queries, i.e., title, abstract, MESH terms, etc., the following URL https://pubmed.ncbi.nlm.nih.gov/31783023/?format=pubmed corresponds to the work by Lee et al. [10].

• Download figure
• Open in new tab

Fig. 5.

Number of entries in PubMed per specific keyword related to Machine Learning Techniques. The terms “Regression” and “Classification” may include other keywords (e.g., Classification with Random Forest) and thus are more frequent. It can be seen that the term “Deep learning” is one order of magnitude higher than all others. The vertical axis is logarithmic.

When visualised per year, the trends are again very interesting (Fig. 6). In general, the entries corresponding to all specific keywords seem relatively stable, except for one, deep learning, which has grown significantly since 2010 and is very close to the entries corresponding to regression and classification. It can be speculated that in a few years, there will be more entries for deep learning than for the other keywords.

• Download figure
• Open in new tab

Fig. 6.

Number of entries in PubMed per year and specific keyword related to Machine Learning Techniques. It should be noticed that entries related most techniques are fairly constant except for “Deep learning” which has increased constantly. It should be noticed that the vertical axis is logarithmic.

The results for specific deep learning keywords are shown in Fig. 7, where the most common term is convolutional neural network, followed by U-Net and ResNet. Convolutional Neural Networks, commonly known as CNNs or Con-vNets, is almost one magnitude higher than all the other methods. The keywords with fewest entries were Auto-Encoders, Transformers and Mask-RCNN. It should be noticed that some of these terms are rather recent and thus would not be accumulating many entries in PubMed yet.

• Download figure
• Open in new tab

Fig. 7.

Number of entries in PubMed per specific keyword related to Deep Learning Techniques and some specific architectures like AlexNet, VGG and U-Net. The vertical axis is logarithmic.

The entries corresponding to software and programming languages are shown in Fig. 8. These are shown in two ways, as total number of entries (a) and yearly (b), in both cases the vertical axes are logarithmic. Fiji and ImageJ returned the highest number of entries followed by Matlab, R and Python. One order of magnitude below were QuPath, ICY, CellProfiler, and this could be expected as the former are general platforms and languages that can be used for a variety of tasks, whilst the latter are more specialised for specific tasks on Quantitative Pathology, Bioimage Informatics and measuring and analysing cell images. As mentioned previously, the queries were performed in different ways and these did not include the keywords cancer, pathology and image analysis, thus they cover all entries indexed in PubMed.

• Download figure
• Open in new tab

Fig. 8.

Number of entries in PubMed related to eight programming languages: Fiji, ImageJ, Matlab, R, Python, QuPath, ICY, CellProfiler and C. Subplot a are the entries for the keywords for the programming languages and all the previous keywords (pathology) AND (cancer) AND ((image) OR (imaging)) AND (2010:2022[dp]) and subplot b is just for the programming languages and the dates. The vertical axis is logarithmic in both graphs.

The lowest number of entries corresponded to C. The relatively low numbers could be due to several factors. The most obvious one is that the C programming language is not widely used by those researchers whose work is indexed in PubMed. It may also be that the work created in C may result in larger tools, which go under other names, for instance, ICY is java-based software platform. One important point to notice is that Matlab is the only of these software platforms that requires a license fee. Whilst this fee is significant for industry, academics can obtain it at reduced cost and in many times it is part of the universities site license. However, this could affect the number of users compared with other platforms that do not require license fees.

Yearly entries show how Fiji has been overtaking ImageJ in recent years. Although Fiji and ImageJ are very closely related, and for some people these are considered to be the same (the actual acronym of FIJI is a recursive Fiji Is Just ImageJ), with the addition of a set plugins that facilitate image analysis, in this publication they are being treated separately to identify the actual name being used in the entries that referred to them. A similar trend can be observed with R and Python overtaking Matlab. A very noticeable increase in entries is that related to QuPath, which has grown at a much faster rate than any other language/software. If this trend continues, it can be speculated that it will soon reach the same number of entries as the programming languages and eventually close to to ImageJ and Fiji.

Whilst this review did not consider the actual biological/clinical tasks such as finding and counting nuclei, estimation of cellularity, semantic annotations, anomaly detection or segmenting organs, these could be investigated with the same methodology as described in this work.

4 Conclusions

This scoping review provides a panoramic view of the computational techniques that have been applied for the analysis of images produced in Cancer research. The techniques included in the areas of machine learning and deep learning have grown to dominate the recent works indexed in PubMed. Whilst other techniques are still recorded in the records of PubMed, most of these are decreasing their proportions as compared with machine and deep learning.

The tools with which these techniques have been applied, i.e. the software platforms and programming languages showed that general tools through which researchers programme their own solutions are popular, but some specialised softwares like ImageJ and Fiji are widely used, with some newer options like QuPath increasing their position in this area.

Declaration of Interests Statement

The authors declare no conflicts of interest.