Artificial intelligence in the respiratory sounds analysis and computer diagnostics of bronchial asthma

Database

The anonymous database, along with digital audio files containing recordings of respiratory sounds, covers the main characteristics of the examined person: age, sex, health information (healthy, exacerbation of bronchial asthma or its remission), recording time, recording point (see below). The database contains 952 recordings of respiratory sounds of asthma patients whose age is from several months to 47 years and 167 records of healthy volunteers in the same age range. The records are distributed among men and women as 67% and 33%, respectively.

Clinical examination of patients and recording of respiratory sounds were performed at the Regional Children’s Clinical Hospital in Perm (Russia). The examinations were carried out in accordance with the Declaration of Helsinki, adopted in June 1964 (Helsinki, Finland), revised in October 2000 (Edinburgh, Scotland), and approved by the Ethics Committee of E.A. Vagner Perm State Medical University. The written informed consent was obtained from the examined persons (at examination of children - from their parents or guardians) in accordance with the Federal Law “Fundamentals of the Legislation of the Russian Federation on the Protection of Citizens’ Health” from July 22, 1993, No. 54871.

Bronchial asthma in patients was diagnosed in accordance with the recommendations presented in the National Program “Bronchial Asthma in Children. Treatment Strategy and Prevention” [14]. According to a comprehensive medical examination, 232 records were made for patients in a stage of exacerbation, 309 for patients in a stage of remission, and 410 in a stage of incomplete remission. Auscultatory data in patients with bronchial asthma were characterized by hard breathing and presence of diffuse dry wheezing over both lungs. Vesicular breathing was heard over the lungs of healthy volunteers with good conduction of sound to all parts of the lungs. At the time of recording, the volunteers were not suffering from any pulmonary diseases or other diseases causing pathological changes in respiratory sounds.

A. Respiratory sound registration

Respiratory sounds were recorded at four points during quiet breathing: in the oral cavity (point 1, 4.3% of all records), above the trachea (point 2, 50.2%), on the chest, the second intercostal space on the right side, (point 3, 23.6%) and at the point on the back (point 4, 21.9% of all records). For most of the persons examined, the recordings were performed at several points and for some patients also at different stages of the disease: exacerbation, incomplete remission and remission. The recording systems and the quality of the records met the requirements formulated [11]. The quality of the recordings was controlled visually (oscillograms of the recordings were shown on a computer screen) and using the program developed by us.

Respiratory sounds were recorded continuously during several respiratory cycles, approximately for 25 s. It allows us to reduce the effect of random variations of the sound intensity on the analysis results.

Respiratory sounds were recorded using mobile phones and computer systems. The audio files recorded with the help of the phones using both the built-in and the external microphones, were sent to the “cloud” to create an anonymous database. The computer recordings were performed using the developed computer-aided systems for recording respiratory sounds [4,5,7,11], external microphones, electronic phonendoscopes and computer sound cards, Adobe Audition system. All systems demonstrated high amplitude-frequency linearity over a frequency range of 100 Hz to 3000 Hz. The sampling rate varied from 22 kHz to 96 kHz.

Method of data processing

The developed methods of computer-aided diagnostics of pulmonary diseases are usually based on analysis of respiratory sound oscillograms, their spectrograms or Fourier spectra [2-11,15,16]. An alternative approach could be the following: to develop diagnostic methods based on machine learning and deep machine learning techniques. Recently, the studies of the possibility of applying such methods in various fields of medicine have been actively carried out (see [12,13,17-21] and references therein). At present, there is a large number of machine learning methods that allows creating decision-making algorithms suitable for a given situation and able to solve various tasks, including medical ones (e.g., classification of various types of cancer [18], analysis of the effectiveness of dialysis [18], diagnosis of pulmonary diseases [12,13,20,36]). Machine learning methods are proposed to be widely used, in particular, in pulmonology: for sorting patients with chronic obstructive pulmonary disease [21], for automatic classification of respiratory sound spectra in patients with pulmonary diseases [22], for diagnosis of bronchial asthma and chronic obstructive pulmonary disease [23] and ranking of clinical assessments in children with asthma [24].

Among the tasks of machine learning, three main ones can be highlighted: the problems of classification, clustering, and regression [25-29]. At solution of the regression problems, the mathematical relationship between a function and an argument (or arguments) is restored. The solution of the clustering problem, which refers to automatic learning, without user / programmer interference (unsupervised learning), it is designed to divide the data into certain categories according to the selected common features. When solving clustering problems, the number of categories, which will be obtained as a result, is unknown. In the case of automatic diagnostics, it is necessary to solve classification problems, which are similar in formulation to clustering problems, but at the same time, the number of categories is known in advance. The main problem comes down to choosing the most efficient algorithm, which allows achieving the maximum accuracy for the test dataset. The purpose of solving classification problems is to assign a phenomenon or parameter to a specific category, in our case, to establish the correspondence of the respiratory sound to a healthy volunteer or a patient with bronchial asthma. The difficulty in choosing an algorithm lies in the fact that there is a large number of different algorithms for solving clustering problems, and each of them is better suited for some particular class of problems.

Our analysis of the methods showed that two methods can be used with great efficiency to classify audio recordings: analysis using a neural network [29] and analysis using a method called a solution tree [25]. Several solution trees can be combined into a so-called random forest [20], which has an extra accuracy compared with a single solution tree, since the final result in the case of classification problem is determined by the “voting” of all trees which make up the random forest. In some cases, the use of the random forest method for classification gives a higher accuracy in comparison with a neural network, but the advantage of a neural network over the random forest method is that with each incorrect solution of the classification problem, the neural network can be additionally trained under human supervision.

Mathematically, a neural network consists of a set of functions (neurons), each of which possesses its own coefficient of connection with other functions [26]. Thus, when an input signal comes to a neuron, as a result of simple mathematical operations, a decision is made which neuron will be the next to be activated. Thus, the procedure reaches the last stage (the last neuron), the result of which will coincide with one of the categories into which the input data should be divided as a result of solving the problem. In our case, the input data are the digital recordings of respiratory sounds. The recordings are divided into two categories at the output: healthy and sick. In addition to the above noted advantages of the neural networks over the random forest method, it is worth mentioning that a trained neural network can fairly quickly work on relatively low-power computers [27]. The latter advantage of networks is important for development of methods of computer-aided diagnostics in real time, remote screening diagnostics and monitoring of the patient condition, as well as for development of outside medical diagnostics and condition monitoring (for example, parental control over the child condition; the result of such monitoring should be recommendation to visit a doctor in time).

Taking into account the above, we have chosen the neural network method. To process the respiratory sound audio files, the program based on the Python language, version 3.8, using the Keras and Librosa libraries was developed. The first library is used to build a neural network, the second is used for simple and quick analysis of audio files. The time required to analyze an audio file is of the order of a second. To build the neural network, a sequential model was chosen from the standard set of the Keras library. It allows creating a network of several layers, train it and get a forecast for the category (define the category), to which the object should belong (audio file), the characteristics of which are supplied to the input. A neural network consists of at least two layers of the so-called neurons, which are the functions, processing the parameters of the analyzed object and activating the element of the next layer depending on the result of work of the previous layer [25-29].

The developed method is based on the following: the audio recording of respiratory sounds is converted into a time series, the characteristics of which can be investigated by any methods suitable for time series analysis. These characteristics can subsequently be used to train the neural network and develop methods for express analysis. The Fourier spectrum of respiratory sounds of asthmatic patients contains frequency ranges, in which the amplitudes of harmonics significantly exceed the amplitudes in the spectrum of healthy patients (see, for example, Fig. 4 in [3] and Fig. 3 in [10]). There is a periodic manifestation of such an increase in time. The duration of such amplifications is about 200 ms (see, for example, Fig. 5 in [3]). This allows one to visually distinguish the respiratory sounds of sick and healthy persons. Note that asthmatic disease is often accompanied by an increase in respiratory rate. These differences between the characteristics of the respiratory sounds of the patients and the healthy volunteers allowed us to choose the following parameters for training the neural network: spectral bandwidth, spectral centroid, zero-crossing rate, spectral roll-off and chromaticity. Our results have shown that a set of these parameters is sufficient for constructing and training a network, application of which makes it possible to achieve a high level of reliability of computer-aided diagnostics.

A spectral width is determined as a width of the spectrum at half its maximum intensity and shows how much the energy is concentrated or dispersed depending on the frequency.

The spectral centroid determines position of the energy center of the spectrum and is defined as where S (k) is the amplitude of the k -th harmonic with the frequency fk.

The number of zero crossings characterizes the sound recording in terms of the presence or absence of high frequency harmonics. Mathematically, it shows how smooth the signal is.

A spectral roll-off is a measure of the waveform that indicates the frequency after which the amplitudes of the high frequency harmonics drop to zero.

A chroma is a vector usually containing 12 elements. To determine the vector elements, the spectrum is divided into 12 sections (classes) and an element is determined by the amount of energy in the corresponding class. Chroma is used to describe the measure of similarity between signals.

To calculate the parameters noted above, the built-in functions of the Librosa library (chroma_stft, spectral_centroid, spectral_bandwidth, spectral_rolloff, zero_crossing_rate) were used. The functions allow one to calculate these parameters quickly; the records contain up to a million points and use of other software can lead to a significant increase in the calculation time. In the standard library, these functions are optimized and run as fast as possible.

The database of respiratory sound recordings was divided into two groups: the first, training group, which contains 374 records of respiratory sounds of asthmatic patients and 146 records of healthy volunteers, was used to train the network; the second, control one (577 - sick and 21 - healthy), was used to check the operation of the network and the correctness of computer-aided diagnostics. In these groups, the examined persons (their records) were almost equally distributed by age, gender, and recording points. None of the recordings included in the first group was used in the second one. The performed separation is dictated, on the one hand, by the need to “balance” the training group (it is desirable that the numbers of recordings of patients and of healthy volunteers were equal), on the other hand, the number of recordings used to train the network should be large (at least several hundred). In this work, when training the network and checking its operation, there was no division of patients according to the disease stages. To demonstrate the universality of the developed approach, the training group contained recordings made on different devices and at different recording rates from 44 kHz to 96 kHz, the control group - only records performed using a computer at a rate of 22 kHz.

Computer analysis of respiratory sounds was carried out at the Ben-Gurion University (Israel). The developed software based on the Python scripting language was used. TKinter library was used as a graphical interface (GUI) convenient for a user.