ELM and K-nn machine learning in classification of Breath sounds signals

Received Apr 7, 2019 Revised Jan 6, 2020 Accepted Jan 29, 2020 The acquisition of Breath sounds (BS) signals from a human respiratory system with an electronic stethoscope, provide and offer prominent information which helps the doctors to diagnosis and classification of pulmonary diseases. Unfortunately, this BS signals with other biological signals have a non-stationary nature according to the variation of the lung volume, and this nature makes it difficult to analyze and classify between several diseases. In this study, we were focused on comparing the ability of the extreme learning machine (ELM) and k-nearest neighbour (K-nn) machine learning algorithms in the classification of adventitious and normal breath sounds. To do so, the empirical mode decomposition (EMD) was used in this work to analyze BS, this method is rarely used in the breath sounds analysis. After the EMD decomposition of the signals into Intrinsic Mode Functions (IMFs), the Hjorth descriptors (Activity) and Permutation Entropy (PE) features were extracted from each IMFs and combined for classification stage. The study has found that the combination of features (activity and PE) yielded an accuracy of 90.71%, 95% using ELM and K-nn respectively in binary classification (normal and abnormal breath sounds), and 83.57%, 86.42% in multiclass classification (five classes).


INTRODUCTION
Recently many researches in the Breath Sounds (BS) areas is conducted by applying many techniques for analyzing and classify a respiratory signal in order to diagnosis the pulmonary pathology, such as (Asthma, COPD, Pneumonia…). However, the acquisition of lung sounds plays an important role to detect and identification of pulmonary diseases. The doctors listening to the lung sounds through the stethoscope placed on the chest or posterior to the patient which is suffering from a type of lung disease, but the problem is sometimes the decision of the doctors concerning types of pathology is not accurate, this is due to many reasons such as a few experiences concerning auscultation and diagnosis. To address this problem a research study has been widely conducted in this area focus on three principal domains (time domain, frequency domain and time-frequency domain).
Breath sound is a complex signal [1] such other biological signals, have a non-linear and non-stationary nature, these properties of the signal lead it to assessed by different techniques in signal processing have the same property of the breath signals. However, these techniques used for three transformation domains which are mentioned previously. In [2] [5] diagnosis the lung diseases (interstitial lung diseases) by using empirical observation as proposed solution with an overall accuracy of 90.0%, A.Cheema, M.Singh [6] use an EMD method for detect Psychological stress from phonocardiography signal the average accuracy of 93.14% to classifying stressed and non-stressed, in [7] R. Palaniappan classify a pulmonary signal using Autoregressive Coefficients and k-Nearest Neighbor as a classifier with an accuracy of 95.18%. In this work we analyzed a breath sounds signals using empirical mode decomposition with Hjorth descriptors (Activity) and Permutation entropy as features, were extracted from each IMFs produced by EMD, finally,a comparative study has been assessed between an extreme learning machine (ELM) and K-nearest neighbor (K-NN) for distinguishing between normal, and adventitious respiratory sounds.

RESEARCH METHOD
Our work was divided into two principal stages namely (Multiclass classification stage, Binary classification stage). The four steps proposed for both stage study namely (database, pre-processing, feature extraction and classification) are presented in Figure 1(a) for multiclass classification, and the second stage is represented in Figure 1

Database
In this paper the database of breath sounds signals used for analysis are the R.A.L.E (Respiration Acoustic Laboratory Environment) Lung Sounds, is the only commercially available database, is an educational program to help doctors and researchers in respiratory signals processing area offer more than 50 breath sounds were recorded using a contact accelerometer (Siemens-EMT25C) covering normal and abnormal respiratory sounds [8] are sampled at 10,240Hz. As this database (R.A.L.E) has a few data, therefore to ensure the credibility of this comparative study we used another data were collected from the internet: The CD of the book [12] In all a 75 breath sounds divided into five classes (Normal bronchial, Wheeze, Crackle, Pleural rub, Stridor) were used in our study, each sound is an effect of particular disease such as Wheeze indicate that the patient suffering from asthma and COPD (Chronic Obstructive Pulmonary diseases), crackle indicate pneumonia or lung cancer.

Breath sounds pre-processing
Breath sounds signals are subject to several artefacts such as heart sounds and noise which simulate real-life conditions. The breath sounds signals (R.A.L.E) that have been filtered by a high-pass filter with 7.5 Hz by 1st order Butterworth to remove DC offset, and a low-pass filter at 2.5 Hz by 8 th order Butterworth [13,14] , and concerning the data were collected we apply a mean and amplitude normalization to reduce the effect of heart sounds. Finally, all samples are downsampled to 8000 Hz sampling frequency according to CORSA (computerized respiratory sound analysis) [15], in this study, the 16-bit resolution and one respiratory cycle are used.

Empirical mode decomposition
The Empirical Mode Decomposition (EMD) method is a new adaptive signal time-frequency processing method proposed by NE Huang in 1998 by NASA and others [16]. It is especially suitable for nonlinearity, analysis and processing of non-stationary signals. The Hilbert transform transforms the wellknown Hilbert-Huang Transform (HHT).
EMD is actually a method of decomposing signals. It is consistent with the core idea of Fourier transform and wavelet transform. Everyone wants to decompose the signal into a superposition of independent components, only the Fourier transform and the wavelet transform it is required to have a basic function, but EMD completely abandons the constraint of the basis function, and only performs signal decomposition based on the time scale feature of the data itself, and has adaptability. Since no basis function is required, EMD can be used for almost any type of signal decomposition, especially for the decomposition of nonlinear, non-stationary signals [17,18]. The purpose of EMD is to decompose the signal into a superposition of multiple intrinsic mode functions (IMFs). In addition, the IMF must satisfy the following two conditions( the function must have the same number of local extreme points and zero crossings within the entire time range, and at any point in time, the envelope of the local maximum the envelope of the (upper envelope) and the local minimum (lower envelope) must be zero on average.
The EMD method is based on the:  The signal has at least two extreme points, one maximum and one minimum.  The characteristic time scale is defined by the time between the two extreme points.  If the data lacks extreme points but has deformation points, the extreme points can be obtained by data differentiation once or several times, and then the decomposition results are obtained by integration. The algorithm flow is as follows: -Identify all extrema of x(t) -Interpolate between minima (resp. maxima), ending up with some envelope emin(t) (resp. emax(t)) -Compute the mean m(t) = (emin(t)+emax(t))/2 -Extract the detail d(t) = x(t) − m(t) -Iterate on the residual m(t)

Features extraction
A helpful feature for express a biomedical signal namely Hjorth descriptors (HD) divided into three main parameters as follows: -Activity: is the most useful parameters in biological signals, simply its variance of the signal represents the energy: -Mobility: Mobility is given by: -Complexity: gives a computational value for the shape of the signal: -Permutation Entropy: Bandt and Pompe are investigated the (PE) Permutation entropy to measure the complexity of the non-linearity and non-stationary nature in time series signals [19]. the Shannon entropy is calculated in PE for the different symbol in the signal and can be calculated as follows:

Statistical analysis
In this study, a statistical analysis of mean and standard deviation was used to test the significance of the activity and PE features. SD and Mean are expressed respectively as follows: where: each value of the dataset. M the total number of data points.

Classification
In this study, two classifiers were used for two classification types (multiclass classification, binary classification), one is the extreme learning machine (ELM) and the other is a k-nearest neighbour (K-NN). detailed of these classifiers are presented in this section:

Extreme learning machine
Huang et al. [20] propose an algorithm for solving a single hidden layer neural network which is an extreme learning machine (ELM). The biggest feature of ELM is that traditional neural networks, especially concerning a single hidden layer feedforward neural networks (SLFNs), are faster than traditional learning algorithms while guaranteeing learning accuracy [21].
For a single hidden layer neural network shown in Figure 2, assume that there is an arbitrary sample ( , ) of which [22]: For a single hidden layer neural network with a hidden layer node, it can be expressed as: Among them ( ), the activation function, which: = [ ,1 , ,2 , … , , ] is the input weight and the output weight, is the offset of the first hidden layer unit.
. Representation and inner product.. The goal of a single hidden layer neural network learning is to minimize the error of the output, which can be expressed as: That exists , and so that Can be expressed as a matrix = Among them it is the output of the hidden layer node, which is the output weight and is the expected output.
= ( 1 , … , , 1 , … , , 1 , … , ) In order to be able to train a single hidden layer neural network, we hope to get 1 , and to make: where = , … , this is equivalent to minimizing the loss function Some traditional algorithms based on the gradient descent method can be used to solve such problems, but the basic gradient-based learning algorithm needs to adjust all parameters during the iterative process. In the ELM algorithm, once the input weight and the bias of the hidden layer are randomly determined, the output matrix of the hidden layer is uniquely determined. The training single hidden layer neural network can be transformed into a linear system = and the output weight can be determined ̂= ϯ Among them ϯ is the Moore-Penrose generalized inverse of the matrix. And it can be proved that ̂ the norm of the solution obtained is minimal and unique [22].

K-nearest neighbour
The K-Nearest Neighbors (K-NN) algorithm is a classification algorithm and one of the easiest to understand machine learning algorithms. In 1968, it was proposed by Cover and Hart [23]. The simplest and mundane classifier may be the kind of memorable classifier, remember all the training data, for the new data, it matches the training data directly, if there is training data of the same attribute, use it directly, come as a classification of new data.The k-NN algorithm finds the k records closest to the new data from the training set and then determines the category of the new data based on their primary classification.
The algorithm involves three main factors: -The training set -The distance or similar measure. In this study a Euclidian distance has been used: In the validation stage, the dataset X is divided into a training set Y (training set) and a test set Z (test set), for the case that the sample size is insufficient such as in our study, and in order to full use of all data set to test the algorithms effect, database X is randomly divided into k packets, one of which is used as a test set each time, and the remaining k-1 packets are trained as a training set, by using k-fold crossvalidation method [24].

RESULTS AND DISCUSSION
In this work the all experiments were performed by using MATLAB R2013b and on a pc with a configuration of Intel CPU Core i5, 4 GB RAM, and Windows 10 operating system. In [25] the authors apply Hjorth descriptors as features and find that the activity feature is the best feature compared with mobility and complexity as shown in (1), (2) and (3). Therefore, in our work, the activity feature was exploited for enhanced this study with combined it with the permutation entropy feature shown in (4), and formed a features vectors to fed into two machine learning algorithms namely ELM and K-NN, to compare them in the classification of breath sounds signals. The EMD decomposes BS signals into a set of IMFs, Figure 3 shows some IMFs of the normal subject. The features (Activity and Permutation Entropy) were extracted from each IMF and tested using a statistical measure of (mean and standard deviation SD described in (1) and (2)) as tabulated in Table 1.  From Table 1 we inferred that there is significant discrimination in the activity and PE features of different classes. Can be observed a mean and SD are different from each class in activity features, but in PE features a little different between classes. From this, we can combine theme to test and compare the classification accuracy of both K-nn and ELM classifiers. These features have been formed as follows: In order to verify the reliability of the outcome of the classifiers, the k-fold cross-validation was used. Figure 4 shows how the ten-fold works. After several tests to choose the k value, we found that k=10 is promised value, therefore it has been used in this study. In the literature review many researchers based on activity or entropy features extraction, nevertheless, this study has combined both activities and PE features for observed the ability of both ELM and K-nn to classify different BS signals. In Table 2 the classification stage is described, and give the classification performance of features (Activity, PE) extracted from IMFs using ELM with RBF Kernel, Polynomial Kernel and K-nn with distance euclidian which is described in equation (17), and 1 to 10 number of neighbours. neighbours. We can say that the ability of the K-nn is higher than ELM in the classification of the Breath sounds signals into several classes (Normal bronchial, Wheeze, Crackle, Pleural rub, Stridor). We can be seen in Table 3, the accuracy found from k-nn is 95% by 6-8-10 neighbours and from ELM with Polynomial Kernel is 90.71% better than RBF kernel in binary classification case. However, according to these results, we can say that, this comparative study shows that the capability of the k-nn classifier is higher compared with that of the ELM classifier in the classification of breath sounds signals from our test conditions. The ability of the k-nn is higher than ELM in the classification of the breath sounds signals into binary and multiclass classification cases. Table 3. Classification performance of (Activity, PE) from IMFs of BS signals in binary classification stage

CONCLUSION
In this study, the performance of the ELM and K-nn classifiers were compared using the Hjorth descriptors (Activity) and Permutation Entropy (PE) features in distinguishing between breath sounds signals with combination these features (Activity, PE). The features extracted were analyzed statistically by calculating a mean and standard deviation to observe the difference between them for each class (Normal bronchial, Wheeze, Crackle, Pleural rub, Stridor). The classification accuracy in multiclass classification case of the ELM and k-nn classifiers is 83.57% and 86.42% respectively, and in binary classification case, the accuracy is 90.71% , 95% respectively. These show that the ability of k-nn in our test conditions (database, methods of analyses the breath signals, and features used) is higher than the ELM classifier in multiclass and binary classification. In future work, the EMD methods will be compared with another method for further analysis of breath sounds signals using a large database.