A novel intelligent fault diagnosis method of rotating machinery based on deep learning and PSO-SVM

2.1. Deep learning neural network

2.1.1. Autoencoder, sparse autoencoder and denoising autoencoder

DNN is composed of a plurality of sparse Autoencoders (AE). Autoencoders are a three layer of unsupervised neural network, which is divided into two parts: encoding network and decoding network and the structure is shown in Fig. 1.

Fig. 1Architectural graph of an autoencoder

The input and output layers in an autoencoder have the same number of nodes. High dimension of input data can be reduced by encoding network, and reconstructs the coding vector of low dimensional space to the original input data. Because the input signal can be reconstructed in the output layer, the coding vector is a characteristic representation of the input data. Autoencoders should not lose the characteristic information of the input signal as far as possible. In order to realize no losing, the encoder must capture the most important factors which can represent input data. In order to extract the features of input signal without loss of useful information, we introduce the Sparse Autoencoder (SAE). SAE requires that most of the nodes are zero while only a few of the nodes is nonzero (the main character). In order to improve the robustness of the autoencoders, we also introduce the Denoising Autoencoder (DAE). DAE introduces the noise with some statistical characteristics into the sample data, and then codes the sample; the decoding network is estimated in the original form of the sample without noise interference according to the data subject to noise interference. So that DAE can learn from the noisy samples to extract more robust features and reduce the sensitivity of DAE to small random perturbations. The structure of DAE is shown in Fig. 2.

Fig. 2Schematic of a denoising autoencoder

The DAE principle is similar to the human body’s sensory system, such as observing objects. If a small part is covered, people can still identify the object. In the same way, DAE can effectively reduce the influence of random factors on the extraction of health information and improve the robustness of the feature expression by adding noise to the coding and reconstruction. The DNN model which is used in this method is the overlapping sparse denoising autoencoder.

2.1.2. DNN pre-training and fine-tuning

In deep learning, DNN is first trained by the unsupervised method, which can help the DNN to capture the fault characteristics of the input signal effectively. Then, DNN is adjusted by the supervised learning method to optimize the expression of DNN on the fault characteristics. This paper uses denoising autoencoder as a non unsupervised algorithm for pre-training, and uses the BP algorithm as the supervised algorithm for fine-tuning stage.

Pre-training is an unsupervised learning from bottom to top. First, the first layer is trained with the training sample data, and the connection weights and the bias parameters of the first layer are obtained. By the principle of denoising autoencoder, DAE model can learn the structure of the data itself and identify the characteristics of the input. The output of the first layer is used as the input of the second layer. Train the second layer to get connection weights and bias parameters of the second layer. And then layer by layer. The final features are reconstructed based on the learning of multiple layers. As shown in Fig. 3, given a label free training sample set ${\left\{x^m\right\}}_{m=1}^M$, the coding network transforms each training sample $x^m$ to the coding vector $h^m$ by the coding function $f_{\theta }$:

1

$h^m=f_{\theta }\left(x^m\right)=s_f\left(w{x}^m+b\right),$

where $s_f$ is the activation function of the coded network; and $\theta$ is the parameter set for the coded network, and $\theta =\left\{w,b\right\}$; where $w$ and $b$, respectively, are the connection weights and bias parameters of the coding network; DAE completes the training of the whole network by minimizing the reconstruction error between $x^m$ and ${\widehat{x}}^m$:

2

$L\left(x,\widehat{x}\right)=\frac{1}{M}{‖x-\widehat{x}‖}^2.$

Fig. 3The pre-training and fine-tuning of the DNN

Fine-tuning is supervised learning from top to bottom. With the labeled training data, the error transports from top to bottom to fine tune the deep neural network. This process is a supervised training process. Specific as shown in Fig. 3, this method uses the BP algorithm to fine tune the DNN parameters. The output of DNN is expressed as:

3

$y^m=f_{{\theta }_{N+1}}\left(h_N^m\right),$

where ${\theta }_{N+1}$ is the parameter of the output layer.

Assume that $x^m$ of the health condition type is $d^m$, DNN is fine-tuned by minimizing the ${\varphi }_{DNN}\left(\mathrm{\Theta }\right)$:

4

${\varphi }_{DNN}\left(\mathrm{\Theta }\right)=\frac{1}{M}{\sum }_{m}L\left(y^m,d^m\right),$

where $L$ is the function of minimizing the error between $y^m$ and $d^m$; where $\mathrm{\Theta }$ is the parameter set of DNN, and $\mathrm{\Theta }=\left\{{\theta }_1,{\theta }_2,...,{\theta }_{N+1}\right\}$.

2.2. PSO-SVM

2.2.1. Support vector machine

The support vector machine based on statistical learning theory proposed by Vapnik et al. [5] is a better method to realize the structural risk minimization principle, which provides a new way of thinking to solve the small sample size classification, nonlinear problem.

The basic idea of SVM is to increase dimension and linearization: define the optimal linear hyper plane, and reduce the algorithm of finding the optimal linear hyper plane to a convex programming problem. Then, based on the Mercer kernel expansion law, the sample space is mapped to a high dimensional and even infinite dimensional feature space by nonlinear mapping, so as to the linear learning machine can be used in the feature space to solve the problem of highly nonlinear classification and regression in the sample space.

Suppose a training set with $m$ samples $T=\left\{\left(x_i,y_i\right)\left|i=1,2,...,m\right.\right\}$, $x_i\in R$, $y_i\in \left\{-\mathrm{1,1}\right\}$ can be separated by a super plane without errors, and the distance between the nearest vector and the hyper plane is the largest, which is called the optimal hyper plane, as shown in Fig. 4.

Fig. 4Optimal hyper plane of the support vector machine

Define two standard hyper planes $H_1:w\cdot x+b=+1$ and $H_2:w\cdot x+b=-1$, where $H_1$ and $H_2$ are over all samples from the nearest classification and parallel to the plane of the classification of hyper plane, the distance between them is $2/‖w‖$.

The maximum interval $2/‖w‖$ is equal to the minimum value $‖w‖$. Taking into account the correct classification of all training samples, the following should be satisfied:

5

$\left\{\begin{array}{l}{y}_i=1,\left(w\cdot x_i\right)+b\ge 1,\\ y_i=-1,\left(w\cdot x_i\right)+b\le -1.\end{array}\right.$

Two can be combined into:

6

$y_i\left[\left(w\cdot x_i\right)+b\right]\ge 1.$

Therefore, the purpose of the support vector machine is to use the following formula to construct the classified hyper plane to classify all the samples correctly:

7

$\left\{\begin{array}{l}\mathrm{m}\mathrm{i}\mathrm{n}\frac{1}{2}{‖w‖}^2,\\ y_i\left[\left(w\cdot x_i\right)+b\right]\ge 1.\end{array}\right.$

Because the objective function and the constraint condition are convex, according to the optimization theory, this problem has a unique global minimum solution. Applying Lagrange multipliers and taking into account the KKT condition:

8

${\alpha }_i\cdot \left\{y_i\left[\left(x\cdot x_i\right)+b\right]-1\right\}=0.$

The optimal hyper plane decision function can be obtained:

9

$M\left(x\right)=sgn\left[\left(w^{*}\cdot x\right)+b^{*}\right]=sgn\left[{\sum }_{i=1}^n{{\alpha }_i}^{*}{y}_i\left(x_i\cdot x\right)+b^{*}\right].$

For linear separable case, by introducing slack variables ${\xi }_i\ge 0$, modifying the objective function and constraints, and then solving the method completely with similar application, we get:

10

$\left\{\begin{array}{l}min\frac{1}{2}{‖w‖}^2+C{\sum }_i{\xi }_i,\\ y_i\left[\left(w\cdot x_i\right)+b\right]\ge 1-{\xi }_i.\end{array}\right.$

If the training data can’t be separated, maximizing the class interval hyper plane requires eliminating those misclassified samples.

In the nonlinear case, the support vector machine uses the nonlinear mapping algorithm in the feature space, that is, the input vector $x$ is mapped to a high dimensional feature space $Z$ by the prior selection of a certain nonlinear mapping.$\phi :R^N\to Z$, $x\to \phi \left(x\right)$. Then in the high dimension space using linear support vector machine to classify. The objective function and constraint conditions are:

11

$\left\{\begin{array}{l}\max L\left(\alpha \right)={\sum }_{i=1}^m{\alpha }_i-\frac{1}{2}{\sum }_{i=j=1}^m{\alpha }_i{\alpha }_j{y}_i{y}_{j}k\left(x_i,x_j\right),\\ \mathrm{s}.\mathrm{t}.\begin{array}{ll}& {\sum }_{i=1}^m{\alpha }_i{y}_i=0,{\alpha }_i\ge 0\end{array}.\end{array}\right.$

Then the classification decision function is obtained:

12

$f\left(x\right)=sign\left[{\sum }_{i=1}^m{\alpha }_i{y}_{i}k\left(x_i,x_j\right)+b\right].$

For nonlinear classification problems, the core idea is to increase the linear spatial dimensions, namely the data sample low-dimensional space is transformed into a high-dimensional space via a mapping function, and thus the nonlinear classification problem is converted to a linear classification problem. Linear classification of samples in high dimensional Hilbert space is carried out to obtain the optimal classification hyper plane and decision function to return to low dimensional space.

2.2.2. Particle swarm optimization

SVM kernel function parameter g and parameter c have a major impact on its classification accuracy. So, in this paper, particle swarm optimization algorithm is used to optimize the parameters of SVM. In this way, we can effectively prevent the low classification accuracy due to the lack of experience in constructing kernel function parameters and penalty factors.

Particle swarm optimization algorithm was first proposed by J. Kennedy and R. C. Eberhart based on swarm intelligence theory in 1995 [10]. It’s mathematical expression is as follows: Assuming that in a $D$ dimensional optimization space, there are a group of $m$ particles, speed of the $i$th particles can be expressed as $v_i=\left(v_{i1},v_{i2},...v_{iD}\right)$, its position can be expressed as $x_i=\left(x_{i1},x_{i2},...,x_{iD}\right)\text{,}$ the optimal position of the current search for the $i$ particle is $p_i=\left(p_{i1},p_{i2},...,p_{iD}\right)$, and the optimal position of the whole population is $p_g=\left(p_{i1},p_{i2},...,p_{iD}\right)$, the particle swarm updating equation is as follows:

13

$v_{id}\left(t+1\right)=\omega v_{id}\left(t\right)+c_1{r}_1\left(p_{id}-x_{id}\left(t\right)\right)+c_2{r}_2\left(p_{gd}-x_{id}\left(t\right)\right),$

14

$x_{id}\left(t+1\right)=x_{id}\left(t\right)+v_{id}\left(t+1\right).$

If $v_{id}>v_{\mathrm{m}\mathrm{a}\mathrm{x}}$, $v_{id}=v_{\mathrm{m}\mathrm{a}\mathrm{x}}$.

If $v_{id}<-v_{\mathrm{m}\mathrm{a}\mathrm{x}}$, $v_{id}=-v_{\mathrm{m}\mathrm{a}\mathrm{x}}$.

$i=\mathrm{1,2},...,m$; $d=\mathrm{1,2},...,D$, where $t$ is the current iteration number, $c_1$ and $c_2$ are the acceleration constant, $r_1$ and $r_2$ are random numbers between [0, 1], and $\omega$ is the inertia weight.

The PSO algorithm is used to optimize the parameters of SVM, and the specific steps are shown in Fig. 5.

Fig. 5Flowchart of PSO algorithm to optimize the parameters of SVM

2.3. Fault diagnosis model based on deep learning and PSO-SVM

The spectral signal of the mechanical vibration signal works as the input layer, and multiple DAEs are stacked together to form DNN. The output of the first level DAE is the second level DAE input, the output of the second level DAE is the third level DAE input, and so on. In this method, the output of the last hidden layer or the last multiple hidden layers is used as the input of the PSO-SVM. PSO-SVM classifies the features which are extracted by the deep learning to complete the fault diagnosis.

A method for mechanical fault diagnosis based on deep learning and PSO-SVM comprises the following steps:

Step 1: The original vibration data of mechanical equipment is used as the input sample, and then the fast Fourier transform is applied to get a new input sample spectrum signal $X$.

Step 2: By means of the linear normalization method, the vibration spectrum signal $X_1$ is obtained by normalizing the vibration spectrum signal $X$.

Linear normalization method: ${\widehat{x}}_i=\frac{x_i-x_{\mathrm{m}\mathrm{i}\mathrm{n}}}{x_{\mathrm{m}\mathrm{a}\mathrm{x}}-x_{\mathrm{m}\mathrm{i}\mathrm{n}}}$.

Step 3: Put bearing vibration spectrum signal $X_1$ into the DNN, then use the DNN to extract features from the signal $X_1$.

Step 4: Put the features which are extracted from DNN into the SVM, and then use the particle swarm optimization algorithm to optimize the support vector machine parameters. After training, put the testing date into the SVM. Finally, we can get the diagnosis results of the mechanical equipment.

Fig. 6Flowchart of fault diagnosis method for mechanical equipment based on deep learning and support vector machine

DNN’s pre-training begins from the bottom, layer by layer to the top. This is the biggest difference with the traditional neural network, so as to avoid the neural network into the local optimal solution. DNN’s fine-tuning is through training the labeled data and use top-down error transmission error, so that the neural network parameters are closer to the global optimum, which can achieve better results.

3.1. Experimental configurations

As shown in Fig. 7 and Fig. 8, the test platform consists of a 2 hp motor (left), a torque transducer/encoder (center), a load motor (right). The test platform bearing comprises a drive end bearing and a fan end bearing. The acceleration sensors are respectively arranged at the driving end of the motor casing and the fan end, at the position of 12 o’clock. The vibration signal is collected by a 16-channel DAT recorder, and the drive end bearing fault data sampling frequency is 48000 samples per second. In this experiment, we select the drive end bearing as the research object. Data are collected when the motor load is 3 HP, and the drive end bearing has different fault conditions: normal, inner fault, outer race fault and rolling body fault. Three fault degrees are used for the fault of the inner ring, the outer ring and the rolling body: 0.007, 0.014, 0.021. Single point faults were introduced to the test bearings using electro-discharge machining with fault diameters of 7 mils, 14 mils, 21 mils (1 mil = 0.001 inches). And fault diameters of 7 mils, 14 mils, 21 mils corresponding to the light, medium and heavy fault level. So, this test has ten different working conditions. For each working condition, we divide the collected data into 120 samples and each sample has 4000 sample points. For each working condition, we randomly select 60 groups of samples as the training samples, and use the remaining 60 groups of samples as test samples. Table 1 summarizes the different fault degrees as follows.

Fig. 7Rolling bearing test platform of West University

Fig. 8Sketch of the rolling bearing test platform

3.2. Pre- training and fine-tuning of DNN

The vibration signals are divided into training samples and testing samples randomly. Then the selected sample is analyzed by fast Fourier algorithm. The time domain sampling number of each sample is 4000. While the spectrum of Fourier transform is symmetrical, we take the first 2000 sampling points of the spectrum as the spectral signal $X$. Then linear normalization method is used to get the new spectrum signal $X_1$. Normalization is not only able to improve the deep learning classification accuracy, but also can reduce the classification computation time. The number of the input layer of the DNN is determined to be 2000. In this paper, the network structure of DNN is set to 2000-500-300-200-10, and the output layer is determined by the classification tag. In addition, in order to enhance the robustness of fault diagnosis, noise encoding network will contain certain statistical properties to sample data.

3.3. Parameter optimization process of particle swarm optimization algorithm

The particle swarm optimization algorithm is set to the maximum number of evolution for 100. The population particle number is 20. The learning factor $c_1$ is 1.9 and the learning factor $c_2$ is 1.7. The parameter $k=$ 0.4 and the velocity in the velocity update formula in front of the elastic coefficient is $wV=$ 1. The SVM Cross Validation parameter $v=$ 3, the range of $c$ is [0, 1, 200] and the range of parameters for g is [0.01, 1000]. An average relative error of support vector machine is used as fitness function. After optimizing through the particle swarm optimization algorithm, the parameters of $c$ and $g$ are obtained, and the prediction model of support vector machine is determined.

3.4. Test results for PSO-SVM

The 600 rolling bearings test samples are selected randomly as training samples for SVM and the rest samples are used as testing samples for SVM. Using the proposed method for fault diagnosis, when the second hidden layer is used as the support vector machine input parameters, its recognition accuracy rate is 98.5 %; When the third hidden layer is used as the support vector machine input parameters, its recognition accuracy rate is 98.333 %; When the second hidden layer and the third hidden layer are used as the support vector machine input parameters, its recognition accuracy rate is 98.667 %, as shown in Fig. 9. In addition, we test the method using reduced training and testing samples (60 each). When the second hidden layer is used as the support vector machine input parameters, its recognition accuracy rate is 90 %; When the third hidden layer is used as the support vector machine input parameters, its recognition accuracy rate is 83.33 %; When the second hidden layer and the third hidden layer are used as the support vector machine input parameters, its recognition accuracy rate is 95 %, as shown in Fig. 11. The above experiments show that, under the condition of small samples, the features of the two hidden layer can be used as the input of support vector machine to significantly improve the accuracy of fault diagnosis. Table 2 is accuracies of different input parameters and their corresponding SVM classification under big sample conditions. Table 3 is accuracies of different input parameters and their corresponding SVM classification under small sample conditions.

Fig. 9Diagnosis results of rolling bearing large samples: a) features extracted from the second hidden layer as the input, b) features extracted from the third hidden layer as the input, c) features extracted from the second and the third hidden layers as the input

a)

b)

c)

We also compare the proposed method with the traditional method. The traditional method combines artificial feature extraction with PSO-SVM for diagnosis. Firstly, commonly used frequency-domain diagnostic features are extracted from the vibration signals obtained from the rolling bearing test platform. Here are the frequency-domain extracted features by artificial means, absolute mean amplitude, RMS, peak, peak to peak, shape factor, crest factor, impulse indicator, margin indicator and kurtosis indicator and so on [20-22]. These feature parameters represent the fault types of vibration signals. As shown in Table 1, the traditional method and the proposed method both use the 600 training samples, which are randomly selected, for SVM to train, and both use the remaining 600 test samples for SVM to test. SVM uses the radial basis kernel function, the kernel function parameters and penalty factors are obtained by the particle swarm algorithm. The difference is that the mentioned method uses DNN to extract features, and the traditional method uses artificial methods to extract features. As shown in Fig. 10, compared with the method based on the combination of the artificially extracted features and the PSO-SVM, the proposed method has higher accuracy of fault diagnosis.

Fig. 10Diagnosis results of rolling bearing small samples: a) features extracted from the second hidden layer as the input, b) features extracted from the third hidden layer as the input, c) features extracted from the second and the third hidden layers as the input

a)

b)

c)

One of basic reasons for high accuracy of the proposed method is that the method can be used to extract fault features from vibration spectrum adaptively. In order to verify the feature extraction capabilities of the proposed method, principal component analysis is used to identify the first three principal components of extracted features from both the proposed method and the traditional method. “First PC” means first principal components; “Second PC” means second principal components; “Third PC” means third principal components. Fig. 12(a) is the scatter plots of principal components for the artificially extracted features. We can see each condition scatter intertwined, showing there is no regular accumulation mode and indicating that the ability of artificially extracting fault feature is weak. Fig. 12(b) is the Scatter plots of principal components for the learned features. Compared with Fig. 12(a), we can see that the scattered points of most conditions are collected together, and the points are effectively separated. These results indicate that deep learning method can extract fault features from vibration spectrum adaptively. Compared with artificially extracted fault features, it not only saves a lot of time and manpower, but also improves the accuracy of diagnosis.

Fig. 11Diagnosis accuracy rate of rolling bearing: a) use the method of combining artificial extracted feature and PSO-SVM, b) use the method of combining deep learning and PSO-SVM

a)

b)

Fig. 12Scatter plots of principal components: a) use artificial extracted features, b) use learned features

a)

b)