A new fault diagnosis method using deep belief network and compressive sensing

2.1. Compressive sensing

In the framework of CS, a measurement matrix $\mathbf{\Phi }\in R^{M\times N}\left(M<N\right)$ has been designed for reducing the dimension of original signal $\mathbf{x}(\mathbf{x}\in R^N)$. Each row of the measurement matrix can be seen as a filter for obtaining linear projections. And parts of the signal information can be obtained by each filter. In summary, the compressive projections $\mathbf{y}(\mathbf{y}\in R^M)$ can be represented as:

Then, a low-dimensional signal $\mathbf{x}$ is used to present the high-dimensional signal $\mathbf{y}$. Generally, we can not directly recover the original signal $\mathbf{x}$ from measurement $\mathbf{y}$ when $M<N$. However, if $\mathbf{x}$ is sparse, this recovery can be achieved since more information is known.

In the CS theory, a sparse basis ${{\left\{{\mathbf{\psi }}_i\right\}}_i}_{=1,\dots ,N}$ is needed for decomposition. If we decompose original signal $\mathbf{x}$ on the sparse basis, then vector $\mathbf{x}$ can be represented as:

where ${\theta }_i$ can be seen as the sparse coefficients. Eq. (2) is equivalent to:

where $\mathbf{\Psi }=[{\psi }_1,{\psi }_2,\dots ,{\psi }_N]$ and $\mathbf{\theta }={\left[{\theta }_1,{\theta }_2,..,{\theta }_N\right]}^T$. Here, ${\mathbf{\psi }}_i$is defined as a $N\times 1$ column vector. Combining Eqs. (1) and (3), we can represent signal $\mathbf{y}$ as:

where $\mathbf{\Theta }$ is defined as sensing matrix. Solving a sparse vector $\mathbf{\theta }$ is a commonly discussed problem [2, 28] with respect to $\mathbf{\Theta }$. The framework of CS is shown in Fig. 1.

According to Candès and Tao, for higher successfully probability of recovery, the sensing matrix $\mathbf{\Theta }$ must follow the restricted isometry property (RIP), that is:

where $\mathbf{s}$ represents a random $k$-sparse vector, and ${\delta }_k\in$(0, 1). After sparse vector $\mathbf{\theta }$ being obtained, the original signal is easily achieved using Eq. (3).

Fig. 1The framework of compressive sensing

2.2. The basic units of DBN

DBN is consist of multiple restricted Boltzmann machines (RBM) and classifiers. RBM is the basic unit of DBN, which comprises two layers of neural network: visible layer and hidden layer. The visible layer and hidden layer are connected through weight $\mathbf{w}$ and biases vectors $\mathbf{c}$, $\mathbf{b}$. However, the units in same layer are not connected, respectively. The structure of RBM is shown as Fig. 2.

For given visible layer $\mathbf{v}$ and hidden layer $\mathbf{h}$, the energy function can be represented as:

where $v_i$, $h_j$ represent the binary states of units, $c_i$, $b_j$ represent the node biases, and $w_{i,j}$ represents the weight. The main purpose of DBN model is to find a stable state, which minimizes the energy error. Therefore, joint probability distribution between visible layer and hidden layer can be represented as follow:

where $Z$ is the energy summary of all visible and hidden layers. Eq. (7) can be seen as a proportional function for guaranteeing the normalized distribution. Moreover, the probability distributions of visible and hidden vectors can be calculated by:

Eq. (9) represents the learning process, that is to extract low-dimensional features from high-dimensional input data. And Eq. (10) represents a reverse learning process in which input data is reconstructed from feature vectors.

Fig. 2The structure of RBM

2.3. The DBN training

The DBN training consists of two steps: (1) For RBM units, a pre-training is performed between two layers using unlabeled data. (2) The fine-tuning is performed utilizing the backpropagation algorithm to adjust the parameters. The first step is an unsupervised learning process, while the second is a supervised process.

Fig. 3The pre-training process of DBN

The pre-training process is shown in Fig. 3, and the dark layer represents the training part for each step. In the pre-training process, a stochastic gradient descent on the negative log-likelihood probability is used for training RBM. The function is defined as follow:

where ${⟨:⟩}_{data}$ denotes the data expectation of distribution, and ${⟨:⟩}_{\mathrm{m}\mathrm{o}\mathrm{d}\mathrm{e}\mathrm{l}}$ denotes the model expectation of distribution. In the actual application, obtaining the precise value of ${⟨:⟩}_{model}$ is so difficult. Thus, it is substituted by $k$ iterations of Gibbs sampling as the final results. The parameters are updated as follow:

where ${\mathbf{\alpha }}^m$denotes the input sample. From Eq. (12), the data generated by a visible layer is passed to the hidden layer. And then the hidden layer is used to reconstruct that vector. During this process, all parameters $w$, $b$, $c$ of DBN will be updated until the number of iterations is satisfied.

After pre-training, the next step is fine-tuning. The ultimate purpose of this step is to further decrease the training error and improve the classification accuracy. Generally, a backpropagation neural network (BPNN) algorithm is performed to adjust the parameters. Assume that the label of input is ${\mathbf{l}}^m$, and the output of DBN is ${\mathbf{y}}^m$, with training error as defined below:

where $\gamma$ represents the error parameter. With the advancement of iterations, $\gamma$ will be updated as follow:

where $\eta$ denotes the learning ratio. The supervised training will continue until the error is less than a certain setting.

4.1. DBN fault diagnosis on compressed gearbox signals

In order to measure the validity of DBN-CS approach, a gearbox experimental platform is set up as in Fig. 5. The experimental table is constituted with motor, torque decoder and power tester. The experiments are conducted under following speeds, 400 rpm, 800 rpm and 1200 rpm. For each speed, four kinds of loads are applied using magnetic powder brake and they are 0 Nm, 0.4 Nm, 0.8 Nm, and 1.2 Nm.

Fig. 5Planetary gearbox test rig

Fig. 6Seeded wear failure

a) Sun gear

b) Planet gear

c) Ring gear

For methods testing, four conditions are applied to the system. One of them is the normal state to compare with three faulty states: sun gear fault, planet gear fault, and ring gear fault. The specific failures are shown in Fig. 6. The sampling frequency is set to 20 kHz and acquired vibration signal has a 12 s duration for each sample. The training and testing samples are listed in Table 2. It can be seen that the number of training samples for each state is 450, and that of testing samples is 150. We will calculate the success accuracies for both training samples and testing samples in the experiments.

In compressive sensing, the compression ratio (CR) is often analyzed for evaluating the compression. Assuming that the length of original signal is $N$ and the length of compressed signal is $M$, then we have:

We firstly take the 25 % compressed signal ($CR=$ 25 %) for example to validate the proposed method. The measurement matrix is designed using the Toeplitz matrix with Gaussian entries as the basis. In this way, the Toeplitz matrix generated by Gaussian entries is supposed to achieve higher reconstruction accuracies. Since $CR=$ 25 %, the dimension of measurement matrix should be set as $M\times N$ ($M=$0.75 $N$).

Fig. 7Amplitudes waveform of original and 25 % compressed signal for gearbox in different conditions: a) normal; b) normal and compressed; c) planet gear fault; d) planet gear fault and compressed; e) ring gear fault; f) ring gear fault and compressed; g) sun gear fault; h) sun gear fault and compressed

a)

b)

c)

d)

e)

f)

g)

h)

In the data preparation phase, we split the entire signal into small samples because the raw signal is too long. It is impossible to perform such high-dimensional matrix operation. Then, we put the compressed signal samples together to reunion the entire compressed signal. As Fig. 7 displays, the peak value of compressed signal is lower and the signal fluctuation is smaller compared to original signal.

The experimental environment is set as follow: the computer processor is Intel(R) Core(TM) i5-8250U 1.6 GHz, and the simulation tool is Matlab R2017a. Moreover, there are some important parameters of DBN requiring careful choices, for example the sample length. Because the number of sampling points in one cycle is $T=Fs/v=20000/\left(400/60\right)=$ 3000, we set the sample length as 4000. And the compressed signal length is 4000×75 % = 3000. The fine-tuning process utilizes BP algorithm to distribute errors of two RBM. The momentum and the learning rate are two factors describing weight updating criteria between two RBM layers. Moreover, a Contrast Divergence (CD) algorithm is used for obtaining the expected value of the energy partial derivative function [30]. The related parameters of DBN are shown in Table 3.

The structure of DBN is another important factor that affects the diagnosis accuracy. For example, structure 3000-600-600-4 means the input layer includes 3000 nodes, and the output layer includes 4 nodes. Since the input and output nodes are certain, we abbreviate the structure as 600-600. Table 4 displays the experimental results of different DBN structures.

It can be seen from the Table 4 that the structure 600-600 achieves the best accuracy 95 %, and average training time is 228.47 s. We can also find that more layers do not mean higher accuracies and two layers is the most efficient.

Fig. 8The confusion matrix of DBN classification

Fig. 9PCA analysis of the DBN learned features

The confusion matrix of best result for DBN diagnosis is shown in Fig. 8. It is observed that the planet gear fault (P) has the best accuracy of 100 %, ring gear fault (R) has the accuracy of 94 %, and normal state (N), sun gear fault (S) have the worst accuracy of 88 %. To gain more insight about the capability of DBN, a principal component analysis (PCA) method is used to show the separability of extracted features. The features obtained from the 1st layer of DBN have been decreased to 3 kinds: PCA1, PCA2, PCA3 as Fig. 9 shows.

More experiments have been conducted for different compressed signals. Table 5-7 display the DBN fault diagnosis results of 10 %, 20 % and 40 % compressed signals. In the testing experiments, DBN achieves 95.17 % identification accuracy for 10 % compressed signal, 92.9 % identification accuracy for 20 % compressed signal, and 64.5 % identification accuracy for 40 % compressed signal. The average training time for appeal results are 265.85 s, 257.14 s, and 196.47 s. With the compression ratio increasing, the accuracies of training samples and testing samples fall down because the useful features have been destroyed gradually.

4.2. Comparison with other classification methods

In order to verify the validation of proposed method, the comparative fault diagnosis methods are tested, such as BP artificial neural network (BPANN), support vector machine (SVM), random forest (RF), and artificial neural network (ANN). The structure of BPANN and ANN is 3000-600-100-4, indicating that there are two layers between input and output. The penalty factor and the radius for SVM classifier is set as 0.216 and 0.3027, referring [27]. The compression ratio varies from 0.05 to 0.9. The algorithms of BPANN, SVM, RF, ANN and DBN trained the model with the same compressed data set and generated a classifier to perform the classification. For each data point, 100 experiments have been performed. And we calculate the average value to evaluate different methods as Fig. 10 shows.

It is inferred that the DBN model is better for compressed signal fault diagnosis, especially when CR is 0.1-0.3. We can also find that BPANN achieves 100 % accuracy when $CR=$ 0.05, which is better than DBN. This is because the main features have been reserved in the compressed signal when CR is low, and BPANN extracts them efficiently. However, as the compression ratio increases, the information of original signal loses gradually. At this time, DBN achieves better performance for its strong feature extraction capability.

Fig. 10Comparison of different classification methods

4.3. Comparison of compressed and reconstructed signal

Here, a comparison of DBN diagnosis on compressed signal and reconstructed signal has been proposed. In traditional monitoring, the compressed signal is firstly reconstructed and then used for diagnosis. However, this may cause great time consumption because of the complex reconstruction algorithms. Therefore, we try to replace this process with DBN diagnosis directly on compressed signal. The new framework is shown in Fig. 11.

In order to compare those two approaches, a Bayesian compressive sensing algorithm is used here for signal reconstruction. For CS theory, the reconstruction algorithms can be divided into three categories: greedy-based methods, convex optimization methods [28], and Bayesian compressive sensing methods [31]. The Bayesian method estimates the maximum posterior probability from a Bayesian perspective. The Bayesian methods are more accurate than the other two kinds, in spite of a bigger time cost.

Referring Section 2, the key to compressive sensing is obtaining a strong sparse representation of original signal $\mathbf{x}$. Extended by $K$-means clustering, K-SVD algorithm [32] adaptively produces the sparse dictionary by training signal blocks. Compared to fixed dictionaries, such as discrete cosine transform, Fourier transform, K-SVD algorithm avoids the dependence on prior knowledge.

Fig. 11The new diagnosis framework based on DBN method

Fig. 12 displays the DBN classification results on both compressed signal and reconstructed signal. The compression ratio (CR) varies from 0.1 to 0.6. With CR increasing, the accuracy of compressed signal falls down quickly, while the accuracy of reconstructed signal is relatively stable. Moreover, it can be seen that the accuracies of compressed signal are superior to those of reconstructed signal when CR varies from 0.1-0.35. In this stage, there is sufficient information preserved in compressed signal for DBN diagnosis. When CR is larger than 0.35, the features in compressed signal are destroyed seriously, which leads to a poor accuracy. At this time, the precisely reconstructed signal presents great potential for DBN diagnosis.

Similarly, Fig. 13 indicates the comparison of running time on both compressed signal and reconstructed signal. The running time of reconstructed signal includes three parts: K-SVD training, Lap-CBCS reconstruction, and the DBN diagnosis. It is found that the DBN running time of compressed signal and reconstructed signal is not much different. However, the total running time of reconstructed signal is far more than that of compressed signal.

Fig. 12The DBN classification accuracy of compressed signal and reconstructed signal

Fig. 13The running time of compressed signal and reconstructed signal

4.4. The effects of noise

In actual signal acquisition process, the noise generated by working machinery and sensor may have influences on data compression and fault diagnosis. To further study the influence of noise, the 10 % compressed, 25 % compressed, and 50 % compressed signals are chosen for comparison. Assuming that the noise follows Gaussian distribution, a random white noise is introduced after signal compression. The signal received is re-defined as:

where vector $\mathbf{w}$ represents the noise signal. For evaluating the noisy level, the signal to noise (SNR) is denoted as:

As shown in Fig. 14, four curves demonstrate that the diagnostic accuracies increase gradually with the increasing SNR. And the 10 %, 25 % compressed signal are not much different with the original signal as SNR varies from 16 to 28.

In conclusion, it has little influence on compressed signals when noise is relatively small (16 $<SNR<$ 28). This indicates a strong robustness of DBN diagnosis method on CS compressed signal. However, compared to the original signal, the compressed signals are kind of unstable with the SNR increasing.

Fig. 14The DBN classification accuracy of original and compressed signal for different SNR