On-line prediction remaining useful life for ball bearings via grey NARX

3.1.1. Datasets description

Two experimental data sets of PRONOSTIA and industrial dataset are used to do experiments. These data are obtained from the test rig sensors in Fig. 4.

Fig. 4Ball bearing test rig: a) PRONOSTIA, b) high speed train traction motor bearing test rig

a)

b)

Firstly, PRONOSTIA dataset has been used in many remaining useful life prognostic studies [7, 9, 11, 15]. There are 17 run-to-failure datasets in order to study the prediction of bearing remaining useful life. Vibration and temperature signals have been saved in ASCII files respectively. Since vibration data are studied in this paper, only vibration data are described as follows. The vibration signal consists of a horizontal vibration signal and a vertical vibration signal. The sampling frequency is 25.6 kHz. 2560 samples are recorded every 10 seconds. Each record is stored in an ASCII file. For each ASCII file, the data obtained by the vibration sensor includes the horizontal direction of vibration data and the vertical direction of vibration data. In addition, the corresponding vibration time is also stored in the ASCII file.

Secondly, the industrial ball bearing test of high-speed train is conducted on NTN bearing test rig. The bearing type of high-speed train is 6311 deep groove ball bearing. Acceleration sensor was used to collect vibration signals of ball bearings in order to monitor bearing health condition. The minimum recording period of the test bed is 10 seconds. The cumulative operation time of two experimental bearings is approximately 3000 and 700 hours. All samples are stored in two database files.

3.1.2. Data preprocessing

PRONOSTIA vertical vibration data is extracted from each sample ASCII file. For vertical vibration data of each sample file, Eq. (1) is used to calculate the time series health index. And then the health index vector $\mathbf{X}$ is then obtained. Fig. 5 describes the whole life cycle health index of the bearing 1-1. As can be seen from Fig. 5, the health index after AGO is smoother and larger than the original data and moving average health index.

Fig. 6 describes the part health index of the bearing1-1 from 1250 to 1350 time unit. From Fig. 5 and 6, it can be seen that the health index processed by MA method is not good at eliminating random fluctuations compared with AGO method. Compared with MA method, AGO method has a large difference value in health index at the same time interval. Since AGO method is more sensitive to data change and has good timeliness, the health index data processed by AGO is used to predict the remaining life of ball bearings.

Fig. 5Health index of the bearing 1-1: a) health index, b) moving average (q= 10) health index, c) accumulated generating operation health index

a)

b)

c)

Fig. 6Part health index of the bearing 1-1: a) health index, b) moving average (q= 10) health index, c) accumulated generating operation health index

a)

b)

c)

3.1.3. Time window processing

Before using the model to predict the nonlinear time series, it is necessary to reconstitute the data format. Suppose the time series is $\left\{x_1,x_2,\cdots ,x_n\right\}$. After reconstructing the data with window size $m$, we get the multi-dimensional feature matrix $\mathbf{X}$ and the prediction vector $\mathbf{Y}$. $\mathbf{X}$ and $\mathbf{Y}$ can be expressed in Eq. (7) and Eq. (8) respectively:

For the $j$ feature vector ${\mathbf{X}}_j$, the expression form is $(x_j,x_{j+1},\cdots ,x_{j+m-1})$:

The expression form of $y_j$ is $\left(x_{m+j}\right)$.

Fig. 7 describes the reconstruction process of the feature vector and the prediction vector when the window width is $m=$ 4. The training set for model training is $\{{\mathbf{X}}_i,y_i{\}}_{i=1}^{n-4}$.

Fig. 7Reconstruction of health index time series data

3.1.4. Performance metrics

The performance metrics are mean absolute error (MAE), root mean square error (RMSE) and Pearson’s correlation coefficient ($R$).

MAE is a measure of difference between two variables. It is given by the following equation:

where $n$ refers to the sample size, ${\widehat{y}}_t$ is the predictive value indexed with $t$, $y_t$ is the observed value indexed with $t$. MAE output is non-negative value. The best value is 0.

RMSE is a measure of the differences between model predictive values and the actually observed values. RMSE is given by:

where $n$ is the sample size, $\widehat{y}\left(t\right)$ is the predictive value indexed with $t$, $y\left(t\right)$ is the observed value indexed with t. A smaller RMSE value implies a smaller error variation between predicted values and observed values. RMSE reflects the degree which the predicted values deviate from the observed values.

$R$ is a measure of the linear correlation between two variables. The equation for $R$ is:

where $n$ refers to the sample size, $x_t$ and $y_t$ are the single sample indexed with $t$, $\stackrel{-}{x}$ is the mean of data set $\left\{x_1,x_2\cdots ,x_n\right\}$, $\stackrel{-}{y}$ is the mean of data set $\left\{y_1,\cdots ,y_n\right\}$. The $R$ value range is between –1 and 1, where –1 is total negative linear correlation, 0 is no linear correlation, and 1 is total positive linear correlation.

3.1.5. Compared approaches

Four methods are used in this paper, which are BP neural network, GRNN, Elman and NARX. BP and GRNN are static neural network. Elman and NARX neural network are dynamic neural networks. Static neural networks are characterized by no feedback, no memory, and the output depends only on the current input. The output of dynamic neural networks with feedback depends not only on current and previous inputs, but also on previous outputs.

BP was put forward by the scientific research group headed by Rumelhart and McCelland in 1986 [20]. BP neural network is a multilayer feed-forward network trained by the error back-propagation algorithm. It is one of the most widely used neural network models. The basic idea of BP is that the learning process consists of two processes, namely, the signals of feedforward and the back-propagation of errors. When forward propagation, the input samples are sent from the input layer, and processed by the hidden layers, and then transmitted to the output layer. If the actual output of the output layer is not consistent with the expected output, it will go back to the error back-propagation stage. In back-propagation, the output is retransmitted to the input layer by the hidden layer in some form, and the error is apportioned to all the units of each layer, thus the error signal of each layer is obtained. The error signal is the basis of correcting the weight of each unit. BP neural network is composed of input layer, hidden layer and output layer. Considering the regression prediction problem here, the transfer function used in BP network is tansig function.

GRNN was proposed by Specht in 1991 [21]. GRNN is a one-pass learning algorithm. Its network structure is highly parallel. GRNN can be used for nonlinear prediction. GRNN is a solution for online dynamical systems. It is a kind of radial basis neural network [22]. The network structure of GRNN is composed of four layers, namely, input layer, pattern layer, summation layer and output layer. Its network input is $X=[x_1,x_2,\cdots ,x_n{]}^T$, and the output is $Y=[y_1,y_2,\cdots ,y_k{]}^T$. GRNN model can be expressed as Eq. (12):

where $Y\left(x\right)$ is the prediction value of input $x$. $y_k$ is the activation weight for the pattern layer neuron at $k$. $K(\mathbf{x},{\mathbf{x}}_k)$ is the Gaussian kernel as formulated $K(\mathbf{x},{\mathbf{x}}_k)=e^{-(\mathbf{x}-{\mathbf{x}}_k{)}^T(\mathbf{x}-{\mathbf{x}}_k)/2{\sigma }^2}$.

In 1990, Elman proposed the Elman network, which is a kind of recurrent neural network [23]. Elman neural network is used in many fields, such as cognitive science, economics and others. Elman network structure consist of four layers: input layer, hidden layer, context layer and output layer. The context layer can provide a short memory. Elman networks are used for predicting time series. Elman neural network model can be expressed as Eq. (13):

where $\mathbf{y}\left(k\right)$ is output vector. $g(\cdot )$ is the transfer function of output neurons. $f(\cdot )$ is the transfer function of middle layer neurons. $\mathbf{x}\left(k\right)=f(\cdot )$ is hidden layer vector. $\mathbf{x}(k-1)$ is a feedback state vector. $\mathbf{u}$ is input vector. ${\omega }^3$ is the connection weight of the middle layer to the output layer. ${\omega }^2$ is the connection weight of the input layer to the middle layer. ${\omega }^1$ is the connection weight of the context layer to the middle layer.

NARX is a dynamic neural network. NARX can learn to predict time series given past values of the same time series, the feedback input, and the exogenous time series. It can be applied for predicting nonlinear sequence data. It can be used as a recurrent dynamic neural network to predict the next value of the input data.

3.2.1. Experimental process

Our goal is to provide multi-step ahead prediction of ball bearings health index. One step ahead is to predict the next data using historical data in a fixed window. For multi-step ahead predicting, the first step is predicted by applying one step ahead predicting. Subsequently, the predicted value is included as the latest component of input series to predict the next step using the one step ahead training method. This procedure is repeated for the continuous predicting. The following content briefly describes the basic steps in experimental research. Firstly, raw data set is from the vertical vibration streaming data of PRONOSTIA ball bearings and industrial ball bearing test data. Secondly, health index feature is extracted by Eq. (1). Thirdly, the data format required by the grey neural network framework is reconstructed by using the time window processing technique. Fourthly, the learning set 1 includes AGO health indexes from Bearing 1-1, Bearing 1-2, Bearing 1-3, Bearing 1-4, Bearing 2-1, Bearing 2-2, Bearing 2-3 and Bearing 2-4. The learning set 2 includes AGO health indexes from bearing 6311-1. The test set 1 consists of 8 AGO health indexes: Bearing 1-5, Bearing 1-6, Bearing 1-7, Bearing 2-5, Bearing 2-6 and Bearing 2-7. The test set 2 is AGO health index of bearing 6311-2. The learning set is divided into training set, validation set and test set, and the corresponding allocation ratio is 75 %, 15 %, 15 %. Fifthly, BP, GRNN, Elman and NARX learn on the learning set respectively in order to predict the health index of ball bearings remaining useful life. Finally, the training model is used to predict the test set. MAE, RMSE, $R$ and test running time are used to evaluate the performance of the four neural networks on the test set.

3.2.2. Performance comparison

The experimental results are divided into two parts. The first part is the life prediction result of ball bearing life cycle. The second part is the prediction result of the later period life of ball bearings. A comparison of the predicted and real values of the four algorithms in the two bearing test sets is shown in Fig. 8 and Fig. 9. The experimental performance results of two test sets are summarized in Table 1, Table 2, Table 3 and Table 4.

From Fig. 8, it can be seen that BP and GRNN methods are getting smaller and smaller with the increasing number of data obtained. This is not good for predicting the remaining useful life of ball bearings with relatively short life. With the increasing number of data acquired by sensors, Elman and NARX method prediction error is relatively stable. This is beneficial for predicting the residual service life of the ball bearings. From the results in Fig. 9, it can be seen that the prediction error of four methods on test set 2 is BP, GRNN, Elman and NARX in descending order.

Table 1 shows the performance of four different models. Results are expressed as mean ± SD. After calculation, the value of MAE, RMSE and $R$ indexes of BP are 49.71±15.84, 53.77±12.87 and 1.00±0.01 respectively. The test running time required for BP on the test set 1 is 0.01±0.00 seconds. The value of MAE, RMSE and $R$ indexes of GRNN is 40.51±10.55,44.89±8.43 and 0.99±0.01 respectively. The test running time required for GRNN on the test set is 0.27±0.17 seconds. The value of MAE, RMSE and $R$ indexes of Elman is 6.84±1.38, 7.54±1.46 and 1.00±0.00 respectively. The average time required for Elman on the test set is 0.01±0.00 seconds. And the value of MAE, RMSE and R indexes of NARX is 1.60±1.41 2.94±3.04 and 0.99±0.02 respectively. The test time required for NARX on the test set is 0.03±0.01 seconds. By comparison, the performance index of NARX were significantly higher than that of BP, Elman and GRNN. GRNN test runs for the longest time in four models. The other three models have a similar test run time. As it can be observed, compared with other methods, NARX method is more suitable for continuous prediction than the other methods using PRONOSTIA data.

Fig. 8Comparison: a) bearing 1-5, b) bearing1-6, c) bearing 1-7, d) bearing 2-5, e) bearing 2-6, f) bearing 2-7

a)

b)

c)

d)

e)

f)

Fig. 9Comparison of 4 algorithms on bearing 6311 life cycles

a)

b)

c)

d)

e)

f)

Considering the late prediction of the remaining useful life of ball bearings is more important, the last one hundred health indexes are selected for comparison. It can be calculated from Table 2 that the values of MAE, RMSE and $R$ of BP are 59.14±36.36,59.29±36.28 and 1.00±0.01 respectively. The test time required for BP on the test set 1 is 0.01±0.00seconds. The values of MAE, RMSE and $R$ of GRNN are 22.92±15.58, 23.43±16.27 and 1.00±0.01 respectively. The test time required for GRNN on the test set 1 is 0.03±0.02 seconds. The values of MAE, RMSE and R of Elman are 9.78±3.27, 9.99±3.18 and 1.00±0.01 respectively. The test time required for Elman on the test set 1 is 0.01±0.00 seconds. And the values of MAE, RMSE and $R$ of NARX are 2.04±2.16,2.85±3.53 and 0.98±0.03 respectively. The test time required for NARX on the test set 1 is 0.01±0.00 seconds. After analysis, BP is still the worst of the four hybrid models. However, the average performance of the last part of the GRNN model is lower than that of the total data. The average value of MAE and RMSE index of Elman method is lower than that of GRNN, but higher than that of NARX. The average values of MAE, RMSE and $R$ of NARX are the lowest among the four methods. The average performance of the last part of the NARX method is quite similar to the average performance of the whole life cycle data. According to the calculation results of Tables 1 and 2, grey NARX predict model has a higher prediction accuracy for predicting the remaining useful life of PRONOSTIA ball bearings.

From Table 3, it can be seen that BP has the largest MAE and RMSE on the test set 2, while the MAE and RMSE of NARX are the smallest. The four algorithms are almost completely positively correlated. The test run time of GRNN is the most, while the test run time of the other models is approximately equal. As can be seen from Table 4, the results of prediction performance of bearing 6311 last 100 health indexes and prediction performance of bearing 6311 all health indexes were similar. Because of the test rig conditions, time and financial constraints, only two bearings were used for the experiment. If the number of bearings increases, the prediction error of industrial bearings will be further reduced in real scenarios. As can be seen from table 1-4, grey NARX is the most suitable model for continuous prediction the remaining useful life of ball bearings than other three models.

After calculating the health index of the bearing during the entire service process, the relationship between the health index of the bearing vibration data and the running mileage in the bearing life cycle can be obtained, as shown in Fig. 10(a). After getting the relationship between the mileage and the health index, the remainder life of rolling bearings can be obtained by using the design life subtracted from the mileage corresponding to the health index, as shown in Fig. 10(b).

Fig. 10Relationship between mileage, health index and remaining useful life (RUL)

a)

b)