Prediction of residual life of rotating components based on adaptive dynamic weighting and gated double attention unit

2.1. Adaptive kernel principal component analysis (AKPCA)

Due to the presence of nonlinear signals that arise during the operation of complex systems, the accuracy of RUL predictions using deep learning algorithms tends to be limited, resulting in significant challenges for achieving precise and reliable predictions. Therefore, reducing data dimensions and constructing a monotonic trend HI is an urgent problem to be solved [26].

To enhance the efficiency of neural networks and streamline computational processes, the 21 time-domain and frequency-domain features were analyzed intricately and integrated into the RUL prediction model, as detailed in Table 1. In addition, to display the situation of extracting 21-dimensional features intuitively, the first six indicators in Table 1 are visually analyzed. As shown in Fig. 1, some features may not well represent the life degradation trend. Therefore, to improve the model performance and prediction accuracy, further extracting 21-dimensional features and fusing them are needed.

Fig. 1Feature visualization analysis: a)-f) correspond to the characteristics of indicators in Table 1

a) Peak to peak

b) Peak

c) Mean

d) Variance

e) Root mean square

f) Absolute mean amplitude

By consolidating these multidimensional features into a new composite feature denoted as HI, the aim is to streamline computational complexity and enhance the overall predictive capability of the neural networks. In addition to neural networks, isometric mapping algorithms (ISOMAP) and kernel principal component analysis (KPCA) in manifold learning are commonly used methods for handling nonlinear data [27]. KPCA, as an extension of PCA, extracts principal elements from highly nonlinear features. By introducing a nonlinear kernel function$\mathrm{\Phi }$, each sample $x_i$ of input data $Q^N$ can be mapped to a high-dimensional feature $F$, and the covariance matrix in feature space $F$ can be used to obtain the payload matrix. The payload Matrix refers to a set of feature vectors obtained by projecting the data in the original feature space into the kernel space. These eigenvectors describe the principal component direction and importance of data in the kernel space, which is equivalent to the representation of data after kernel transformation in the feature space. Finally, the covariance matrix in space $F$ can be calculated as follows:

Among them, $n$ is the quantity of input data. The loading matrix is the eigenvector of a covariance matrix $C$, represented as:

Among them, $\lambda$ is the eigenvalue, and $V$ is the eigenvector $C$ of the covariance matrix. Given a set of vectors $\alpha ={\left({\alpha }_1,{\alpha }_2,\cdots ,{\alpha }_n\right)}^T$, the description of the feature vector $V$ is as follows:

By selecting the kernel function $K\left(x_i,x_j\right)$, the kernel matrix $K$ can be represented by $K\left(\mathrm{\Phi }\left(x_i\right),\mathrm{\Phi }\left(x_j\right)\right)$. By combining Eqs. (1) and (3) above, the following could be concluded:

Then, Eigenvalue $m\lambda$ and eigenvector $K$ can then be computed, and the nonlinear mapping space $F$ of $V^k\left(k=\mathrm{1,2},\cdots ,n\right)$ for sample $x$ in the final eigenvector is defined as:

Therefore, KPCA can determine whether the signal is normal with $Hotelling{T}^2$statistics. $Hotelling{T}^2$ can be calculated by the following formula:

Among them, $t_i\left(1\times h\right)$ represents the principal component, $P\left(n\times h\right)$ represents the load matrix, $\mathrm{\Delta }\left(h\times h\right)$ represents the diagonal matrix formed, and $X_i\left(1\times n\right)$ represents the data in the $i$ row of the $1\times n$-sample space. Hence, this innovative approach not only effectively retains the distinctive characteristics of the original data but also seamlessly uncovers intricate nonlinear connections inherent within these features.

However, the selection of kernel functions in the KPCA algorithm needs to be combined with specific data. To address this issue, we have made improvements. A new evaluation index was constructed by combining trends with monotonicity indicators. By integrating this evaluation indicator with KPCA, the kernel function can be selected based on the characteristics of the data adaptively. Among them, monotonicity can evaluate the trend property of a feature increasing or decreasing on the timeline. For feature sequence $\lambda =\left[{\lambda }_1,{\lambda }_2,\cdots ,{\lambda }_n\right]$ and time series $t=\left[t_1,t_2,\cdots ,t_n\right]$, ${\lambda }_t$ is the corresponding feature value at time $t$, and $N$ is the maximum number of samples. The $Trend$ calculation is as follows:

Among them, $\overline{\lambda }$ is the average of ${\left\{{\lambda }_i\right\}}_{i=1:N}$, and $\overline{T}$ is the average of ${\left\{t_i\right\}}_{i=1:N}$. Furthermore, the calculation of monotonicity is as follows:

Among them, $d\lambda$ refers to the differentiation of the feature sequence. Finally, a comprehensive evaluation index can be obtained, with a range of [0, 1]. The closer it is to 1, the better the construction of HI. The preset threshold is set within the comprehensive indicator range, where 0.8 is used to indicate that the feature sequence has good monotonicity. The calculation is as follows:

Based on common kernel functions: Gauss, Poly, Laplace, Linear, RBF, and Sigmoid. The test results of AKPCA are shown in Table 2, where 1 indicates a monotonic trend and 0 indicates no monotonic trend.

Fig. 2Comparative analysis of HI based on ISOMAP, KPCA Poly, and KPCA linear

a) ISOMAP

b) KPCA-Poly

c) KPCA-Linear

Fig. 3HI images of other kernel functions

a) KPCA-Laplace

b) KPCA-RBF

c) KPCA-Gauss

d) KPCA-Sigmoid

In addition, the ISOMAP algorithm and the KPCA algorithm based on Poly and Linear kernel functions were compared in Fig. 2. The results show that the trend of the ISOMAP algorithm is similar to that based on the KPCA algorithm. However, due to the high computational complexity of the ISOMAP algorithm, especially when dealing with large-scale data, it incurs significant computational time and space overhead. In addition, in the situation of data with noise and outliers, the algorithm can be affected, resulting in inaccurate results. In addition, compared to the KPCA algorithm, the ISOMAP algorithm is also more difficult to implement. Therefore, in the subsequent experiments, the KPCA algorithm based on the Linear kernel function was chosen considering the setting of prediction thresholds.

To verify the accuracy of AKPCA, HI images of other kernel functions as shown in Fig. 3 were displayed. The HI images based on Laplace, RBF, Gauss, and Sigmoid kernel functions do not exhibit a monotonic trend, which further proves the effectiveness of AKPCA.

2.2. Exponentially weighted moving average

In Fig. 2, it can be observed that the data shows a good degradation trend. Meanwhile, there is also a large degree of discreteness, which is unfavorable for subsequent RUL prediction. To solve this problem, the Exponentially weighted moving average was adopted to optimize the constructed HI. This algorithm considers the weight of historical data and assigns different weights to different historical data through exponential weighting. This algorithm not only reduces data noise and fluctuations effectively but also produces relatively smooth results. As shown in the following formula, by introducing a positive hyperparameter $\alpha \in \left(\mathrm{0,1}\right)$, excessive smoothing is avoided and important health information is cleared:

Fig. 4Comparison before and after HI optimization

a) Original HI of pitting fault

b) HI optimized for pitting faults

c) The original HI of tooth breakage fault

d) Optimized HI for tooth breakage fault

Among them, $x_t$ and $x_t^{\mathrm{\text{'}}}$ refer to the observed and estimated values at the time $t$. In this study, a specific value of 0.4 was selected (with lower values indicating greater smoothness). The results depicted in Fig. 4 demonstrate the favorable outcomes achieved through the implementation of the exponential weighted translation algorithm for optimization. Notably, HI showcases a consistent monotonic progression and enhanced smoothness, contributing significantly to the enhancement of predictive accuracy.

In addition, by comparing images with different smoothing factors (Fig. 5), it can be concluded that the smaller the smoothing factor, the higher the smoothness. Therefore, increasing the smoothing factor can avoid excessive smoothing and ensure that important details are not cleared.

Fig. 5Comparative analysis of different smoothing factors

a) The smoothing factor is 0.1

b) The smoothing factor is 0.4

c) Without treatment

2.3. Adaptive dynamic weighted gated dual attention unit

GRU successfully solves the problem of the disappearance of RNN gradient by introducing storage cells and related gate structures and can store information for a longer time [28, 29]. GRU is further improved based on LSTM, with a relatively simpler structure that only includes two gates: reset gate and update gate. Among them, the update gate integrates two gates of LSTM: input gate and forget gate, making GRU superior to LSTM in computation time and with less computational complexity. The reset gate can directly handle the previous hidden state. Some scholars have confirmed that through these two gating units, the GRU can control the output of information and determine which information can be output, thereby effectively preserving information from longer sequences and avoiding them from being cleared. The GRU architecture is shown in Fig. 6.

Fig. 6GRU structure

However, a single GRU is limited in its ability to make long-term predictions. Although GRU is designed to solve the problem of long-term dependence, due to the problem of gradient disappearance or gradient explosion in some cases, it is difficult to retain and transmit long-term memory, which affects its performance in long-term prediction tasks. Specifically, due to the problem of gradient propagation, GRU may suffer from long-term memory loss when processing long sequence data, which makes it difficult for the network to capture long-term dependencies in the sequence. Fortunately, the introduction of GDAU [30] has effectively addressed this issue. This innovative gate structure is designed to prioritize resetting and updating gate output information and includes an additional attention gate that focuses on input data as well as information from the previous hidden state. By incorporating two attention gates, the GRU model's capacity to address long-term dependency challenges is enhanced significantly. This not only reduces the required computational resources but also enables accurate predictions to be made even with limited sample data.

However, this network ignores the importance of adaptive dynamic weighting, so a self-adaptive dynamic weighting GDAU method is proposed based on GDAU architecture. We introduce a truncated normal distribution represented as $\psi \left(\stackrel{-}{\mu },\stackrel{-}{\sigma },a,b;x\right)$ in GDAU. Among them, $\stackrel{-}{\mu }$, $\stackrel{-}{\sigma }$ represents mean and variance. $a$, $b$ represents the truncation interval. The probability density function is:

Its cumulative distribution function is:

Among them, $\varphi \left(\cdot \right)$, $\mathrm{\Phi }\left(\cdot \right)$ represent the probability density and distribution function of the standard normal distribution respectively. The inverse distribution of the cumulative distribution function: Let $0\le q\le 1$ find $a\le x\le b$ so that:

From this, it can be inferred that:

It $\alpha =\frac{a-\stackrel{-}{\mu }}{\stackrel{-}{\sigma }}$, $\beta =\frac{b-\stackrel{-}{\mu }}{\stackrel{-}{\sigma }}$ is set, the mean is:

The variance is:

Therefore, by generating truncated normal distribution random numbers, random variables with specific means and standard deviations can be obtained. These random variables can be used as weights in the network to adjust the importance or influence of different inputs. Moreover, the dynamic adjustment of these weights allows the network to adapt flexibly to varying scenarios, taking into account the specific characteristics and distribution of the input data. This allows the model to allocate higher weights to critical features and time steps automatically, ultimately enhancing the accuracy and performance of Remaining Useful Life (RUL) predictions.

4.1. Data 1 description

The dataset sourced from Chongqing University was utilized first. As depicted in Fig. 8, The testing machine is equipped with essential components such as a hydraulic cooling, torque control experimental operation system, and gear transmission platform. This study is divided into Remaining Useful Life (RUL) prediction under varied operating conditions, encompassing two distinct types of faults: tooth breakage and pitting corrosion. Considering that most of the data is in a stationary stage, we only used the last 600 samples to predict the RUL of the gears. Due to the good monotonic trend of the constructed HI, the last feature point of HI is selected as the fault threshold to terminate the prediction process.

Fig. 8Experimental platform

4.2. Parameter determination

The number of input units in the network plays an important role. In addition, learning rate is another key hyperparameter that can affect the training effectiveness and convergence speed of the model. Therefore, using grid search technology, these two hyperparameters are optimized by setting different input unit numbers and learning rates. Figs. 9 and 10 show the prediction error graphs, and the optimal parameters of the network were obtained based on them. Based on the prediction time in Table 3, it was determined that the number of neurons in the input layer, hidden layer, and output layer is set as 160, 22, and 1, respectively, along with employing an optimal learning rate of 0.05.

Fig. 9The impact of the number of input cells on RUL

Fig. 10The impact of learning rate on RUL

4.3. Experimental results and analysis

As depicted in Fig. 11, the prediction performance of the proposed RUL prediction method across varying RUL values is illustrated. It is evident that at RUL values of 25, 45, 65, and 85 minutes, the predicted RUL closely aligns with the actual RUL, displaying a consistent trend in the predicted values with the actual RUL values. The graph highlights that the proximity of the predicted point to the failure point correlates with the accuracy of gear RUL prediction, showing a higher accuracy. Furthermore, an increase in the number of known feature points leads to a significant enhancement of the network's predictive capability. When the number of feature points reaches 575, the accuracy of the predictions reaches an impressive 99 %. It is worth noting that 10 experiments were conducted on all subsequent methods to avoid unexpected errors, and their average values were recorded.

Fig. 11Comparison of different RUL prediction effects

a) The actual RUL is 25

b) The actual RUL is 45

c) The actual RUL is 65

d) The actual RUL is 85

This study employs a scoring function (Score) and mean absolute error (MAE) as evaluation metrics for assessing the performance of the predictions. The Score metric imposes heavier penalties on late predictions, thereby providing a more comprehensive evaluation of the RUL prediction accuracy. The calculation of the score is specifically determined based on a detailed algorithm that considers various factors involved in the prediction process. The specific calculation of the score is as follows:

where, $ActRu{l}_{}$ and $GRul$ refer to actual $Rul$ and predicted $Rul$. $N$ indicates the number of experiments. The calculation of MAE is:

To verify the superiority of the proposed method, a comprehensive comparison was conducted with both traditional and advanced diagnostic models. By subjecting the proposed method to rigorous evaluation alongside established techniques, we aimed to highlight its effectiveness and innovative capabilities in the realm of predictive maintenance. Among them, the traditional methods include (1) DNN, (2) LSTM, (3) GRU; Advanced methods include (4) GDAU [30], (5) ON-LSTM [17], (6) MMA-LSTM [18], and (7) DTGRU [19]. From Figs. 12 and 13, it is evident that our method showcasing the highest scores and minimal errors consistently outperformed both traditional and advanced diagnostic models. This strongly suggests that our method offers superior accuracy in estimating the RUL. Furthermore, the comparison of life prediction results across various models clearly illustrates the significant enhancement in performance achieved by incorporating adaptive dynamic weighting modules into the GDAU model. The inclusion of these modules not only brings the life prediction results closer to actual values but also enhances the overall precision of life prediction, thereby solidifying the effectiveness and reliability of our approach.

Fig. 12Comparison of scores for various methods

Fig. 13MAE comparison of various methods

To further prove the superiority of the model, we use the F1 score to evaluate the model. According to the results shown in Table 4, our method has achieved the best F1 score, which shows its excellent performance in model performance. Were, true positive (TP), false negative (FN), false positive (FP), true negative (TN). Therefore, the formula for calculating the F1- score is:

To verify the performance of the proposed method under continuous prediction, we adjusted the number of input units (changed to 100) and the number of samples for continuous prediction (increased to 350). Fig. 14 shows the results of continuous prediction, indicating that continuous prediction can be achieved by changing the number of input units. In the figure, the predicted degradation trend is consistent with the actual degradation trend, which verifies the effectiveness of the method.

Fig. 14Effect of continuous prediction

Moreover, assessing the convergence speed of neural networks is crucial for evaluating their overall performance. In this context, the constructed Health Indicator (HI) was utilized to verify and analyze the convergence speed of the network. Fig. 15 shows the prediction error curve. It can be observed that after approximately 150 epochs, the network reached a stable state with minimal subsequent changes, which effectively proves the predictive stability of the model. In addition, to assess the generalization capability of the model, the RUL prediction was conducted on gears with pitting faults. The results illustrated in Fig. 16 highlight the continued strong performance of the proposed method in accurately predicting RUL for various types of faults. This outcome further corroborates the effectiveness and robustness of the model in handling different fault scenarios, underscoring its reliability and versatility.

4.4. Data 2 description

To verify the generalization of the model, a multifunctional bearing fault prediction experimental platform (BPS) developed by Spectra Quest was selected. We have developed our experimental platform as shown in Fig. 17 which comprises a radial loading device, speed display, vibration acceleration sensor, and data acquisition device. This setup was utilized to collect vibration data from the rolling bearings over their entire life cycle. This extensive testing and data collection strategy was implemented to validate and demonstrate the efficacy of the proposed method in predicting RUL. The bearing type is ERK16, and the experimental parameters are set as follows: a radial load of 8500 N is applied, with the speed fluctuating within the range of [1450-1550] r/min. Vibration data is sampled at a frequency of 12800 Hz, with a sampling interval of 5 minutes and a duration of each sampling period set at 4 seconds. These specific experimental conditions were carefully selected to ensure accurate and comprehensive data collection for the RUL prediction analysis of the rolling bearings.

Fig. 15Prediction error curve

Fig. 16RUL of pitting failure

Fig. 17Test bench for predicting the lifespan of variable speed bearings

In this experiment, the neural network architecture was carefully designed with 50 neurons in the input layer, 17 neurons in the hidden layer, and 1 neuron in the output layer. A learning rate of 0.05 was determined to be optimal for the training process. The selection of these parameters aims to enhance the accuracy and reliability of the remaining life prediction analysis. We selected the last 300 samples for validation and determined the failure threshold based on the size of the last feature point. By utilizing the feature matrix composed of 285 experimental samples as the training dataset, predictions were generated for 15 different time points. The outcomes obtained via the suggested approach are illustrated in Table 5, where the AKPCA experimental results are detailed. Additionally, Fig. 18 showcases the training curve, prediction curve, and actual curve, providing a comprehensive visual representation of the analysis process and results. Based on the above curves, it is evident that the predicted curve closely follows the trend of the actual curve, suggesting that the proposed method forecasts the remaining lifespan of bearings under variable speed conditions effectively. Moreover, the average prediction error was found to be a mere 2.43 % through numerous experiments, showcasing the model’s robust generalization capability. The error curve depicting the prediction is illustrated in Fig. 19, revealing a gradual decrease and stabilization of prediction error with an increasing number of iterations. This observation serves to reinforce the efficacy of the proposed method, showcasing its consistency and reliability in predicting outcomes.

Fig. 18Failure thresholds, training, actual, and prediction curves for 300 experimental samples of real bearings

Fig. 19Prediction error curve

4.5. Discussion

Considering the environmental interference factors (including noise, electromagnetic interference, etc.) and the changes in speed and load, some features extracted may not well represent the life degradation trend in some cases. This emphasizes the importance of feature fusion because different feature sources can provide information from different aspects and angles. Through feature fusion, we can fuse this diverse information and get a more comprehensive and comprehensive feature representation to better reflect all aspects of data and the relationship between features. In addition, feature fusion also helps to reduce the sensitivity to noise or missing data, thus improving the stability and anti-interference ability of the model. Therefore, to improve the model performance and prediction accuracy, we adopt the strategy of extracting 21-dimensional features and fusing them. Such careful consideration and operation can effectively enhance the robustness of data features and provide more powerful support for further analysis and prediction. Moreover, judging from the results of the two case studies, we still achieved excellent prediction results even for complex working conditions and sudden changes. Besides, the invisible data can be reliably predicted.