Fault diagnosis of rolling bearing based on cross-domain divergence alignment and intra-domain distribution alienation

2.1. Subspace cross-domain divergence alignment subspace alignment

The subspace learning method usually assumes that the data in the source domain and the target domain will have similar distribution in the transformed subspace. In general, since the source domain and the target domain sample data acquisition environment are different, it is assumed that they are in different subspaces, and often similar matrices will lead to similar distributions [19]. Therefore, we need to align the domain data matrix in the appropriate subspace, assuming that there is a sample data ${\mathbf{D}}_s={\left\{\right(x_s^i,y_s^i\left)\right\}}_{i=1}^{n_s}$ with $n_s$ known labels as the source domain, and a sample data ${\mathbf{D}}_t={\left\{\right(x_t^j,?\left)\right\}}_{j=1}^{n_t}$ with $n_t$ unknown labels as the target domain, where, $x_s^{}\in {\mathbf{X}}_s\in {\mathbb{R}}^{n_s\times d}$, $x_t\in {\mathbf{X}}_t\in {\mathbb{R}}^{n_t\times d}$, $d$ is the dimension of signal. The purpose of subspace alignment is to find a transformation matrix $\mathbf{Z}\in {\mathbb{R}}^{d\times d}$ and transform the source domain data into a subspace that can be aligned with the target domain data. The following optimization objective function can be obtained:

where ${‖·‖}_F^2$ represents the Frobenius norm. Learning the transformation matrix $\mathbf{Z}$ makes the source domain data ${\mathbf{X}}_s$ adapt to the target domain data ${\mathbf{X}}_t$, so that the two domain data are as similar as possible. Solving matrix $\mathbf{Z}$ requires that the number of samples in the two domains is equal, that is, $n_s=n_t$. However, due to the differences in the environment in which the samples are located, the difficulty of sample acquisition is also inconsistent, so it is difficult to ensure that the two domain data are equal.

In order to solve this problem, we use the overall divergence matrix in the domain to replace the samples in the domain, which can not only ensure the equal dimension of the two domains, but also mine the class structure in the domain.

2.2. Overall divergence

Suppose that ${\mathbf{u}}_j$ denotes the center point of class $j$ in the domain, $\mathbf{u}$ denotes the center point of the whole domain, $n_j$ denotes the number of samples in class $j$, and ${\mathbf{X}}_j$ denotes the sample set of class $j$. The calculation equations of between-class scatter matrix ${\mathbf{S}}^b\in {\mathbb{R}}^{d\times d}$, within-class scatter matrix ${\mathbf{S}}^w\in {\mathbb{R}}^{d\times d}$ and overall scatter matrix ${\mathbf{S}}^o\in {\mathbb{R}}^{d\times d}$ can be obtained:

where $\theta$ is the regularization parameter and $\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}$ is the diagonal matrix of the matrix.

2.3. Cross-domain divergence alignment

In order to better align the source domain and the target domain in the subspace and perform cross-domain transfer of the intra-domain structure, this paper implements subspace divergence alignment by embedding the overall divergence matrix of the source domain and the target domain in the subspace to reduce the domain space offset. Since the target domain sample data is unlabeled, it is necessary to first use the source domain label data to train a KNN classifier to obtain the pseudo label of the target domain. Assuming that ${\mathbf{S}}_s^o$ and ${\mathbf{S}}_t^o$ are the overall divergence matrices of the source domain and the target domain, respectively, and the transformation matrix is $\mathbf{Z}$, the divergence alignment process can be expressed as:

Thus, it can be derived from Eq. (5):

Therefore, an optimization result $\mathbf{Z}={\mathbf{S}}_t^{\frac{1}{2}o}{\mathbf{S}}_s^{-\frac{1}{2}o}$ can be obtained. When the overall divergence matrix of the source domain cannot be inverted, its Moore-Penrose pseudo-inverse can be used to correct the linear transformation matrix $\mathbf{Z}$. Finally, we get the new source domain sample data ${{\mathbf{X}}_s^{New}=\mathbf{X}}_s\mathbf{Z}$ after divergence alignment.

2.4. Inter-domain weighted conditional distribution alignment

The above divergence cross-domain alignment is equivalent to the overall alignment of domain data and the transmission of intra-domain information. It does not minimize the distance between cross-domain samples of the same class. Therefore, it is necessary to adapt the conditional distribution between domains to minimize the distance between samples of the same class in the source domain and the target domain.

In feature-based transfer learning, Maximum Mean Discrepancy (MMD) is often used to measure the cross-domain feature distribution. The conditional distribution $D_f^{\left(c\right)}({\mathbf{D}}_s,{\mathbf{D}}_t)$ based on the maximum mean difference is usually expressed as:

Among them, $H_K$ is the feature space, $\varphi (·)$ represents the reproducing kernel Hilbert space (RKHS), $n_s^{\left(c\right)}$ is the number of samples belonging to class $c$ in the source domain, and $n_t^{\left(c\right)}$ is the number of samples belonging to class $c$ in the target domain.

However, in the actual convergence process of the classifier, the target domain uses pseudo labels obtained by the weak classifier. Even if the number ratio of the same class in the source domain and the target domain is the same in practice, in the process of iteration, due to the error between the source domain and the real label, the same class ratio of the two domains will be different [21]. In the process of conditional distribution adaptation, the weight of each class is added, and the weighted conditional maximum mean difference is used to replace the original conditional distribution difference. The expression is as follows:

where, $w_s^c$ and $w_t^c$ represent the weight coefficients of class $c$ samples in the source domain and the target domain, respectively, calculated by $w^c=\sqrt{P\left(y^c\right)}$, and $P\left(y^c\right)$ represents the prior probability of class $c$.

2.5. Distribution alienation between classes within the domain

In order to further mine the intra-domain structure information and retain the intra-domain label distribution difference, the label distribution distance between the source domain and the target domain is adjusted. Similarly, this paper uses the mean difference of the maximum mean difference in the Hilbert space of the reproducing kernel to represent the distance between the classes in the domain, and obtains the distribution distance between the source domain and the target domain:

where, $H_K$ is the feature space; $\varphi (·)$ represents the reproducing kernel Hilbert space (RKHS); $n_s^{\left(c\right)}$ and $n_s^{\left(\stackrel{-}{c}\right)}$ are the number of samples belonging to class $c$ and non-class $c$ in the source domain, respectively. $n_t^{\left(c\right)}$, $n_t^{\left(\stackrel{-}{c}\right)}$ similarly; $C$ is the total number of categories.

Since the inter-class distribution in the source domain uses real labels, the labels will not change during the iteration process, while the inter-class distribution in the target domain uses pseudo labels obtained by the weak classifier (KNN). Therefore, it is only necessary to introduce weighting coefficients into the inter-class distribution distance of the target domain to obtain:

where, $w_t^c$ and $w_t^{\stackrel{-}{c}}$ represent the weight coefficients of the target domain class $c$ and non-class $c$ respectively, and $w_t^c=\sqrt{P\left(y_t^{\left(c\right)}\right)}$, $w_t^{\stackrel{-}{c}}=\sqrt{P\left(y_t^{\left(\stackrel{-}{c}\right)}\right)}$, $P\left(y_t^{\left(c\right)}\right)$ and $P\left(y_t^{\left(\stackrel{-}{c}\right)}\right)$ represent the prior probabilities of the source domain class $c$ and non-class $c$ respectively.

2.6. An optimal classifier is constructed based on structural risk minimization

According to the representation theorem [22], the classifier $f$ can be expressed as:

Given a Hilbert space $\mathcal{H}$ and a kernel function $K(x_i,\mathbf{X})$, the structure risk function of the mapping $\varphi :\mathcal{X}\to \mathcal{H}$ from the original space to the Hilbert space is as follows:

where, $f\in {\mathcal{H}}_K$ is the classifier in kernel space, $y_i$ is the label of $x_i$, ${‖·‖}_K^2$ is the square norm, $\eta$ is the regularization parameter, and the classifier with low expected error is obtained by minimizing the structural risk function. The final classifier is expressed as:

where, $D_f^{\left(c\right)}({\mathbf{D}}_s,{\mathbf{D}}_t)$ is the inter-domain weighted conditional distribution term, ${\stackrel{-}{D}}_f^{\left(c\right)}\left({\mathbf{D}}_{\text{s}}\right)$ is the source domain inter-class distribution alienation term, ${\stackrel{-}{D}}_f^{\left(c\right)}\left({\mathbf{D}}_t\right)$ is the target domain weighted inter-class distribution alienation term, $\lambda ,\gamma ,\delta$ is the regularization parameter, and $R_f({\mathbf{D}}_s,{\mathbf{D}}_t)$ is the Laplace regularization term, which is used to further mine the geometric similarity in the data space and ensure that there is no over-fitting in the iterative process. It is expressed as:

The pairwise similarity matrix is expressed as:

where $sim\left(·,·\right)$ is a similarity function, and ${\mathcal{N}}_p\left(x_i\right)$ is the $p$ nearest neighbor set of point $x_i$. By calculating the Laplacian matrix $\mathbf{L}=\mathbf{I}-{\mathbf{D}}^{-\frac{1}{2}}{\mathbf{S}\mathbf{D}}^{-\frac{1}{2}}$, where $\mathbf{D}$ is the diagonal matrix of the similar matrix $\mathbf{S}\in {\mathbb{R}}^{{(n}_s+n_t)\times {(n}_s+n_t)}$, the regularization is finally obtained.

Then, according to the representation theorem and kernel technique, the classifier function is transformed into:

where, $\mathbf{Y}=[y_1,\dots ,y_{n_s+n_t}]$$\in {\mathbb{R}}^{1\times {(n}_s+n_t)}$is the label matrix predicted according to the label information of the source domain. $tr(·)$ represents the trace of the matrix. $\mathbf{E}\in {\mathbb{R}}^{{(n}_s+n_t)\times {(n}_s+n_t)}$, ${\mathbf{M}}_c\in {\mathbb{R}}^{{(n}_s+n_t)\times {(n}_s+n_t)}$, ${\stackrel{-}{\mathbf{M}}}_c^{\left(s\right)}\in {\mathbb{R}}^{{(n}_s+n_t)\times {(n}_s+n_t)}$ and ${\stackrel{-}{\mathbf{M}}}_c^{\left(t\right)}\in {\mathbb{R}}^{{(n}_s+n_t)\times {(n}_s+n_t)}$ represent the diagonal indicator matrix, the inter-domain weighted conditional distribution matrix of class $c$, and the intra-domain distribution matrix of the source domain and the target domain, respectively. It can be calculated:

Let the partial derivative $\partial f/\partial \mathbf{\beta }=0$, get the optimal solution:

where, $\mathbf{I}$ is the unit matrix, $\mathbf{K}$ is the kernel matrix, $\mathbf{M}=\sum _{c=1}^C{\mathbf{M}}_c$, ${\stackrel{-}{\mathbf{M}}}_{}^{\left(s\right)}=\sum _{c=1}^C{\stackrel{-}{\mathbf{M}}}_c^{\left(s\right)}$ and ${\stackrel{-}{\mathbf{M}}}_{}^{\left(t\right)}=\sum _{c=1}^C{\stackrel{-}{\mathbf{M}}}_c^{\left(t\right)}$, and the optimal classifier $f$ is obtained by replacing the Eq. (22) with the Eq. (12).

4.1. Different working condition bearing fault diagnosis experiment

In this paper, the open bearing vibration data set of Case Western Reserve University (CWRU) is used to verify the bearing fault diagnosis algorithm. The test device is shown in Fig. 2. The left side is a 2 horsepower three-phase induction motor, and the right side is a dynamometer for generating rated load. The two are aligned through a torque sensor. The measured object is a deep groove ball bearing installed at the motor drive end, and the vibration sensor is installed on the upper side of the motor drive end.

Fig. 2CWRU bearing experimental device

The selected data has four health type: normal (NO), inner ring fault (IF), ball fault (BF), outer ring fault (OF). Each of the three fault states has three fault sizes (0.007', 0.014', 0.0021') and four motor conditions (0 HP/1797 rpm, 1 HP/1772 rpm, 2 HP/1750 rpm, 3 HP/1730 rpm). The sampling frequency is 12 KHZ.

In this paper, the transfer task of bearings under different working conditions is set up. The bearing vibration data of any working condition is taken to construct the source domain sample set, and the data of another different working condition is taken to construct the target domain sample set, so as to simulate the bearing fault type identification under variable working conditions in practice. Among them, each health status has 100 samples, each sample contains 2048 data points, and each sample set has a total of 1000 samples. The data set is marked to obtain 4 sample sets, and two different data sets constitute a transfer task, so that 12 transfer tasks of variable condition fault diagnosis can be obtained. The data set is shown in Table 1.

4.2. Different working condition small sample bearing composite fault diagnosis experiment

In order to verify that the proposed method not only has better recognition ability on the public bearing fault data set, but also has better performance on other bearing composite fault data sets, and when the training samples are insufficient, it also reflects excellent generalization ability. The performance of the proposed method is evaluated by comparing it with several most popular transfer learning methods.

In this paper, the simulation experiment platform of bearing compound fault is shown in Fig. 3. The test bearing of this experiment is a detachable deep groove ball bearing, its model is 6205EKA. Three different working conditions (500 N/1800 rpm, 1000 N/1200 rpm, 1500 N/600 rpm) were set up in the experiment, and the sampling frequency was 16384HZ. The selected data have eight health states: normal (NO), outer ring fault (OF), inner ring fault (IF), ball fault (BF), inner ring + ball fault (IF+BF), outer ring + inner ring fault (IF+OF), outer ring + ball fault (OF+BF), outer ring + inner ring + ball fault (IF+OF+BF). Among them, each health type has 100 samples, each sample contains 2048 data points, and each sample set has a total of 800 samples. The data set is marked to obtain a sample set of three different working conditions, and the two data sets constitute a transfer task, so that six transfer tasks of different working condition fault diagnosis can be obtained.

Fig. 3Bearing compound fault test bench: 1 – motor; 2 – coupling; 3 – spindle; 4 – bearing housing; 5 – carbon brush; 6 – test bearing housing; 7 – vibration acceleration sensor; 8 – bearing; 9 – base; 10 – power; 11 – load bolts

At the same time, this paper sets the bearing compound fault identification task of small sample training set under different working conditions: take the bearing vibration data of any working condition, randomly select N samples from 100 samples of each bearing type, so as to construct the source domain small sample set, and take all the data of another different working condition to construct the target domain sample set, so as to simulate the actual small sample under the bearing variable condition compound fault type identification.

The data set description is shown in Table 3 and Table 4.

5.1. Experimental results of different working condition bearing

In order to demonstrate the effectiveness of the proposed method, the proposed method is compared with other machine learning algorithms, five mainstream transfer learning methods and a typical traditional algorithm, as follows:

SVM: A common baseline method in classification tasks.

BDA [21]: Weighted joint distribution adaptation method.

GFK [23]: Manifold feature learning method, marginal distribution adaptation on Grassmann manifold.

TCA [24]: Marginal distribution adaptation method.

JDA [15]: Joint marginal distribution and conditional distribution adaptation method.

MEDA [14]: Manifold embedded adaptive distribution adaptation method, which can dynamically change the weight of marginal distribution and conditional distribution.

Among them, GFK and TCA belong to the marginal distribution alignment method; BDA, JDA and MEDA are joint distribution adaptation methods that consider both marginal distribution and conditional distribution.

The transfer learning methods all use the KNN algorithm to obtain weak labels for the target domain, and the subspace dimension d is set to 5. The regularization parameter $\theta$ in this algorithm is set to 0.9; the parameters $\rho$ and $\lambda$ were set to 10 according to reference [14]. The parameters $\gamma$, $\delta$ and $\eta$ are set by traversing –1 to 1 interval of 0.1, and finally determined as $\gamma =$ 0.1, $\delta$ = –0.1 and $\eta =$ 0.1; the number of iterations $T=$ 10; the same hyperparameters involved in the other algorithms are set to the same value.

The bearing fault diagnosis results of seven classification algorithms under different transfer tasks are obtained, as shown in Table 5.

It can be seen from the table that the recognition rate of all tasks on the CWRU dataset is higher than that of other algorithms. The reason is that the two sets of data with different working conditions have different spaces. The method in this paper fully considers the divergence offset of inter-domain data, aligns the data divergence, and then further mines the distribution structure in the domain, adjusts the inter-class distance, and introduces the weighting factor to align the distribution, which effectively avoids the feature distortion, and can continuously adjust the spatial distribution of the data through iteration. The results show that the proposed method has good performance in the recognition accuracy of multi-fault size and multi-fault type under different working conditions. It is 5.34 % higher than the most widely used transfer learning method MEDA, and the standard deviation is the lowest, which proves the stability of the algorithm.

5.2. Identification of compound faults under different working conditions under small samples

In the compound fault identification task of bearing variable condition under small sample, the number of samples N of each type in the source domain is 2, so as to construct the small sample source domain, and the target domain takes all the sample data (100 samples for each fault type). The source domain target domain data volume ratio of each transfer task is 1:50 (labeled: unlabeled). Other hyperparameters are set to the same value. Since the source domain samples need to be randomly selected, the experimental results are accidental. Therefore, each task is repeated for 30 times. The average of the experimental results is taken as the final result, and the bearing fault diagnosis results of the seven classification algorithms under different transfer tasks are obtained. Table 6 shows the results of all tasks.

The bearing compound recognition rate of the proposed method is about 8 % higher than that of the MEDA method in the small sample source domain, and the recognition rate on each transfer task is better than other methods. The results show that the proposed method can not only effectively and accurately identify the location and size of bearing faults, but also effectively identify the composite faults of bearings in the case of insufficient labeled samples.

6.1. Analysis of bearing compound fault identification performance under small sample data

In order to reflect the superior performance of the proposed method in the case of small source domain samples, the proportion of labeled source domain samples and unlabeled target domain samples is continuously improved, and compared with other methods to obtain the results shown in Fig. 4.

It can be seen from the figure that the bearing composite fault recognition rate of each method increases with the increase of the proportion of samples in the source domain and the target domain, which shows the importance of label information. In the transfer task, to align the bearing different working condition data domain and reduce the distribution distance of the different working condition data in space, it is often necessary that the amount of source data and target data should not be too different. However, in engineering practice, because the fault data of rolling bearings are difficult to obtain, a large number of labeled fault data cannot be collected, so it is necessary to solve the problem of low transfer efficiency under small samples.

When the number of samples in each class of the source domain is 1 (that is, when the ratio of samples to the target domain is 0.01), the recognition rate of the proposed method for the composite fault of rolling bearings is much higher than that of other methods, and the average recognition rate is 11.69 % higher than that of the MEDA. When the two-domain sample ratio is only 0.03, the average recognition rate can be almost the same as that of other methods when the two-domain data ratio is 1:1. Moreover, in the process of increasing the number of source domains, the accuracy of this method has been in a leading position. Whether it is a small sample or a large sample, the proposed method is superior to other methods in the identification of composite faults of rolling bearings under different working conditions.

Fig. 4Accuracy of different methods at different sample ratios

a) All results (Proportion of sample: 0.01-1)

b) Partial results (Proportion of sample: 0.01-0.2)

6.2. Hyperparameter analysis of the overall divergence matrix

In the subspace embedding divergence matrix for divergence alignment, we introduce the regularization parameter $\theta$. In order to reflect the influence of regularization parameters in the divergence matrix on the diagnostic model, the regularization parameter $\theta$ is analyzed below. The sample ratio of the source domain and the target domain is set to 1:1, and the value is taken at an interval of 0.1 from 0 to 1. Other parameters remain unchanged. Three transfer tasks are selected in the CWRU data set and the composite fault data set for testing, and the experimental results are shown in Fig. 5 and Fig. 6.

It can be seen from the figure that the hyperparameter $\theta$ plays an active role in the different working condition fault identification of rolling bearings. The recognition rate is also gradually improved in the process of gradually increasing the parameters. The accuracy of a single transfer task is improved by 1 % to 8 %, which proves that the introduction of regularization parameters plays a positive role in maintaining the structure of within-class divergence.

6.3. Convergence analysis

In order to study the algorithm convergence of the proposed method, four tasks in the CWRU dataset (F0→F1, F0→F3, F1→F0 and F2→F0) and all tasks in the composite fault dataset are selected for algorithm convergence analysis. The results are shown in Figs. 7 and 8.

Fig. 5Effect of regularization parameter θ on CWRU transfer task

Fig. 6Effect of regularization parameter θ on compound fault transfer task

Fig. 7Iteration process (CWRU data)

Fig. 8Iteration process (Composite fault data)

For the transfer task of CWRU data set, the iteration is fast, and it tends to converge after 3 to 5 iterations. In the composite fault data, some tasks obtain better recognition performance in the first iteration. In the subsequent iteration, the recognition rate only fluctuates within a small range, and there is no decrease in the recognition rate. The remaining tasks also tend to be stable after 2 to 3 iterations, indicating that the proposed method has good convergence.

6.4. Ablation analysis

Ablation research is the process of removing some parts of the model to better understand the behavior of the model. In order to deeply analyze the contribution of divergence alignment, cross-domain weighted conditional distribution alignment and inter-class distribution alienation in this method, the first three tasks (F0→F1, F0→F2 and F0→F3) of the CWRU dataset are selected for ablation analysis. Three ablation models based on this method are constructed, which are Omits divergence alignment (ODA), omits weighting factor (OWF) and omits inter-class distribution alienation (OIDA), and compared with this method to obtain the results shown in Fig. 9.

It can be seen from the figure that each part of the fault diagnosis model proposed in this paper plays a positive role in the whole model. Among them, divergence alignment contributes the most to the whole model, and the recognition accuracy is increased by 30 % at most. The accuracy of the transfer task is also slightly improved by the weighting factor and the inter-class distribution alienation term, which proves that each part of the method is indispensable.

Fig. 9Ablation study results