Comparison study of fast independent component analysis and constrained independent component analysis

1. Introduction

The issue of Independent Component Analysis (ICA) was proposed in the 1980s, although the name of the ICA was not adopted at the time. Common [1] proposed a blind source separation independent component analysis (ICA) method based on instantaneous aliasing signal analysis, and clarified the concept of ICA. A Kullbak-Leibler criterion is proposed as a comparison function, and a high-order approximation and comparison function based on the probability density function is used to measure the statistical independence of each component of the signal. In 1995, A. J. Bell and T. J. Sejnowski [2] analyzed the entropy of linear system maximizing output nonlinear nodes based on information theory, and obtained the criterion function of maximum information transmission.From this, an adaptive blind source separation and blind deconvolution method (informax) is derived which can recover the source signal from the mixed signal. The algorithm uses kurtosis as a measure function, which is very effective for separating speech signals (linear aliasing), but it is difficult to separate negative entropy signals. Constrained Independent Component Analysis (cICA) was originally proposed by Wei Lu and J. C. Rajapakse [3] applied it to EEG in 2000. In 2005, Wei Lu and J. C. Rajapakse [4] proposed a complete cICA algorithm framework. Since then, many literatures have studied it, and the literature [5-7] studied the cICA algorithm. The literature [8, 9] describes in detail how to establish a reference signal in the cICA algorithm, and points out the method and steps of establishing the reference signal and the importance of the reference signal in the cICA algorithm. In the field of application, cICA is mainly used in medical engineering, especially electrocardiogram, electroencephalogram and electrocardiogram. By combining independent component analysis with prior information in medical engineering, cICA improves the accuracy and effectiveness of independent component analysis. However, in the field of mechanical fault diagnosis, the application of constrained independent component analysis is relatively small.

In China, Professor He Zhenya from Southeast University [10] was the first scholar to start blind source separation research. He proposed a series of new criteria based on feature analysis and blind source separation and blind deconvolution of higher-order spectra, and achieved fruitful results. Some scholars in China have also followed suit, researching and innovating different blind source separation algorithms. They have achieved some results in software and hardware and have been developed in the fields of communication, speech and biomedicine [11].However, there are relatively few studies on constrained independent component analysis, the literature [12] points out that the theoretical development of cICA and proposed the development of ICA with reference signal, the application of ICA-R, including the selection of threshold parameters, the CompleteI ICA-R algorithm is a very flexible ICA algorithm compared to traditional ICA. The establishment of the reference signal and the influence of the parameters on the extraction of the sign signal are mainly described. In [13], constrained independent component analysis was applied to the separation of electroencephalogram (EEG) bit differences. The feasibility of this method was proved by simulation and corresponding engineering application research. Literature [14-16] respectively studied the cICA algorithm and cICA in the infrared point detection application.

2. FastICA algorithm and cICA algorithm

1) Introduction to FastICA algorithm.

Assuming a certain observable vector $x$, let $x$ satisfy the ICA model $x=As$, $A$ is an unknown mixture matrix, $s$ is the source signal. First of all, the raw data is preprocessed – whitened, resulting in a whitening signal $z=Qx$, here $Q$ is a whitening matrix, $z$ to meet the condition is $E\left\{z\right\}=$ 0, $Cz=I$. FastICA algorithm is to find a separation matrix $W$, make $y=Wz$ as independent as possible, here $y$ is the estimate of the source signal $s$. For an independent component of the estimate $y=wTz$, the negative entropy $J\left(wTz\right)$ is maximized, that is non-Gaussian maximization. And must satisfy the condition is $E\left\{\left(wTz\right)2\right\}=‖w‖2=$ 1, then the optimal solution of $E\left\{G\left(wTz\right)\right\}$ is satisfied:

In the formula: $\beta$ is a constant, $g\left(·\right)$ is the derivative of $G\left(·\right)$. The following fixed point iteration can be obtained:

$W$ must be normalized after each iteration, however, the convergence property of this formula is not obvious, because non-polynomial moments do not have the same good algebraic performance as real cumulants. So, the fixed point iteration adjusted as follows:

The next step is to normalize $w$ as a unit norm. For the calculation of coefficients $\alpha$ in the formula, using Newton method.

According to Lagrange conditions, the extreme value of $E\left\{G\left(wTz\right)\right\}$ under the constraint $E\left\{\left(wTz\right)2\right\}=‖w‖2=$ 1 is obtained at the point where the gradient of the following Lagrange multiplier is zero:

The following Newton method to solve this equation, available approximate Newton’s iterative equation is:

This formula is the basic formula of fixed point iteration in FastICA algorithm.

2) Introduction of constrained independent component analysis algorithm.

It is known that $x_1\left(t\right)$, $x_2\left(t\right)$, …, $x_N\left(t\right)$ are $N$ input mixed signals $y_1\left(t\right)$, $y_2\left(t\right)$, …, $y_L\left(t\right)$ are the $L$ estimated source signals $r_1\left(t\right)$, $r_2\left(t\right)$, …, $r_L\left(t\right)$ are $L$ reference signals. The partial prior information with the source signal is not exactly the same as the source signal. In the constrained independent component algorithm, the similarity measure between output signal and reference signal is added to the objective function of traditional ICA, so that only $L$ line vectors $w_i$ ($i=$ 1, 2, …, $L$) $L$ out of the desired signal source signal $y_i（t）=w_{i}Tx（t）$. If we use only one reference signal $r_j\left(t\right)$ to find a row vector $w_i$ of $W$, we get $yi（t）=w_{i}Tx（t）$, it is the closest source to $r_j\left(t\right)$, which is called a unit of cICA. Constrained independent component analysis is based on the independent component analysis to add a constraint, the formation of constrained optimization problem, the basic model is:

The optimization objective function in the above formula is the contrast function of ICA. In the constrained independent component analysis (ICA), the expected independent components are extracted, and the number of independent components to be extracted can be flexibly selected. $g$$（w）\le$ 0 is inequality constraint. The prior information of the extracted signal is added to the ICA algorithm to force the algorithm to converge to the desired independent component. $h（w）=$ 0 is the equality constraint, the independent components of the output are unitized.

The similarity between the definition output signal $y$ and the reference signal $r$ is denoted by $\epsilon (y,r)$, if there is one and only one independent component is close to the reference signal, the following relation can be obtained:

In the formula, ${\mathbf{w}}^{*}$ is the separation vector corresponding to the independent component which is selected to separate, and $w_i$ ($i=$ 1, 2,…, $n-1$) is the separation vector corresponding to the other independent components. So, for the output only when $\mathbf{y}={\mathbf{w}}^{\mathrm{*}T}\mathbf{x}$, the following equation holds:

For the $z$ here is a threshold parameter, its value to meet the condition: $\xi \in \left[\epsilon \right({\mathbf{w}}^{\mathrm{*}T}\mathbf{x},r),\epsilon ({\mathbf{w}}_1^T\mathbf{x},r\left)\right]$. For the model Eq. (6) is a constrained optimization problem that can be solved by Lagrange multipliers.

3. Simulation experiment

1) Simulation experiment of FastICA algorithm.

In order to further illustrate the application of ICA principles and algorithms, this section presents a set of simulation experiments. First, four independent non-Gaussian source signals are generated according to Eq. (9) as shown in Fig. 1:

where $S_1$ is a sine signal, $S_2$ is a funny curve, $S_3$ is sawtooth wave, $S_4$ is pulse noise signal, sampling point $N$ is 500. Now produce a random mixed matrix $A$, Perform random mixing according to Eq. (3) to obtain observable time-domain waveforms as shown in Fig. 2. Perform the decomposition operation according to the FastICA algorithm described in Section 2.3.5. The resulting time-domain waveforms are shown in Fig. 3.

As can be seen in Fig. 1 and Fig. 3, although the signal is very complex, but after FastICA algorithm, the source signal has been better separation, separation accuracy is high, but the separation order and the signal amplitude has changed, this also validates two uncertainties in ICA decomposition: Separation order uncertainty and amplitude uncertainty. However, the waveforms of the separated signals are basically the same, that is, the waveform does not change in frequency domain, which does not affect the analysis and processing of signals. So, in the practical engineering application is allowed to accept such uncertainty.

Fig. 1Time waveforms of source signals

Fig. 2Time waveforms of mixing signals

Fig. 3Time waveforms of separation signals

2) Simulation experiment of constrained independent component analysis.

In order to further explain the basic principles of cICA and the application of the algorithm, this section gives a set of simulation experiments. First, four independent non-Gaussian source signals are generated according to Eq. (10) as shown in Fig. 4:

Here $f\_\text{1}=$ 550 Hz, $f_2=$ 143 Hz, $f_3=$ 50 Hz, $f_4=$ 38 Hz, $s_1$ is the gear fault signal that the gearbox amplitude modulation and the frequency modulation work together. The carrier frequency is $f_1$, the modulation frequency is $f_3$. $s_2$ is a simple bearing modulation signal, the carrier frequency is $f_2$, the modulation frequency is $f_4$. $s_3$ is to generate random white noise sequence of points $N$, $s_4$ is a funny curve. Through $f_s=$ 10 kHz sampling frequency acquisition data, sampling points for the 5000, take the first 1024 data.

Fig. 4Time waveforms of source signals

Now a random mixing matrix $A$ is generated, which is randomly mixed according to Eq. (8). The resulting time-domain waveform is shown in Fig. 5.

In practice, the gearbox parameters and speed of the gearbox are known, so the meshing frequency can be calculated. According to the steps of the reference signal establishment and the method thereof, a rectangular square wave reference signal is established to make its period equal to the meshing frequency of the fault gear of the simulated gearbox, and the constraint independent component analysis is performed to extract the fault characteristic frequency. The obtained result is shown in Fig. 6. Fig. 6 shows the generated periodic square-wave sequence equal to the gear meshing frequency and the gear fault signature extracted as a reference signal.

Fig. 5Time waveforms of mixing signals

Fig. 6Time waveforms of the appropriate reference signal and the extracted y1

Amplitude spectrum analysis is performed on the extracted fault signatures, as shown in Fig. 7. Fig. 7 shows the amplitude spectrum of the extracted signal $y_1$ in Fig. 6. It can be seen from the figure that there is mainly a gear fault engagement frequency of 550 Hz, and there is a significant modulation side band with a modulation frequency of 50 Hz, which is the result of the joint effect of AM and FM. Therefore, the mixed signal can be correctly separated by cICA method to extract the fault signature signal.

Fig. 7Signal amplitude spectrum of the extracted y1

4. Conclusions

This paper compares the mathematical foundations of FastICA and cICA, and analyzes the shortcomings of fast independent component analysis and the advantages of constrained independent component analysis. Constrained Independent Component Analysis (cICA) was developed from Independent Component Analysis (ICA), and the concepts and principles of independent component analysis still apply to constrained independent component analysis. The corresponding simulation results show that the algorithm is effective in extracting signal characteristics. In practical engineering applications, cICA has a wide range of applications in the fields of mechanical fault diagnosis, image analysis and processing, and signal denoising.