Visual target tracking based on fractional – order bidirectional hybrid attentional feature fusion

2.1. HFAF hybrid feature extraction module

For different dataset inputs, the pixel sizes to be processed by the target tracking model varies. Therefore, it is initially necessary to undergo operations for the preprocessing module of the model, such as crop and reshape to unify the input sizes of the bidirectional network. Specifically, in the following Fig. 2, it is hypothesized that the input image of the search branch is $x_S$, and the input of the template branch is $x_T$. After the crop operation, the reasonably cropped target-size images $X_S$and $X_T$ are obtained. At this time, the tracking object of our concern is in the center between $X_S$ and $X_T$.

Fig. 1FoBAF-T frame structure

In numerous classical target tracking tasks, convolutional networks are commonly employed as the primary architecture for feature extraction. With the prevalence of transformer networks, a retrospective examination of past research reveals that convolutional neural networks have the advantage of local modelling and spatial preservation on data with spatial structure. When serving as the backbone for feature extraction, CNN exhibits a light-weight nature that is currently lacking in Transformer models, in terms of parameter quantity and computational efficiency. Moreover, for images rich in spatial data structures, CNN naturally possesses advantages in hierarchical abstracted representations, handling local scales, and fine-grained textures. However, fully attention-based Transformer architectures offer superior advantages in the association modeling of sequential data and in handling long-range and historical dependencies, which are tasks with high dynamic requirements such as target tracking and virtual reality, and these benefits are difficult to match by convolutional neural networks. Therefore, in this paper, in light of the aforementioned research background and by integrating the merits of both in terms of different feature expressions, the attention mechanism is combined with the convolutional networks to obtain the feature extraction module – HFAF in the FoBAF-T framework.

The HFAF includes a Resnet50 feature learning phase and a feature aggregation phase. In the first phase, 1×1 convolution is used to map the features to a deeper space achieving parameter sharing for the purpose of increasing the network's nonlinear expressive ability. This also serves to reduce the number of parameters, thereby mitigating the risk of overfitting. In the second phase, fractional-order channel attention designed is introduced to capture the features of different channels in a fine-grained way. This allows the network to have greater flexibility in modeling complex distribution scenarios. Upon obtaining the channel attention weights, the hybrid convolutional pathway employs these weights to perform weighted fusion across the various channels, thereby achieving a fusion of channel attention with convolutional operations. Finally, to control the integration of the information from the two pathways, two learnable parameter controllers are added to nonlinearly integrate the information from the convolutional kernel attention pathway. The altered ResNet50 hybrid fractional-order channel attention structure adopted by the HFAF is shown in the following Fig. 2.

The first four stages of the original ResNet50 are retained and the down sampling step size is adjusted to 1 in stage4 to ensure the originality of the feature details. To ensure the richness and globality of feature extraction, the original convolutional layers are replaced by atrous convolution with a step size of 2 in this stage to expand the feeling field of the convolution operation. Let the residual block in this residual structure consist of two convolutional layers and a jump connection, and the output feature maps are $y_T$, $y_S$:

where $i$ is the number of layers, $W_i$ is the weight parameter of the $i$-th convolution operation, $F$ is the nonlinear mapping function constituted by two convolutional layers, $z_{y_S}$, $w_{y_S}$, $h_{X_S}$, $w_{X_S}$, $z_{y_S}$ are the pixel sizes of the input and output images, respectively, and the target image is the same. The features $X_{T_1}$, $y_T$ and $X_{S_1}$, $y_S$ obtained from stage 3 and stage 4 are jointly fed into the encoder to achieve the extraction of features at different levels in order to improve the representation of the model for multi-scale features.

Fig. 2HFAF structure diagram

2.2. FoMCA, fractional order multi-channel attention module

FoMCA belonging to HFAF is shown in Fig. 1. Let the input to the search branch of the fractional-order multichannel attention feature aggregation phase be $Y_S$, the mapping size of feature $Y_S$ be $w\times h$, and the number of channels be $z$. The channel attention weights $\omega$ are:

where Softmax is the activation function, $g\left(Y_S\right)$ is the global feature context, $B$ denotes batch normalisation, $W_2$ is the ascending layer, $W_1$ is the descending layer, and $\delta$ denotes the ReLU operation. Due to the limitation of channel attention on multi-scale feature space, this paper aggregates multi-scale contexts on the basis of channel attention, and uses point-by-point convolution (PWConv) as a local channel attention aggregator to improve the characterisation ability of channel attention (henceforth referred to as CA) for small objects and locally weak signals. The local context $L\left(Y_S\right)$ is computed by the following equation:

Next, the node function needs to be utilised to play a role between the similarity calculation and weighted combination to convert the similarity into attention weights. The softmax function is usually chosen to convert the similarity into a probability distribution so that the sum of all the attention weights is 1. The softmax function helps to ensure that the attention weights are normalised [6-7] so that they correctly represent the importance of the different input elements, which effectively captures the associations between the input elements and produces the appropriate attention weights.

Thus, the multichannel attention feature ${Y_S}^{\mathrm{\text{'}}}$ can be obtained as:

where $M\left(Y_S\right)$ denotes the attention weights generated after $Y_S$ passes through the multi-channel attention network, which are weighted by the local context and global context.

The node function is processed within the improved multichannel attention module using fractional order differentiation. Caputo fractional order derivatives are used here to describe the nonlocal relationships during feature fusion. fractional order derivatives produce a memory effect in the attention module by capturing historical information. This memory effect allows the model to better adapt to changes in target appearance and noise interference in the scene. Specifically, the fractional order derivative retains more historical feature information, thus maintaining stable tracking performance in complex scenes such as target occlusion and illumination changes. In addition, the fractional order derivative can also enhance the robustness of the model, making the model more sensitive to small changes in the input data, thus improving the overall functionality of the system, where $\mathrm{S}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}=p\left(Y_S\right)=\frac{e^{Y_S}}{{\sum }_{j=1}^r{e}^{Y_S}}$, is obtained by a first order derivation of $D^{0.5}p\left(Y_S\right)=\frac{1}{\mathrm{\Gamma }\left(0.5\right)}{\int }_0^{Y_S}\frac{p^{\left(1\right)}\left(t\right)dt}{(Y_S-t{)}^{0.}}$:

A 0.5 order derivative for $p\left(Y_S\right)$ is obtained:

Substituting $p\left(Y_S\right)$ and expanding by derivation yields:

Similarly, the 0.3 and 0.8 order derivatives of $p\left(Y_S\right)$ are obtained, respectively:

The first order differential image of the nodal function $\mathrm{S}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}$ and its 0.3 order differential image are shown below.

The Softmax function provides the probability distribution calculation for the output of the feature extraction network. Subsequently, multi-class decisions are implemented based on the Softmax probability calculation results, eventually generating the prediction outcome.

Fig. 3Softmax function image

a) First order differential image

b) 0.3 order differential image

As can be seen from the above figure, after replacing the integer-order differentiation with fractional-order differentiation, the convergence of the curve tends to become gentle, which is due to the characteristics of fractional-order differentiation, which no longer singly considers the current value in the process of convergence, but builds on the historical information of the signal computation, describing some nonlinear and complex features that are not available in the function under the integer-order differentiation, and thus complex pixel feature probabilities is able to be considered from multiple perspectives and more stably.

From Fig. 3, it can be seen that the trend of change in $\sigma$ based on fractional differentiation is different from that under integer order. Softmax(0.3) is less variable than Softmax(1). This indicates that the Softmax in fractional order has a stable and delicate convergence behavior, but its computational cost is not high. That is to say, the greater the rate of change of node functions within a unit is, the faster the model converges work, and under the same conditions with equal parameters, the shorter time it takes, but the probability calculation is incomplete and unstable, and it is not suitable for data bases with complex light texture conditions. The FoMCA module proposed in this paper is shown in Fig. 4.

Fig. 4HFAF structure diagram

2.3. FoBAF, fractional order attention feature fusion encoder

Upon obtaining the feature maps $Y_S$ and $Y_T$ from the HFAF module, these are fed into the fractional-order bidirectional attention fusion network (FoBAF-Net) to yield a bidirectionally fused enhanced feature map $Y_F$. Specifically, this paper designs a fusion network based on fractional-order attention to integrate features extracted from the template map and the search region. This network consists of three modules: the fractional-order self-attention module (FoSAM), the multi-head cross attention module (MCrossA), and the bidirectional fusion model (BFM), serving to preserve and restore texture detail features while focusing on the distribution of feature attention, respectively [8-10].

Fig. 5FoMSC multi-head attention mechanism

a) FoSAM

b) MCrossA

The above figures illustrate the main structure of the fractional-order bidirectional attention fusion network. Fig. 5(a) demonstrates the conversion of the multi-head self-attention from a conventional Transformer back to a self-attention mechanism, employing convolutional compression mapping to reduce the spatial dimensions of feature embedding vectors. This approach decreases the model's computational cost, accelerates the fusion process, and enhances the feature vectors processed through HAFA. Irrelevant semantic features derived from HAFA are discarded, while those with high relevance are preserved. Fig. 5(b) extends the fractional-order differential-based cross-attention module to multiple heads, capturing the correlations among input features and enhancing the model's capacity to represent the features of tracked targets in complex scenes [11-13].

2.3.1. FoSAM, fractional order compressed self-attention module

The dual-branch vectors $Y_T$, $Y_S$ transmitted from the HAFA module, alongside the feature vectors $X_{T1}$ and $X_{S1}$ output from the third stage of ResNet50, exhibit distinct attributes in their feature information. After concluding the extraction process in the preceding three stages, the resultant features are relatively shallow, comprising a greater proportion of low-level textures, edge details, and color information, thus possessing a local advantage. In contrast, the processing in the fourth stage yields deeper, abstract high-level semantic information, characterized by complex global features. Following the fractional-order multi-head channel attention described in Section 2.1, the establishment of relationships between channels enhances the features of interest while suppressing irrelevant channels, thereby mitigating noise interference within challenging attributes. This paper aims to fuse features with the aforementioned distinct information to enhance the model’s representational capability [14-17].

The self-attention mechanism can weight the importance of different features by calculating the similarity between them. The self-attention mechanism based on fractional-order node functions allows for nonlinear modeling of the similarity between features, providing a more flexible method for weight allocation. This enables the model to focus more on key target features, enhancing its performance capabilities. For the incoming variables $Y_T$, $Y_S$ and $X_{T1}$, $X_{S1}$, along with the spatial dimensions $h_{T1}$, $h_T$, $h_{S1}$, $h_S$, $w_{T1}$, $w_T$, $w_{S1}$, $w_S$ and channel counts $d_{T1}$, $d_T$, $d_{S1}$, $d_S$, a dimensionality reduction mapping is performed prior to entering the FoBAF fusion module to accelerate operation and reduce computational costs. Feature representations are typically high-dimensional and encompass rich information from the input data. Such data pose significant computational demands on the self-attention mechanism, particularly concerning image feature computations, where high complexity can lead to inefficiencies, contradicting the application requirements for target tracking. To better leverage these feature representations, embedding and encoding are employed to yield more meaningful feature representations, thus enhancing model performance. Consequently, through convolutional operations, the image features extracted by the HAFA module undergo spatial dimensional compression to obtain more compact vectors that are meaningful for subsequent attention encoding. Initially, dimensionality reduction mapping is conducted to ensure uniformity across $w\times h\times d\text{,}$ followed by flattening to derive the corresponding $q_{X_{T1}}$, $k_{X_{T1}}$, $v_{X_{T1}}$, $q_{Y_T}$, $k_{Y_T}$, $v_{Y_T}$, $q_{X_{S1}}$, $k_{S_{T1}}$, $v_{S_{T1}}$, $q_{Y_S}$, $k_{Y_S}$, $v_{Y_S}$. Finally, the resulting embedded vectors are concatenated to yield the processed $q_T$, $k_T$, $v_T$, $q_S$, $k_S$, $v_S$:

14

$\begin{array}{lll}{q}_T=Concat\left(q_{X_{T1}},q_{X_T}\right),& & q_S=Concat(q_{X_{S1}},q_{X_S})\end{array},$
$\begin{array}{lll}{k}_T=Concat\left(k_{X_{T1}},k_{X_T}\right),& & k_S=Concat(k_{X_{S1}},k_{X_S})\end{array},$
$\begin{array}{lll}{v}_T=Concat\left(v_{X_{T1}},v_{X_T}\right),& & v_S=Concat(v_{X_{S1}},v_{X_S})\end{array}.$

Fig. 6Reduced dimensional mapping operation (light red parts indicate sliding windows)

From the implementation of FoMCA discussed in Section 2.2, it can derive the expression based on the fractional-order node function:

15

$p^{\left(\alpha \right)}\left(Y_i\right)=\frac{1}{{\mathrm{\Gamma }}_{\left(\alpha \right)}}\underset{0}{\overset{Y_i}{\int }}\frac{p^{\left(1\right)}\left(t\right)dt}{(Y_i-t{)}^{\alpha }},$

where $Y_i$ represents the input to the fractional-order multi-head cross-attention module and $i$ indicates the dual-stream branch $i\in \{T,S\}$. By reverting the multi-head self-attention in the Transformer model to single-head and replacing the node function within the self-attention mechanism, it can derive the self-attention weights:

16

$SelfAttention\left(q,k,v\right)=FoSoftmax\left(\frac{q\times k^T}{\sqrt{d_k}}\right)v.$

2.3.3. BFM, bidirectional fusion module

After processing the features from both branches of the Siamese structure through the FoBAF-Attention module, it obtains the feature tensor $Y_T$ from the template branch and the feature tensor $Y_S$ from the search branch. These, along with the low-level feature vectors output from the Block3 stage of the HFAF module, are simultaneously input into the Bidirectional Fusion Module (BFM), ultimately resulting in the feature $Y_F$ after multi-attention fusion.

The FoSAM and MCrossA modules are capable of globally capturing the relationships among various positions within the sequences of feature vectors received from HFAF. This contrasts with the fixed-size convolutional kernels used in ResNet50 at each layer. As a result, the feature tensors obtained after processing through the encoder’s FoSAM and MCrossA possess a larger receptive field. During its learning process, FoSAM simultaneously considers the global information of the input sequence, thereby capturing contextual information over a broader range and expanding the receptive field of the feature tensor. In contrast, the receptive field of the feature vectors output from the third layer of ResNet50 is relatively small, primarily derived from local convolution operations constrained by kernel size and stride.

Herein, it is hypothesized that $Y_T$ represents a feature map with a larger receptive field. More specifically, $Y_T$ is a feature map, which belongs to the high-level semantic feature map, and $X_{T1}$ is the learned residuals in ResNet50 Block3, which belongs to the low-level feature map. Based on the multi-scale attention channel weights $M$, this two-stage BAF feature iterative fusion strategy can be expressed as:

21

$Y_F=Y_S\uplus Y_T=M\left(Y_S+Y_T\right)\otimes Y_S+\left(1-M\left(Y_S+Y_T\right)\right)\otimes Y_T,$

where $\uplus$ is the feature integration calculation. The integration weight $M(Y_S+Y_T)$ consists of real numbers between 0 and 1, allowing the network to perform soft selection or weighted averaging between $Y_S$ and $Y_T$.

This paper summarizes different forms of feature fusion within deep networks in Table 1. $G$ represents the global attention mechanism. Although various methods for feature fusion in different scenarios may differ in implementation, these details diminish once abstracted into mathematical forms. Thus, a carefully designed approach can unify these feature fusion scenarios, improving the performance of all networks by replacing original fusion operations with this unified method. Table 1 further illustrates those linear methods, such as addition and concatenation, do not rely on contextual information. Feature refinement and modulation are non-linear but can only partially perceive the input feature maps. In most cases, they primarily utilize high-level feature mappings. Unlike partially context-aware methods, fully context-aware methods encounter an unavoidable challenge: how to preliminarily integrate input features. As input to the attention module, the initial fusion quality significantly influences the final fusion weights. Since this remains a feature fusion issue, an intuitive approach is to employ another attention module to merge the input features.

Fig. 7Schematic diagram of BFM bidirectional feature fusion structure

After obtaining the fused feature vector $Y_F$, it is fed into the decoder for decoding to obtain the output map of similar responses. In this paper, classification regression is chosen to implement the tracker prediction head network.

Table 1Example of complex scene properties

Context-aware	Type	Formulation	Example
None	Addition	$X+Y$	RestNet
	Concatenation	$W_A{X}_{:,i,j}+W_B{Y}_{:,i,j}$	U-Net
Partially	Refinement	$X+G\left(Y\right)\otimes Y$	SENet
	Modulation	$G\left(Y\right)\otimes X+Y$	GAU
	Soft Selection	$G\left(X\right)\otimes X+(1-G(X\left)\right)\otimes Y$	Highway Networks
Fully	Modulation	$G(X,Y)\otimes X+Y$	SA
	Soft Selection	$G(X+Y)\otimes X+(1-G(X+Y\left)\right)\otimes Y$	SKNet
		$M(X+Y)\otimes X+(1-M(X+Y\left)\right)\otimes Y$	Ours

3.1. Training loss functions

Loss functions are essential in training neural networks. By measuring the difference between the predicted values and the true labels, loss functions capture the complex relationships between samples and labels. The loss function value reflects the error between the models’ predictions and the ground truth, where a smaller value indicates more accurate prediction. Therefore, the goal is to reduce the loss value, turning it into an optimization problem to find the optimal solution, thus enhancing the models’ predictive capabilities.

Since different loss functions apply to different tasks, they guide model learning in varying ways. In single-object visual tracking tasks, loss functions are typically divided into two categories: classification loss functions and regression loss functions. Cross-entropy loss is a classification loss function, while IOU loss (Intersection Over Union) and Smooth L1 loss are both regression loss functions.

3.1.2. GIOU loss function

Intersection Over Union (IOU) loss measures the error between the predicted bounding box and the ground truth bounding box, as shown in Fig. 8.

Fig. 8IOU diagram

Eq. (24) illustrates the calculation process of the IOU loss function:

24

$Los{s}_{IOU}=1-IOU\left(A,B\right)=1-\frac{C}{D}.$

The parameters in Fig. 8 will not be elaborated here. However, the IOU loss function has some shortcomings: IOU cannot reflect the degree of overlap between the predicted and ground truth boxes when they do not intersect. In such cases, the loss value is calculated as 0, which does not produce a corresponding gradient, thereby failing to fulfill the purpose of learning and training. The GIOU loss function used in this section addresses the gradient issue when the two boxes do not overlap by introducing an additional penalty term $\psi$, as shown in Eq. (25):

25

$Los{s}_{GIOU}=Los{s}_{IOU}+\psi =1+\frac{C}{D}+\frac{\left|C-D\right|}{\left|C\right|}.$

Among them, $\psi =\left|C-D\right|/\left|C\right|$ is the penalty term, along with $C$ and $D$, shares the same meanings as in Eq. (24). To some extent, LossGIOU mitigates and resolves the limitations of the IOU loss function.

3.2. Online update and window penalty

This paper makes minimal modifications to the tracking prediction head, inheriting the online template update and window penalty from previous work. In the inference phase, a conventional method is used by applying a Hanning window penalty to select the bounding box with the highest core as the regression result, further improving the models’ accuracy and robustness.

4.1. Quantitative analysis

To evaluate the effectiveness of the algorithms in this paper, tests and evaluations are performed on the test sets of OTB100, LaSOT and TrackingNet, respectively.

4.1.1. Experimental quantitative analysis of the OTB100 dataset

The tracker of this paper’s algorithm in OTB100 achieves a superior AUC (0.692) and is compared with 11 trackers namely ToMP, TransT, Ocean, STARK [5], SiamRPN++, ECO, ATOM, SA-Siam [18], GradNet [19], DiMP-50 [20], DaSiamRPN. The results of the one-way evaluation (OPE) curves can be obtained as shown in the Fig. 9.

Fig. 9Results of quantitative analysis of OTB100

To verify the performance of the proposed algorithm in this paper, the same protocol and parameters are used in the quantitative experiments. On the data results in the following Table 3, it can be seen that the algorithm is in the advanced level in terms of precision and accuracy, and that the mixture of fractional-order attention mechanism as well as the improvement of the feature fusion module is a significant contributor to the accuracy enhancement of the tracker. Upon further analysis, the algorithm performs most prominently in the task challenging attributes of OTB100 for sequences with frequent light changes and strong background interference. Due to the attention of FoBAF-T on the target texture features, together with the paper’s regression-based discriminative prediction, the tracker is made more robust to target selection, which is specifically reflected in the improvement of 71 % success rate, 92.3 % accuracy rate, and target frame selection is less dependent on the stability of the target state.

Table 3OTB100 quantification results

Tracker	FoBAF-T	ToMP	TransT	Ocean	STARK	SiamRPN++
Success Rate	0.710	0.701	0.649	0.665	0.676	0.689
Precision	0.923	0.917	0.920	0.907	0.919	0.915
Tracker	ECO	ATOM	SA-Siam	GradNet	DiMP-50	DaSiamRPN
Success Rate	0.690	0.671	0.656	0.639	0.686	0.658
Precision	0.910	0.882	0.864	0.861	0.899	0.881

4.1.2. Experimental quantitative analysis of the LaSOT dataset

Fig. 11 shows the experimental evaluation results of the success rates obtained by SiamRPN, SiamRPN++, ATOM, DiMP, Ocean, TransT, SeqTrack and FoBAF-T among the 11 control algorithms running under 14 challenging attributes of the LaSOT dataset. Fig. 10 visually demonstrates the advancement of the algorithm improvements proposed in this paper in complex scenarios.

Fig. 10Comparison of AUC obtained by different algorithms on the OTB100 benchmark

In target tracking, in order to assess the accuracy and reliability of the algorithm, we focus on statistical features such as uncertainty and confidence intervals. Uncertainty assesses the reliability of the tracking results and gives us insight into the potential error and variability of the tracker’s predictions. To compute the uncertainty of tracking results, we employ the P and Pnorm metrics to measure the uncertainty in the target tracking task. Specifically, precision is a measure of how accurately the tracker predicts the position of a target object in a given video sequence. It is usually derived by calculating the ratio of correctly predicted frames to the total number of frames. In target tracking, higher accuracy means that the tracker is able to predict the position of the target object more accurately, thus reducing tracking uncertainty. However, accuracy is affected by a variety of factors, such as target occlusion, morphological changes, motion blur, etc., which may lead to a decrease in accuracy, thus increasing tracking uncertainty. Pnorm is a metric that normalizes accuracy by taking into account the effect of the size of the true bounding box. In target tracking, targets of different sizes may lead to different tracking difficulties, so the performance of the tracking algorithm can be evaluated more fairly using Pnorm. A higher value of Pnorm indicates that the tracking algorithm can maintain high accuracy on targets of different sizes, which reduces the tracking uncertainty to a certain extent.

Table 4LaSOT quantification results

Tracker	FoBAF-T	ToMP	TransT	Ocean	STARK	SiamRPN++
AUC	73.9	68.5	64.9	56.0	67.1	49.6
$P_{norm}$	82.1	79.2	73.8	65.1	77.0	56.9
$P$	79.6	73.5	69.0	56.6	-	49.1
Tracker	ECO	ATOM	SeqTrack	TrDiMP	DiMP-50	DaSiamRPN
AUC	32.4	51.5	72.1	63.9	56.9	58.6
$P_{norm}$	33.8	57.6	81.7	–	65.0	–
$P$	30.1	50.5	79.0	61.4	56.7	–

In the Table 4, it is evident that FoBAF-T performs slightly better than the latest released algorithm, achieving an AUC score of 73.9 %. The Fig. 11 illustrates the outcomes of the quantitative evaluation conducted under 14 complex scene attributes. This evaluation reveals that the proposed FoBAF-T tracker exhibits a notable advantage in the areas of background discrimination similarity and scale change attribute. This suggests that the model possesses a superior capability in discerning both similarity and scale changes [21-24]. FoBAF-T has a P-value of 73.9 on the ToMP test set, which indicates that it is able to track the target accurately in most cases with low uncertainty. While STARK has a P-value of 56.0 on the SiamRPN++ test set, which indicates that STARK has some errors in the tracking process, and the low P-value increases the uncertainty of target tracking.

4.2. Ablation experiment

In this paper, the improvement contributions of different components in FoBAF-T are evaluated through ablation experiment, and the ablation tests are carried out on OTB100 and UAV123. The experimental structure is shown in Table 6.

The components of the ablation experiment are as follows:

HFAF Fractional Hybrid Attention Feature Extraction Network: Here, FoBAF-T modifies ResNet50 and combines it with the channel attention that integrates multi-scale context through replacing the node functions, obtaining a reconstructed feature extraction network. Subsequently, the obtained features are fed into the fusion network.

Fig. 11Comparative results of different challenge attributes on the LaSOT dataset

FoBAF Feature Fusion Network: Compared to the baseline algorithm, the fundamental difference between the feature fusion network in this paper lies in the node function design and the strategy of Siamese branch fusion, which is decomposed into a two-stage structure with a focus on the contribution of the two methods to the similarity measure in the algorithm. The experimental data obtains in Table 5 indicates that if the features are directly weighted and summed without any other processing, the desired result of the algorithm in this paper cannot be achieved [25-27].

4.3. Comparative experiment

To visually observe the robustness of the algorithms in this paper under a variety of difficult scenario factors, the experiments in this subsection selected the visualization test results of the algorithms in this paper and the comparison algorithms on randomly selected sequences in three datasets, namely, OTB100, LaSOT, UAV123, and ImageNet. This subsection presents four video sequences. The sequence of tracking frames is represented from left to right, respectively. Among them, the green bounding box represents the real tagged value results; the red bounding box represents the tracking prediction results of the algorithm in this paper; the light blue bounding box represents the DiMP algorithm tracking prediction results; the dark blue bounding box represents the SeqTrack tracking prediction results; the pink bounding box represents the SiamRPN tracking prediction results; and the yellow bounding box represents the baseline algorithm TransT tracking prediction results. The above qualitative analysis is shown below in Fig. 12.

Fig. 12Results of qualitative analysis under randomized sequences of four test sets

It can be observed from the following experimental results that in the bolt sequence involving challenges of fast movement and deformation, when the target is occluded by similar backgrounds in close proximity around it, the tracking boxes of both the SiamRPN and DiMP algorithms exhibit drift and tracking identity errors. However, compared with the compared algorithms Ours-A and FoFTr-T, the tracking results of the algorithm presented in this paper are more stable. In the Diving challenge involving fast motion and motion blur, the tracking bounding box of SiamRPN drifts due to the cluttered background. When there are drastic shape changes and complex backgrounds around frame 99, DiMP suffers from severe drift and inaccurate tracking box size. The algorithm presented in this paper is stable and capable of adapting to the rotation and deformation of the target. In the Group2 sequence that involves fast movement of small targets and scale variations, the algorithm in this paper yields tracking results that are more in line with expectations compared to the baseline algorithm TransT and the advanced tracker SeqTrack.