An infrared image detection algorithm for power equipment based on search optimization for YOLOv5

2.1.1. A priori knowledge-based approach

Relying on expert experience, the geometric characteristics of the target and the contextual connection between the target and the background for reasoning, the detection effect is evaluated through the representation and inference of a priori knowledge and the construction of a new knowledge rule base. The judgement process of the a priori knowledge-based method is relatively repetitive and easily influenced by individuals, and the judgement time is long; therefore, this approach cannot provide timely feedback on the detection results.

2.1.2. Method based on template matching

The method is implemented in two steps: First, a template image for each category is constructed via manual design or is learned from the dataset. Then, the template image is superimposed on the given image, the degree of difference between all of the regions in the original image where the target is likely to appear and the template image is calculated, and the degree of difference is used as a metric to determine the location of the target. For example, Leninisha et al. [17] proposed a geometrically deformable model based on width and colour for extracting traffic road information from images. Han [18] et al. used additional information provided by a visual sensor system to improve the reliability of template matching, and experiments illustrated the effectiveness of the method. The template matching-based method has simple steps, but it is too dependent on the template used for detection, is not suitable for detecting multicategory targets, and has poor robustness.

2.1.3. Methods based on manual features

This method involves three steps: First, the candidate region is obtained, and the region in the image where the target may exist is initially located. Due to the uncertainty of the location, shape and size of the target appearing in the image, the sliding window algorithm is usually used to obtain the corresponding region of the target. A representative method is selective search (SS), which combines small-scale regions with similar features into large-scale regions with similar features, which are then merged into a large-scale region, thus obtaining an object image with consistent internal features and a high recall rate [19]. The second is feature extraction, i.e., using manually designed target feature description algorithms to extract local features from selected candidate regions, among which the more typical feature descriptors are HOG (Histogram of Oriented Gradient) [20] and SIFT (scale-invariant feature transform) [21]. In the third step, classification and localization, the features extracted from the candidate region are classified and judged by classifiers, where the commonly used classification algorithms are SVM [22], random forest [23] and various cascade classifiers [24]. Finally, the final target of interest is obtained via non-maximum suppression of the overlapping target candidate frames. Manual feature-based target detection methods can meet the needs of natural image target detection to a certain extent, but due to the difficulty of using manually designed features to express the high-dimensional semantic information of an image, they suffer from the problems of weak generalisability and weak robustness in target detection.

2.2.1. Candidate region-based method

This method focuses on feature selection of candidate regions; this process is based mainly on the CNN algorithm, which divides the detection process into two parts. In the first part, the candidate regions that may contain the target are extracted from the image, and in the second part, the candidate regions are corrected and classified to obtain the desired result. Girshick [25] proposed the Fast-CNN method, which is based on the CNN, but its computational complexity is excessive. He et al. [26] proposed Faster R-CNN, which uses a region-generating network to obtain candidate frames from feature maps and completes all of the steps of target detection in a single deep network, applying the detection mode from the input image directly to the output result. Dai [27] used a fully convolutional neural network for application to Faster R-CNN; in this approach the individual categories are evaluated via location-sensitive feature maps to obtain the corresponding category probability, which accelerates the speed of network detection while ensuring the accuracy of positioning.

2.2.2. Regression-based methods

These methods do not need to generate candidate frames but directly extract the features in the image to predict the target’s category and location information; the representative methods are YOLO and SSD. In YOLO, Zhao [28] proposed the use of clustering method to estimate the prediction bounding box of the target on the basis of YOLOv3, and used Markov chain to determine the distance between the initial cluster and each candidate point, and simulation experiments illustrated that the algorithm has a significant performance improvement. Xiong [29] used background subtraction combined with YOLO to obtain the locations and features of small dynamic targets, and the results revealed that the algorithm has good accuracy and low recall. Zhai [30] proposed a trainable Spiking-YOLO for low-latency and high-performance target detection, and the simulation results revealed that this algorithm has low latency and recall, high performance and detection, and significantly improved precision. Wu [31] proposed the YOLO object detection algorithm based on complex environments, which performs well on mAP values. Zakria [32] and others used a single-stage deep learning target detection model to study an improved YOLOv4 algorithm, and the results revealed improved target detection accuracy and robustness. Luo [33] proposed the YOLOv5-Aircraft method based on the YOLOv5 network, finding that the algorithm can improve the accuracy and speed of target detection, and Xu [34] added a small-scale detection layer, a bottleneck transformer and a CBAM mechanism to the YOLOv7 structure to improve the small-scale target capability of the model. Furthermore, related studies have been conducted on YOLOv8 [35-37]. The above findings indicate that the YOLO family of algorithms constitutes a relatively classical algorithm in the field of target detection, which is prone to reduced target localisation accuracy due to the lack of a priori information when predicting the target position. This phenomenon is due mainly to the lack of the key step of region sampling, which leads to less satisfactory results when detecting images with small targets. In terms of the SSD, Gong [38] proposed the enhanced SSD (FCR-SSD) based on feature cross-strengthening, and simulation experiments revealed that it has high accuracy and good detection speed in the small target detection task. Zhai [39] proposed the DF-SSD algorithm, which uses DenseNet-S-32-1 instead of VGG-16, and introduces a multiscale feature-layer fusion mechanism. The experimental results show that the method has higher detection results for small objects and objects with specific relationships. Huo [40] proposed an SSD algorithm for small object detection based on self-attention combined with feature fusion-SAFF-SSD, and simulation experiments revealed that the method produces good detection results. The above findings indicate that the SSD algorithm independently inputs the image features extracted from different convolutional layers into the corresponding network detection branches, which tends to cause repeated detection, leading to poor detection of small targets in images.

3.1.1. Random initialization location

The SSA starts with a random initial location. The location of the squirrels is represented by an $d$-dimensional vector, there are $n$ squirrels in the forest, and the initial location of each squirrel is expressed as follows:

where $F{S}_{i,j}$ denotes the $j(j\in (1,d\left)\right)$-dimensional position of the $i(i\in (1,n\left)\right)$th squirrel, $U\left(\mathrm{0,1}\right)$ denotes a random number between 0 and 1 that follows a uniform distribution, and $u{b}_j$ and $l{b}_j$ denote the upper and lower bounds of the $i$th squirrel in the $j$-dimension, respectively.

3.1.2. Adaptation degree ranking and classification

After initializing the position vector for each squirrel, the decision variables are entered into the fitness function, and the resulting fitness value is $fs=(f{s}_1,f{s}_2,\cdots f{s}_n)$, where the fitness value for each squirrel is represented as follows:

The fitness value indicates the quality of the food source searched by the squirrels. All of the fitness values are sorted in ascending order via Eq. (3), and the squirrels are categorized according to the sorting results. The squirrel with the smallest fitness value is in the hickory tree $F{S}_{ht}$, the three squirrels whose fitnesses are sorted between 2-4 are in the oak tree $F{S}_{at}$, and the rest of the squirrels are in the common tree $F{S}_{nt}$, which are defined in Eq. (4)-(6) as follows:

where $index$ denotes the location of a specific squirrel.

3.1.3. Squirrel location update

Squirrels glide back and forth between different trees in search of food, and this foraging behaviour is affected by the probability $P_{dp}$ that a predator is present. There are three possible scenarios for squirrel position updating in foraging behaviour:

Scenario 1: Squirrels in Oak Trees Move Towards Hickory Trees.

A squirrel flies from the oak tree towards the hickory tree, and the new position of the squirrel is expressed as follows:

where $F{S}_{at}^t$ denotes the position of the squirrel on the oak tree in generation $t$, $F{S}_{ht}^t$ denotes the position of the squirrel on the hickory tree in generation $t$, $t$ denotes the current number of iterations, $d_g$ is the sliding distance, $G_c$ is the sliding constant (usually taken as 1.9), $R_1$ is the random number obeying the uniform distribution between (0, 1), and $P_{dp}$ is 0.1.

Scenario 2: Squirrels in Common Trees Move Towards Oak Trees.

A squirrel flies from the normal tree towards the oak tree, and the new position of the squirrel is expressed as follows:

where $F{S}_{nt}^t$ denotes the position of the squirrel on the common tree in generation $t$ and where $R_2$ is a random number that obeys a uniform distribution between (0, 1).

Scenario 3: Squirrels in Common Trees Move Towards Hickory Trees.

A squirrel flies from the normal tree towards the hickory tree, and the new position of the squirrel is expressed as follows:

where $R_3$ is a random number that obeys a uniform distribution between (0, 1).

3.1.4. Seasonal monitoring and random winter relocation

The foraging behaviour of squirrels is strongly influenced by the season. Compared with frequent activities in summer, in mid-winter, squirrels are less active to conserve energy, leading to stagnation. Seasonal monitoring is thus introduced into the algorithm to prevent the algorithm from falling into local optima, and the number of seasons is calculated as follows:

The minimum value of the seasonal constant is as follows:

where $t$ and $t_{\mathrm{m}\mathrm{a}\mathrm{x}}$ denote the current iteration number and the maximum iteration number, respectively. When $S_c^t<S_{min}$, the winter season is over, and Eq. (12) is used to randomly locate the position of the squirrels in the common tree who cannot find food:

$lev{y}^{\mathrm{\text{'}}}$ flight is a mathematical tool for enhancing the global optimization algorithm, which can cause the population to evolve away from or avoid falling into local optima. In this algorithm, $lev{y}^{\mathrm{\text{'}}}\left(n\right)$ is expressed as follows:

where $r_a$ and $r_b$ obey a normal uniformly distributed random number between (0, 1) and $\beta$ is 1.5:

where $\mathrm{\Gamma }\left(x\right)=(x-1)!$

3.2.1. Input side

This component mainly consists of three parts. The first part is mosaic data enhancement, that is, random scaling, cropping, lining up and splicing of images, which can improve the robustness of the detection model. In the second part of the adaptive anchor frame calculation, the initial anchor frame is set before training, and an inverse update of the model parameters is completed according to comparison with the real frame during training. In the third part of adaptive image scaling, i.e., when the letterbox function is improved, by calculating the most suitable image size, the image features are better preserved, and the detection speed is improved.

3.2.2. Backbone

The backbone is the main part of the YOLOv5 model. It primarily extracts the feature information from the target area, and it uses CSPDarknet as the backbone, which aims to fully extract the target feature information in the detection area through the Focus and CSP structure to improve the detection performance of the whole model.

3.2.3. Neck

The neck structure makes full use of the target feature information extracted by the backbone network in the detection area and further performs weighted fusion of this feature information and processing; finally, the output results are classified and localized by the output.

3.2.4. Output

The output is responsible for classifying and predicting the feature information extracted from the target area in the backbone network after compression and fusion by the neck. The network uses 3 sets of 1×1 convolutional structures with output feature mappings of 76×76, 38×38 and 19×19. Each detection layer finally outputs a 30-channel vector.

4.1.1. CSP module optimization

The backbone network of the YOLOv5 architecture consists of several CSP modules. This module mainly reduces the computational cost and memory usage during network training and maintains the performance of the model. By dividing the feature map into two parts and processing them independently at different stages, the number of computations can be reduced, and the accuracy and precision of the model can be improved; however, due to the large number of parameters in the convolutional kernel of the module, a large number of computations are needed to detect the target. On this basis, the design of the Faster Net network structure inspired the optimization of the CSP module, and this work proposes the CSP_Faster module instead of CSP1_1, CSP1_2 and CSP1_3. In this module, to maintain consistency in terms of the number of channels in the output feature map and the ordinary convolution operation, the point-by-point convolution operation is attached to the local convolution, which can pay more attention to the centre of the image, thus reducing the amount of computation. The improved CSP_Faster structure is shown in Fig. 1.

Fig. 1CSP_Faster structure

4.1.2. Introduction of the SKNet attention mechanism

In the traditional convolutional neural network, the size of the convolutional kernel is not consistent, so it cannot effectively capture the image features of different scales; therefore, the SKNet attention mechanism is incorporated in the YOLOv5 model, and a mechanism is introduced into the last layer of the backbone to connect it with the neck. This connection can ensure the completeness of the information of the image features and can improve the feature information expression ability of the graph, as shown in Fig. 2.

Fig. 2SKNet attention mechanism

4.1.3. Using the SIoU loss function

A problem in target detection is that the background in which the target image is located is complex and generates more false detection frames. Therefore, the SIoU function, which introduces the vector angle between the real and predicted frames and redefines the correlation damage function, can effectively reduce the total degrees of freedom of the loss and improve the training speed and accuracy. A description of the relevant parameters is provided in the [41].

4.2.1. Henon chaos-based initialization

The squirrel search algorithm generates initialized populations randomly in the search space, which cannot traverse the search space uniformly, resulting in some limitations in the search range. To solve this problem, the algorithm is optimized by using the improved Henon chaos algorithm, which has better uniformity and faster iteration speeds than a single chaotic mapping. The formula is as follows:

where $a$ and $b$ are the parameters.

4.2.2. Predator probability optimization

In the process of updating the positions of individual squirrels, the predator probability $A$ causes the squirrels to use different search strategies. In the early stage of the algorithm, a global search is needed to expand the whole solution space, and in the late stage of the algorithm's optimization search, the individual squirrels tend to gather near the optimal solution, which causes the algorithm to fall into a local optimum; therefore, more local development of the algorithm is needed. To address this issue, an adaptive optimization strategy is adopted to optimize the predator probability $B$ according to Eq. (16), which can ensure that $C$ is able to find a balance between the global search and local exploitation of the algorithm as the number of iterations increases:

where $P_{\mathrm{m}\mathrm{a}\mathrm{x}}$ and $P_{\mathrm{m}\mathrm{i}\mathrm{n}}$ are the maximum and minimum values of predator probability, respectively, and where $t_{\mathrm{m}\mathrm{a}\mathrm{x}}$ is the maximum number of iterations.

5.2.1. Ablation experiment

To further illustrate the effect of the improved measures on the performance of the algorithms, the three improved measures for YOLOv5 were compared with HCSSA-IYOLOv5, and the results of the comparison are shown in Fig. 8. The figure shows that the $P$ value of all four algorithms gradually decreases with increasing $R$ value, but the $P$ value of HCSSA-IYOLOv5 is better than the results of the other three improvement measures, which indicates that the algorithm proposed in this paper has better performance.

Fig. 8Comparative results of the ablation experiments

5.2.2. Experimental comparison of different scenes

To further illustrate the recognition effect of the algorithms on infrared images in different weather conditions (heavy rain, lightning, gusty wind, and hail), the lightning arrester infrared image was chosen as the research object, and the PR results of the four algorithms are shown in Fig. 9.

Fig. 9PRs of the four algorithms in different scenarios

a) Heavy rain

b) Lightning

c) Gusty wind

d) Hail

According to the detection results, in four different scenarios, the corresponding PR values of our proposed algorithms have better results than the other three improved YOLO algorithms, which have better application results, illustrating that CSP_Faster improves feature information recognition, the SKNet mechanism ensures the integrity of the image feature information, and the SIoU loss function obtains better classification results. After the optimization of the SSA algorithm, the model’s performance improves, and the feature acquisition ability of infrared images is obviously enhanced.

5.2.3. Comparison of the recognition time

To further illustrate the performance of this algorithm, we collected 100 infrared images of four kinds of power equipment in different scenes to compare the recognition time of the four algorithms, and the results are shown in Table 2. The results in the table indicate that these algorithms have different results in terms of the power equipment recognition time, but the advantages of the proposed algorithm are more obvious. When detecting targets in images of surge arresters, compared with those of the YOLOv3, YOLOv4, and YOLOv5, the recognition rates increased by 2.95 %, 2.61 % and 1.85 %, respectively. Furthermore, the recognition rates for circuit breakers increased by 6.49 %, 5.61 % and 2.54 %, respectively; the recognition rates for mutual inductors increased by 3.07 %, 2.63 % and 1.95 %, respectively; and the recognition rates for the insulator improved by 2.13 %, 1.94 % and 1.32 %, respectively. These results indicate that the optimization of the CSP module, the introduction of the SKNet attention mechanism and the use of the SIoU loss function in the YOLOv5 model increase the performance of the model and thus reduce the identification time.

5.2.4. Comparison of detection target effects

Figs. 10-13 present the results of the target detection experiments for the four algorithms on infrared images of the four devices. Fig. 10 shows the lightning arrester detection results; the YOLOv3 detection result is 0.821, the YOLOv4 detection result is 0.834, the YOLOv5 detection result is 0.854, and the ISOA-IYOLOv5 detection result is 0.893.

Fig. 10Recognition by the four algorithms on surge arrester infrared images

a) YOLOv3

b) YOLOv4

c) YOLOv5

d) HCSSA-IYOLOv5

Fig. 11Recognition by the four algorithms on infrared images of circuit breakers

a) YOLOv3

b) YOLOv4

c) YOLOv5

d) HCSSA-IYOLOv5

Fig. 11 shows the circuit breaker detection results; the YOLOv3 detection result is 0.794, the YOLOv4 detection result is 0.801, the YOLOv5 detection result is 0.825, and the HCSSA-IYOLOv5 detection result is 0.852. The detection results of the transformer are shown in Fig. 12, where the YOLOv3 detection result is 0.788, the YOLOv4 detection result is 0.793, the YOLOv5 detection result is 0.815, and the HCSSA-IYOLOv5 detection result is 0.863. In Fig. 13, the insulator detection results are shown, and the YOLOv3 detection result is 0.812, the YOLOv4 detection result is 0.823, the YOLOv5 detection result is 0.834, and the HCSSA-IYOLOv5 detection result is 0.892. According to the above detection results, the HCSSA-IYOLOv5 algorithm proposed in this paper outperforms YOLOv3, YOLOv4, and YOLOv5 by 8.87 %, 7.67 %, and 5.11 %, respectively, on average. These results show that the algorithm has good target detection and recognition effects.

Fig. 12Recognition by the four algorithms on infrared images of transformers

a) YOLOv3

b) YOLOv4

c) YOLOv5

d) HCSSA-IYOLOv5

Fig. 13Recognition by the four algorithms on insulator infrared images

a) YOLOv3

b) YOLOv4

c) YOLOv5

d) HCSSA-IYOLOv5