A steel scrap recognition model based on machine vision

1. Introduction

Steel scrap recognition is an essential process in the steel recycling industry. It involves identifying and sorting various types of steel scrap based on their physical characteristics, such as size, shape, and composition. Steel scrap can come from various sources, such as automobiles, appliances, and construction materials, and can be contaminated with non-metallic materials that need to be removed before the steel can be recycled. Traditional methods of steel scrap recognition involve manual sorting and visual inspection, which are time-consuming and can lead to errors. In recent years, machine vision technology has been developed to automate and improve the accuracy and efficiency of steel scrap recognition processes [1].

The current analysis of certain less prevalent residual elements, such as antimony (Sb), tungsten (W), cobalt (Co), nickel (Ni), and molybdenum (Mo), found in the steel scrap that steel plants process, is quite limited. Historical data from Miranda et al. [2] reveals an examination of these elements within the steel compositions from two distinct regional steel plants, spanning from 2011 to 2018, encompassing approximately 21,000 heats. Ali et al. [3] concentrate on advancing industrial symbiosis to foster a circular economy by employing design for value addition: exemplified through a case study that repurposes the sheet metal waste from the automotive industry into Voronoi facade systems. A pioneering approach for the recovery of size-specific sheet metal scrap from the automotive sector is suggested. Zhu et al. [4] delve into the annual steel flow in the United States, conceptualizing the U.S. steel flow with nodes that symbolize processes and products. Gao et al. [5] introduce a semi-supervised convolutional neural network (CNN)-based methodology for detecting surface defects in steel. They propose a semi-supervised learning technique for identifying steel surface defects, leveraging the power of CNNs. Guan et al. [6] put forth an innovative algorithm for the identification of steel surface defects, grounded in enhanced deep learning network models and incorporating feature visualization and quality assessment. Karimi et al. [7] explore the post-heating performance of CFST columns under various temperatures (20, 250, 500, and 750 °C), considering the volume fraction of scrap-tire rubber aggregate in sand (0, 5, and 10 %), the volume percentage of steel fibers (0, 1, and 1.5 %), and the ratio of the outer diameter to the thickness of the steel tube (43 and 25.4). Fakhri et al. [8] seek to assess the impact of scrap metal additives on the mechanical and self-healing attributes of asphalt mixtures when subjected to microwave heating. Çelik et al. [9] scrutinize the influence of lathe scrap fibers, a byproduct of Computer Numerical Control (CNC) lathe operations, on the performance of concrete. Additional significant contributions to the field are made by Gao et al. [10-12]. Based on the above problems, the fusion of different elements will directly lead to significant changes in the shape and color of the steel. These significant appearance changes are helpful for machine vision analysis of the classification and danger level judgment of steel scrap materials.

In recent years, more and more researchers have begun to try to apply artificial intelligence to the field of steel scrap recognition. Xu et al. [13] proposed a steel scrap recognition method based on deep learning, which classifies steel scrap based on the CSBFNET model and achieves good results. However, this method is based on simulation experiments and lacks further discussion on practical application. Zheng et al. [14] proposed an improved CNN model to recycle Scrap Bundles, which has high accuracy and noticeable effects in the practical application process. However, the defect is that the type of scrap identification is single, and it cannot effectively identify the dangerous substances doped in the scrap pile in the complex environment. Zhou et al. [15] proposed a Grabbing Robot System Based on Grasp Quality Convolutional Neural Network. By taking the approach of robot arm intelligent system to replace artificial. However, the efficiency of this method is low, and an efficient artificial intelligence model is needed to improve the recognition accuracy and sorting efficiency of steel scrap recycling. It can be found through the research of the above personnel. The problem of steel scrap recycling has attracted more and more attention, and researchers believe that introducing artificial intelligence models can solve the problem of steel scrap recycling more efficiently. The fact is that there is a lack of an efficient, safe method that can identify multiple types of materials in the scrap pile for the actual production environment.

However, the difficulty of steel waste identification lies in the variety of shapes, sizes, and compositions of steel waste, which brings challenges to the accurate identification and classification of steel waste. In addition, scrap steel can be contaminated with non-metallic materials, such as plastic, rubber, and dirt, which can affect the quality and value of recycled steel. Traditional waste identification methods, such as manual sorting and visual inspection, are time-consuming and error-prone, especially the mixing of some dangerous goods has brought significant challenges to scrap recycling. How to quickly and efficiently classify the recycled steel accurately and ensure the recycling process's safety, the automatic machine vision model can effectively solve this bottleneck.

2. Framework design

The identification of steel scrap needs to distinguish the types of steel scrap (section steel, plate steel, round steel, etc.) and the dangerous goods and closed objects that are not scrap. Secondly, it is necessary to judge the quality grade of scrap according to the thickness and length of the scrap. Finally, the recycling of scrap steel piles was evaluated. yolov5 is used to design this model, and the scrap recycling frame diagram is shown below:

As shown in Fig. 1, the scrap recognition system is mainly trained by three yolov5 sub-models. The Scrap type identification sub-model mainly completes the classification of scrap types and identifies whether there are dangerous goods and closed objects in the scrap pile. The corresponding alarm will be issued if dangerous goods and closed objects are in the scrap steel. Scrap type identification migrates the training results to Scrap profile recognition model 1 to provide classification experience. Scrap profile recognition model 1 aims to classify scrap steel further to identify the quality of scrap steel better. This step is crucial. Scrap profile recognition model 1 recognition accuracy of scrap steel will directly affect the calculation of scrap price. The scrap profile recognition model 2 mainly identifies other cheaper scrap steel and migrates the classification results to the model 1.

Fig. 1Design architecture diagram

The whole model is implemented by yolov5, and the specific implementation algorithm is referred to literature [16]. First, The yolov5 loss function consists of two parts: classification loss and localization loss. The classification loss uses the cross-entropy loss function, which is defined as:

where $N$ is the number of samples, $C$ is the number of classes, $y_{i,c}$ is the ground truth label of class $c$ for the $i$-th sample, and ${\widehat{y}}_{i,c}$ is the predicted probability of class $c$ for the $i$-th sample. The localization loss uses the mean squared error (MSE) loss function, which is defined as:

where $S$ is the size of the feature map, $K$ is the number of boxes predicted per cell, $y_{i,j,k}^{obj}$ indicates whether the $k$-th box in the $j$-th cell of the $i$-th image contains an object, $Io{U}_{i,j,k}$ is the intersection over union (IoU) between the predicted box and the ground truth box, ${\widehat{y}}_{i,j,k}$ is the predicted confidence score for the $k$-th box in the $j$-th cell of the $i$-th image, and $\lambda$ and ${\lambda }_{noobj}$ are hyperparameters. The total loss is defined as:

where ${\lambda }_{loc}$ is a hyperparameter that balances the classification and localization losses. Finaly, the mathematical formulation of the yolov5 prediction algorithm can be summarized as follows: yolov5 network processes the input image and outputs a tensor of predicted bounding boxes for the image. Each bounding box is represented by five values $(x,y,w,h,c)$ and a vector of probabilities for object classes. The confidence score of each bounding box is calculated as the product of object probability and IOU. The network selects the highest confidence bounding box for each grid cell and applies a confidence threshold to filter out low-confidence detections. Finally, the remaining detections are subjected to NMS to eliminate redundant detections.

3. Experimental results

3.1. Forward preparation

In the process of the experiment, a large number of scrap steel with different specifications and materials need to be marked, which is conducive to the understanding of the workers in the factory, to adapt to the local conditions of steel recycling in China. In the experimental results stage of this paper, much Chinese pinyin is used to mark the scrap steel. Table 1 and 2 compares Chinese pinyin and English in the experimental results for unified understanding.

Table 1 mainly shows the classification of scrap steel, which is distinguished and labeled according to the shape of steel.

Table 1Steel scrap classification name comparison

Chinese label	English
Gangban	Steel plate
Gangquan	Steel ring
Gonggang	Industrial steel
Jiaogang	Angle steel
Yuanguangang	Round pipe steel
Fangguangang	Square pipe steel
Fengbiwu	Enclosed object
Liuhuawu	Sulfur compound
Uxinggang	U shaped steel
Cxingcaogang	C shape steel

Table 2Steel scrap discrimination name comparison

Chinese label	Size label	English	Example
Zhongfei	1020	Heavy duty steel scrap	Steel nail, steel skin, parts and components
	0610		Bending steel, sheet metal
	0406		Rebar steel, railway track, wheel shaft
Xiaofei	0204	Medium steel scrap
Baofei	0001	Light steel scrap
Tongliao	0102	Copper material

Table 2 classifies the scrap into three types: Heavy duty steel scrap, Medium steel scrap, and Light steel scrap, according to the weight and shape of the scrap. The purpose of this is to distinguish the quality of scrap steel better. Generally speaking, the value of Heavy-duty steel scrap is the highest, medium is the second, and light is relatively low. In addition, it is difficult to avoid a large amount of copper doping in scrap steel recycling. Hence, the identification of copper is also essential, so copper is also included in the scope of consideration.

Fig. 2 shows part of the data sampled in the field. There are two main types of sampled data. One is independent sampling, which takes pictures of different shapes of scrap steel separately to obtain samples for labeling. The advantage of this is that the model can more accurately find and identify various types of scrap steel. The second method is centralized sampling, which takes large-scale photos of the scrap steel pile and performs local manual labeling. Although the second method is inaccurate, it is more in line with the actual needs and can further improve the model’s accuracy to meet the actual production needs. Next, yolov5 is used for model modeling, and experimental results are given. The environment of this experiment is shown in Table 3.

Fig. 2Steel scrap sample images

a)

b)

c)

d)

Table 3Experimental environmental

Operating system	RAM	GPU	CPU	Environment
Windows 10	8G	960M	i7-6700HQ	Pycharm:pytorch

3.2. Analysis of experimental results

The experimental results in this chapter are mainly divided into two parts: scrap classification and scrap quality grading. All experiments are written in python language, and the specific code and all results of the experimental process are open sources on GitHub (https://github.com/yangshaojie0123/Steel-Scrap-recognition).

3.2.1. Steel Scrap classification result

In the natural recycling environment, the composition of scrap material is complex, and the quantity of goods is enormous. Even if there is a large number of manual recycling work on site, it is difficult to accurately evaluate the overall recycling quality of the scrap pile, which directly leads to the low efficiency of scrap steel recycling and the waste steel that cannot be recycled is often bought at a high price. In addition, if many closed objects (such as gas tanks) in the scrap pile cannot be found in time, there are also serious safety risks in the recycling process of steelmaking. Therefore, it is necessary to distinguish and train the specific types of scrap steel, which is a crucial step to meet the practical application.

The experimental process was epoch20 times, and the detailed experimental results are shown in Fig. 3 and Table 4.

Table 4Steel scrap classification training results

obj_loss	cls_loss	precision	recall	mAP_0.5
0.13036	0.10075	0.92679	0.62549	0.71004

It can be seen from Fig. 3 and Table 4 that the model can quickly and accurately classify scrap steel. Although many categories are classified, the model precision can also reach 0.9267, and the value of mAP_0.5 can also reach 0.71004, which shows good performance. On the other hand, manual acceptance of scrap recycling in real-time cannot meet this accuracy requirement. More specifically, we sample the scrap recycling process in a natural environment, and the training results are shown in Fig. 4.

According to Fig. 4, the model can quickly distinguish the types of steel scrap, and the recognition effect is satisfactory in the actual process of steel scrap recycling. The model can give real-time scrap classification suggestions in the actual production environment. This improves recycling efficiency and resource utilization, identifies dangerous materials in real time to prevent casualties and property losses, and gives objective and fair scrap recycling pricing.

Fig. 3Steel scrap classification training process diagram

a)

b)

Fig. 4Practical effect of steel scrap classification

a) The actual overall rendering of the recycling process

b) More detailed rendering of scrap classification

3.2.2. Steel scrap discrimination result

Although the steel scrap classification has an excellent effect, this task must fully meet scrap recycling needs. Finally, the scrap grade is evaluated and priced [17]. Machine vision is also needed to rate the scrap material and judge the material and weight of different scrap materials. Only by judging steel scrap according to light, medium and heavy can it provide a basis for the actual recovery. Fig. 5 and Table 5 show the specific training results for this purpose.

It can be seen from Fig. 5 and Table 5 that the model can quickly and accurately discriminate the scrap steel. The classification labels are more complex because scrap discrimination involves indicators such as thickness, length, and shape. As a result, precision has decreased to 0.8789. On the other hand, because the discrimination of scrap steel leads to more stringent requirements on the model, MAP_0.05 also decreases to 0.5527. However, the model as a whole still shows surprising results. It is difficult for workers in the factory to distinguish the thickness and length of different wastes in real-time, not to mention to identify impurities such as closures and copper materials in many piles. The precision of 0.8789 and recall of 0.54982 is enough to provide an adequate evaluation basis for the recovery work of on-site workers. The actual training results are shown in Fig. 6.

As shown in the results of Fig. 6, in the face of vast and complex steel scrap heap recovery work. Finally, the model can complete the task, accurately distinguish different types of scrap steel, and then know that the field staff can complete the accurate recycling of scrap steel.

Fig. 5Practical effect of steel scrap discrimination

a)

b)

Fig. 6Practical effect of steel scrap discrimination

a) The overall effect of steel scrap discrimination

b) More detailed rendering of scrap discrimination

Table 5Steel scrap discrimination training results

obj_loss	cls_loss	precision	recall	mAP_0.5
0.12858	0.024864	0.8789	0.54982	0.5527

4. Conclusions

In this paper, the automatic recovery identification model of scrap steel based on machine visual guidance can be accurately classified for the type of different scrap steel, and the quality of scrap steel is determined. The paper provides more scientific and accurate guidance for scrap steel recycling in the steel industry. Better service in today’s society’s demand for carbon neutralization and carbon emissions, and improve the recovery quality of steel. In addition, the model also has strong extensibility, and the future accuracy of the recovery of scrap steel can be further improved, indicating that machine vision is feasible and efficient for the recovery of scrap steel. In the future, we will consider deploying more lightweight models and multiple cameras for image capture. It has a more accurate recognition effect for different kinds of scrap thickness, material, shape and other indicators. The recovery rate of scrap identification can be significantly improved, helping enterprises to recycle scrap more efficiently.