Threshold analysis method for aircraft avoiding convective weather

2.1. Data spatiotemporal synchronization processing method

Doppler weather radar is the main tool for detecting and warning thunderstorms and severe convective weather. It consists of radar data acquisition system (RDA), radar product generator (RPG) and primary user processor (PUP), which can provide combined reflectivity (CREF ), hail index (HI), echo top height (ET), medium cyclone (M), storm track information (STI), radar reflectivity factor (R) and other information; Track data is the fusion of radar track data and flight plan data, reflecting important parameters of target flight route, speed, and even flight purpose. Refreshed every 8 seconds, it records the flight dynamics information of takeoff, landing, and overflight flights between airport pairs, including longitude, latitude, height, time, flight number and other information.

Since the sampling frequency of weather products (the time resolution is 6 minutes, 10 minutes, 1 hour, etc.; the spatial resolution is 1°*1KM, 1°*2KM, 1°*4KM, etc.) is inconsistent with the time and spatial granularity of track data, and aircraft flight is affected by weather samples differently, it is necessary to synchronize the information of the two in time and space. This process is realized by calling toolkits such as Os, Numpy, Pandas and Basemap with the help of the python language. After virtually superimposing the two-layer grid information of weather conditions and aircraft flow on the three-dimensional map of the airspace in the same time period, refer to the definition of the Weather Impact Traffic Index (WITI) [18], use functions, such as ‘concat function’, ‘merge function’ or ‘resample function’ to re-divide the airspace into several dense cell grids, that is, each grid point $ij$ (the total number of rows is $i$, the total number of columns is $j$) will contain two values of convective weather radar and the number of aircraft flights information. Finally, organize the information in the raster into a dataset.

2.2. K-means algorithm threshold range determination

The K-means algorithm is a typical partition-based clustering algorithm. Since it does not require a given training set to train a model, it belongs to an unsupervised learning algorithm. Based on the original characteristics of the data, this paper removes the abnormal points caused by unarranged or little arranged aircraft in a grid under no convective weather. Then divide the data set into two categories: the aircraft that can fly normally without being affected by weak convective weather, and others that cannot fly normally under the effect of the strong convection weather. The algorithm steps are as follows:

Step 1: Randomly select 2 points from the sample set with $n$ total convective weather data each day as the initial center of each cluster in the K-means algorithm, and divide the data into the clusters closest to the two points according to the minimum distance principle of Eq. (1) to complete a division:

where, $x$ represents each sample point in the sample set, and $C_k$ is to describe one of the two categories.

Step 2: In the newly generated cluster, recalculate the center point of each cluster according to the algorithm of Eq. (2), and divide it again:

where, ${\mu }_k$ represents the center point of each cluster.

Step 3: Constantly iterate step 2 until the result of each division remains unchanged, when the function is considered to have converged. Stop iteration to get the cluster center point $Z_1$ that the aircraft can fly normally without being affected by weak convective weather and the final cluster center $Z_2$ in two categories where the aircraft is affected and cannot fly normally in strong convective weather.

Step 4: The threshold value of the convective weather characteristic of daily aircraft obstruction can be obtained by Eq. (3):

Step 5: Eq. (4) shows the threshold range of the convective weather characteristic for which the final aircraft is blocked:

The algorithm is simple in principle, fast in convergence, easy to implement, and has a good clustering effect. It is suitable for finding the threshold range of convective weather characteristics that affect the obstruction of aircraft and provides basic data for the best avoidance threshold in the next random forest classification algorithm.

2.3. Determination of optimal avoidance threshold based on random forest classification algorithm

The principle of random forest is to build a forest composed of many decision trees in an irregular way, and all decision trees are unrelated to each other. When a new input sample enters the forest, each decision tree in it will judge which class the sample should belong to, and the class that is selected the most will be the answer [19]. It works by generating multiple classifiers, and each decision tree in the random forest learning and making classification judgments independently to finally combine into a single prediction. Random forest is an ensemble algorithm based on the decision tree, which can effectively process a large amount of data with a high accuracy rate. The result is better than any of those from one single classification and has a better effect on determining the best avoidance threshold. The classification modelling process of the optimal avoidance threshold that affects aircraft flight under convective weather based on the random forest is shown in Fig. 1.

Fig. 1Optimal avoidance threshold modeling process based on random forest

(1) Data set selection. Read the daily affected threshold range of aircraft obtained by the K-means algorithm in 2.2. and combine the 0-1 classification to construct a feature variable set of radar echo intensity value and aircraft flow value and split it into training set and testing set at the rate of 8:2.

(2) Build a CART decision tree. The Classification and Regression Tree (CART) is a binary decision tree that can handle classification problems. When using the CART algorithm for node division, it is necessary to calculate the Gini coefficient value of each feature and select the smallest one for division, and then recursively construct a decision tree to generate classification rules. This paper constructs a CART decision tree by using the Gini coefficient to select the best data segmentation features. As shown in Eq. (5), the size of the Gini coefficient is positively correlated with the error rate of the selected samples:

where, $p_k$ represents the probability that the selected sample belongs to $k$ categories, so the probability of this sample being wrongly classified is ($1-p_k$), with $K$ categories in the sample set.

(3) Generation of training subsets for each decision tree. In the random forest classification model, each decision tree has a corresponding training subset. Assuming that there are $N$ decision trees, $N$ training subsets must be randomly selected from the original training set. Use a sampling method with replacement to input the training set and train a single decision tree with randomly selected samples and features with replacement; Each node selects an optimal one among $m$ features for analysis until it cannot be split, then a decision tree is trained. Repeat this step to generate $n$ trees.

(4) Build a Random Forest. After randomly selecting the training set, randomly generate subsets of features for each decision tree. Features in this subset are called random features. In this paper, the original features are randomly selected. First, z feature subsets are randomly selected from the original feature set, and the number of features in each subset is A. For decision trees, each node is split on a subset of features. The decision tree grows to the maximum in the CART method without pruning.

(5) Decision-making of Random Forest. Finally, a random forest classifier is formed by the generated multiple decision trees to classify the data, and the result is determined by the voting of all decision trees [19] [20]. Use the test set data to verify the accuracy of the model. The test sample $a$ is used as the input of the random forest, and the output of the $k$th decision tree is:

The output of the random forest classification model is:

where $f_k$ represents a single decision tree classifier model; $i$ represents the classification result of a single decision tree; 0 means that the aircraft can fly normally without being affected by weak convective weather, while 1 means that the aircraft is affected and cannot fly normally in strong convective weather; $f_{RF}$ represents the output of the random forest classification model; $n$ indicates the total number of decision trees that the random forest model contains.

2.4. Algorithm reliability evaluation

To evaluate the reliability of the algorithm, the Accuracy Rate (AR), the Missing Alarm Rate (MAR), and the False Alarm Rate (FAR) are defined for judgment. The Eqs. (8-10) are as follows:

where $Q$ is the total number of data; $T_i$ is the number of correct predictions; $F_m$ is the number that has an impact that was predicted to have no impact; $F_n$ is the number that has no impact that was predicted to have impact.

3.1. Data source and preprocessing

In order to verify the effectiveness of the method, the radar reflectivity factor R, the core index that most directly reflects convective weather, is selected as the avoidance factor of important cloud characteristic parameters for research. It is the most important basis for weather classification, which reflects the flight risk index of aircraft in the airspace. The research scope is from 112° to 119° east longitude, 36° to 43° north latitude, and 1000 meters to 13000 meters altitude. The meteorological information research object is selected from the hourly-updated radar reflectivity network data with the grid resolution of 0.01° latitude and longitude and each moment corresponding to 13 altitude layers. Simultaneously, the radar track data of the same range is selected as the research object of flow information.

First, divide the meteorological data into 70×70 grids by 0.1 degrees, as shown in Fig. 2 and take the maximum radar reflectivity factor in each range to form the radar reflectivity factor value $R$ in the new grid range. Then, match the processed track data in line with altitude and time as standards, which are from the four-dimensional grid (longitude, latitude, altitude, time) divided by meteorological data. Finally, the information on aircraft flying over, taking off and landing in each grid will be obtained, as shown in Fig. 3. Then, to facilitate the subsequent exploration of the relationship between the flow rate and the strength of the radar reflectivity factor $R$, the threshold value of the radar reflectivity factor $R$ is determined. Based on the fusion of meteorological and track data, make groups according to different radar reflectivity factor $R$ values and calculate the aircraft flow value that corresponds to each radar reflectivity factor $R$ while removing the outliers. Table 1 shows the groups of radar reflectivity $R$ and the corresponding aircraft flow value (sorties).

Fig. 2Schematic diagram of mesh division

By reviewing the historical radar data for each day in June and July 2021, analyzing the influence of weather on aircraft operation, selecting typical weather processes and corresponding track data, 20 typical convective weather days and 1022 influencing factors are selected as the research objects to analyze the affected characteristics of the aircraft during actual flight.

Fig. 3Fusion results of meteorological and track data (partial data)

3.2. Determination of threshold range

Based on the statistics of the aircraft flow data corresponding to the radar reflectivity factor $R$ on 20 typical convective weather days from June to July, scatter plots and cluster maps of the daily radar reflectivity factor $R$ and the flow value is drawn in Fig. 4.

Table 2 shows the daily center point clusters and convective weather feature thresholds. It can be seen from the above that when the threshold range of the radar reflectivity factor $R$ is between [30, 36], it is the center point of the two clusters of the clustering algorithm. After exceeding this threshold range, the change of the flow value tends to be stable, so the threshold range of the radar reflectivity factor $R$ can be determined to be [30, 36] dBZ.

3.3. Determination of the optimal threshold

To scientifically, objectively, and accurately find the radar reflectivity factor $R$ that can affect the operation of aircraft in convective weather, this paper uses the random forest classification method in machine learning for 0-1 classification and obtain the correlation between the radar reflectivity factor $R$ and the aircraft operation. This process was implemented by python.

The number of generated decision trees (n_estimators) and the maximum depth of the tree (max_depth) are two relatively important parameters in the random forest model. These two parameters will greatly affect the quality of model training results, for other parameters use default values. Cross-validation is used to find optimal parameters. Under this method, first is to select the number of n_estimators that generate decision trees, and then select max_depth, the maximum depth of the tree. The selection process is shown in Fig. 5 and Fig. 6. It can be seen from these two figures that the accuracy rate is the highest when the number of decision trees is 5 and the maximum depth of the tree is 2.

Fig. 4Scatter and cluster diagrams of some radar reflectivity factors R and flow values

a) June 8, 2021

b) June 23, 2021

c) July 1, 2021

d) July 17, 2021

Fig. 5Decision tree quantity selection process

Fig. 6The maximum depth selection process of the tree

For the more accurate determination of the optimal avoidance threshold of the radar reflectivity factor, this paper uses the above model to take 7 dBZ radar reflectivity (the minimum radar reflectivity factor value after removing abnormal points from the original data) as the initial threshold value for the existing data, in which the scope ends at 66 dBZ at 1 dBZ intervals. Calculate the corresponding random forest classification results under different radar reflectivity factors $R$, and the results are shown in Table 3.

Seen in Table 3, while verifying the above model, the classification result changes at the threshold of 34 dBZ. That is, when the avoidance threshold exceeds 34 dBZ, the aircraft cannot fly normally due to the impact of strong convective weather. Therefore, 34 dBZ is defined as the best avoidance threshold that affects the flight of the aircraft. This is similar to the conclusion made in the classification of convective weather characteristics in our country, saying “it is defined as strong convective weather when the radar reflectivity is greater than or equal to 35 dBZ, in which the aircraft needs to avoid flying”.

Fig. 7Comparison between decision tree and random forest algorithm

3.4. Inspection and evaluation of classification algorithms

To verify the advantages and disadvantages of the algorithm, the author compares this algorithm with the decision tree algorithm. Fig. 7 shows the compared results of the two algorithms, where the correct rate of the random forest algorithm is 3.44 % higher than that of the decision tree. The false alarm rate is 2.45 %, which is reduced by 1.96 %, and the missing alarm rate comes as low as 0.97 %, reduced by 1.48 % in comparison. In summary, the classification prediction model based on random forest can distinguish whether the radar reflectivity factor has an impact on the aircraft operation, and can prove the validity of the model, thus can be applied and prompted in the aviation business.