Comparison of UAV images processing softwares

1. Introduction

The mapping of terrain is a long lasting process that could last from half a year up to several years. Recently designed technology constructed by the scientists and engineers of Space Science and Technology Institute (SSTI) together with the students of A. Gustaitis Aviation Institute (AGAI) would assist in updating that process. For the purposes of minor terrain mapping like cases of maps updating, military purposes, studies of nature and for the other intentions it is recommended to deploy the constructed UAVs.

The UAVs are available in various sizes, shapes and structures; they signify themselves by the manifold characteristics [1]. Previously, those were ordinary model aircrafts under remote control, but recently, there is a tendency to use the autonomous unmanned aerial vehicles, which move according to the pre-programmed flight pathway or they could be operated by the help of a more complicated system of dynamic automation. For areal imaging purposes a digital, calibrated and integrated camera is present in an UAV, which is to acquire the Earth’s surface photographic images. The images made during the UAV flights can later be processed by applying the methods of photogrammetry, with substantially lower costs than they used to be in cases, when the images were taken from an airplane with the complicated and rather expensive equipment, devices and facilities [1-4].

SSTI and students of AGAI designed two UAV systems for aerial photography. One of the systems was used in four test flight projects. During the flights by UAV made lot of images. The images were processed by different software: Pix4d Mapper and PhotoMod. These experimental projects explain basics of images processing and shows the advantages and disadvantages and difference between used software’s.

2. Platform of UAV System

An unmanned aerial vehicle, commonly known as drone is an aircraft without a human pilot onboard. Its flight is controlled either autonomously by onboard computers or by the remote control of a pilot on the ground or in another vehicle. The tested UAV is an electric powered flying wing that is flown autonomously through its own onboard autopilot system. An Inertial Measurement Unit (IMU) provides stabilization for the aircraft. GPS (Global Position System) navigation is used to guide the aircraft to its destination. The UAV is landed autonomously by parachute via point and click on the included Ground Control Software or performing belly landing. The main performances are show in Table 1.

Table 1Flying wing X-5 and 1.8 m RC performance

UAV	X-5	1.8 m plane
Length	0.5 m	1,5 m
Wingspan	1 m	1.8 m
Gross weight	2 kg	5 kg
Payload	0.5-0.7 kg	0.5-0.7 kg
Max speed	80 km/h	70 km/h
Cruising speed	55 km/h	50 km/h
Flight time	30-45 min.	30-45 min.
Range	30-50 km	20-40 km
Engine	Electrical	Electrical
Control	Fully automatic	Fully automatic
Take off	Hand launch	Hand launch
Landing	Parachute	Belly landing
Telemetry of control	430 MHz direct control about 15 km	430 MHz direct control about 15 km
People require in service	1	1
Camera	Canon S100	Canon S100

The planes performances are very similar. One of them is flying wing type and second one – conventional. Due to the airframe type (flying wing) X-5 is faster, smaller. Both of them are made of foam and control systems are the same. It is Arduino-based ArduPilot. Ground unit consists of a computer with special software and telemetry module.

The UAV had an installed digital camera Canon S100 intended for test flights. X-5 has the camera mounted in the fuselage, with automatic lens coverage and 1.8 m plane has photo camera hang up under the wing. The camera lens optics was calibrated by the special German program Tcc according to the calibration test field – cube and the mode self-calibration processes by Pix4d Mapper software (see an article Chapter 4) [5, 6]. Flight over four tested objects was made by 1.8 m plane.

3. Data of UAV test flights

The test flights were performed on four areas located near Vilnius city (villages Kazbiejai and Melagenai) also at Taurage and Klaipeda (Lithuania). The selected objects areas, flight data, time and height, processing images number are shown in the Table 2.

The first two selected areas comprise buildings, roads, cultivated fields and natural vegetation. Next two test flights were made at urban city area and had multi-storey buildings. The flight height was 200-350 m. Consequently, there are 1:5000 scale images.

4. Photogrammetric software for images processing

The images were processed by two photogrammetric software Pix4d Mapper (Switzerland) and PhotoMod (Russia). The results are further described.

Pix4d Mapper has function, which automatically shows image center at Google Map. There is a need just upload images to Pix4d Mapper. Images center location by GPS provides information of working area. Useless images can be deleted. Selected images can be processing. The images processes are shown at Fig. 1. There are 3 main processing steps [7, 8].

Fig. 1Image processing by Pix4d Mapper

During initial processing (Step 1) digital photo camera is calibrated and images are orientated equally. Compact cameras are extremely sensitive to temperature differences, vibrations and shocks and these elements require a complete calibration for each flight. Pix4D software includes a powerful camera auto-calibration algorithm that takes the full information of each pixel of your images to estimate the optimal camera and lens calibration for each flight. This feature is critical to ensure perfect accuracy at any climate condition, without any manual and tedious user intervention involving checkerboard patterns that can be error prone steps. Pix4d Mapper software is initiate self-calibration, which calculate photo camera focal length $c_k$, principal point location ($x_0$, $y_0$) and radial, tangential distortions ($K_1$, $K_2$, $K_3$, $T_1$, $T_2$). Results of camera Canon S100 self-calibration parameters processing images of all test areas objects are in the Table 3.

The focal length is a property of the camera, its sensor and optic. It varies with temperature, shocks, altitude and time. The calibration process starts with parameters of an initial camera model and optimization of parameters with respect to the images. It is normal that the focal length is slightly different for each project. An initial camera model should be within 5 % of the optimized value to ensure a fast and robust optimization [9]. Test flight data (Table 3) shows that in the Taurage object focal length is different from other object. Probably the weather conditions made biggest impact on results. All the project results are valued and the mean lens focal length is calculated $c_k=$5.732 mm (in this case).

Sometimes not all the images can be calibrated due to low key points at over covering area. There have to be more than 2000 joint key points for successful camera (images) calibration [10]. If there is forested area, there is a need to increase the frontal overlap area to 80-95 %. This must be done to increasing joint key points number.

There are results of key points and matching points number in the test flights (Table 4).

All the images of the villages Kazbiejai and Melagenai, and Klaipeda city have been calibrated. Calibrated images had from 6000 to 15 000 matching points. 10 images of Taurage city were not calibrated due to low matching point number. Ortho-mosaic preview shows not rendered (white places) orthophoto area (Fig. 2).

Fig. 2Preview ortho-mosaic before densification

To correct this failure additional flight should be performed over the area and pictures with bigger longitudinal coverage taken.

Exterior image orientation. Exterior orientation in photogrammetry systems includes images interior and relative orientation. There are measured principal points during interior orientation process and these points location is determined during self-calibration (Table 3). In Pix4d Mapper all of these steps are operated automatically without user interference. The process of relative orientation is following [10]:

• Measuring of the tie points on stereo pairs in overlapping areas and triplet zones (if we have three images) manually or automatically;

• Input and measurement of ground control points (GCP). Majority of software recognize GCP points, but some software including PhotoMod cannot generate orthographic model without GCPs. Pix4d Mapper generates both of them. No GCP were entered. That means the geo-location of the project is done by averaging the position of the optimized geo-tags. GPS devices used for geo-tagging of original images may suffer from a global shift, leading to a global shift in the project of several meters.

The difference between the image geo tags (GPS) and the optimized camera positions in the test areas are showed in Table 5.

Data of Table 5 show that proceeding exterior image orientation without GCP key points gives accuracy 2-15 m of ortho-mosaic. In Fig. 3 the example of matching points and optimized camera position (green points) in village Kazbiejai are shown.

Fig. 3Identified matching point cloud and optimized camera position

When relative orientation is accomplished, triangulation process follows. Triangulation is a process when tie (matching points) points are transformed to geodetic coordinates system. All these photogrammetric work stages are performed automatically in the Pix4d Mapper software. There is no need to be familiar with basics of photogrammetry to do triangulation calculations.

Next image processing step (2) is to determine point cloud densification which means filtering and smoothing of point clouds (Fig. 2). The generation of point cloud can lead to noisy and erroneous points. The noise filtering algorithm corrects the altitude of these points with the median altitude of the neighboring points. The noise filtering radius defines the size of the neighborhood to be considered. For a noisy point, only the points within the distance of radius will be used to correct this point. For example for a project with a 5 cm/pixel GSD (Ground Sampling Distance), a radius of 20 GSD units represents a distance of 1m. This means that points further than 1 meter of the noisy point will not influence its correction. The GSD is the distance between pixel centers measured on the ground in these test projects shown in the Table 6.

The bigger the value of image GSD is, the lower the spatial resolution of image is and less details are visible in the ortho-mosaic. The GSD is related to the flight height: the higher the altitude of the images, the bigger the GSD value. Even when flying at a constant height, the images of a same project cannot have the same GSD. This is due to the differences in elevation of the terrain and the inclinations of the camera while shooting. The final ortho-photo corrects this issue and is created using one unique GSD.

Once the noise filter has been applied, a surface is generated from the point cloud. This surface can contain areas with erroneous small bumps. The surface smoothing algorithm corrects these areas by flattening them. It is important to understand that the surface smoothing deals both with the points and the surface generated from the points, whereas the noise filtering deals with points only.

DSM and ortho-mosaic generation contains five sections:

• Raster DSM: allows to select the output file format for the raster DSM;

• Grid DSM: allows to select the output file format for the vector DSM;

• Ortho-mosaic: allows to select the output file format for the ortho-mosaic as well as different options related to the ortho-mosaic generation;

• Triangle model: allows generate the Triangle Model while processing step 3. DSM and Ortho-mosaic Generation;

• Set as default options: option deactivated by default. When this option is deactivated the changes in one project will not apply to future projects. Clicking on this checkbox, options of this tab will be used as default options for future projects.

Fig. 4Elevation view and ortho-mosaic of village Kazbiejai project

Ortho-mosaic removes the perspective distortions from the images using the 3D model and then blends the ortho-rectified images together. A high number of matches/key points (more than 1000) is required to generate the 3D model. This method handles any type of terrains as well as large datasets. Geo-reference is well supported. Distances are preserved and therefore ortho-rectified images can be used for measurements purposes. Pix4D’s advanced algorithms allow the computation of a 3D point and elevation for up to every pixel of the original images. This feature enhances the accuracy of DSMs and provides true ortho-rectification to the mosaics.

Once the project has been processed, it is possible to use the results using the Mosaic Editor. The Mosaic Editor allows to (Figs. 4, 5):

• Visualize the DSM (raster GeoTIFF Digital Surface Model);

• Improve the visual aspect of the ortho-mosaic.

The ray Cloud Editor allows to:

• Visualize the different elements of the reconstruction: camera positions, point clouds, GCPs, automatic tie points, etc.

• Verify the accuracy of the reconstruction;

• Improve the visual aspect of the point cloud;

• Measure distances, surfaces and stockpiles.

Fig. 5Ortho-mosaic of village Melagenai project

Improve the quality of ortho-mosaic in an easy and fast way. The Pix4d Mapper can remove moving objects by editing seam lines and selecting the most appropriate images. Seam lines and 3D views are shown in the Figs. 6, 7.

Fig. 6Seam line view

Fig. 73D view

Output result format is very important for next software to read the data [8, 10]. 2D and 3D output results easily readable by standard GIS (Geo TIFF, point cloud in LAS, LAZ, XYZ, PLY format), CAD and Photogrammetry software packages and Google tiles export in KML output format. Also object can be export in DXF, SHP and KML formats.

Pix4d Mapper software automatically generates an easy to use “5 Point Checklist” after initial processing that lets instantly assess the quality of the project. With just a quick glance, optimal results can be obtained by following the tips proposed in the “Help” feature of the report. The report displays scores of image, dataset, matching and geo-referencing quality, as well as an overview of the ortho-mosaic and DSM. It also provides detailed, quantitative measurements of the AAT, BBA and ground control points accuracy.

PhotoMod. To upload images to Google Maps a special function is needed. PhotoMod does not have such a function. Some other software like Digital Photo Professional has to be used. This software helped to select images of flight projects (Table 2). The images of village Kazbiejai project which were selected and processed with photogrammetric software PhotoMod are shown in Fig. 8.

Fig. 8Selected project images by PhotoMod

Majority photo-geometrical determinations have been done, in manual or semiautomatic mode, so user had to had good knowledge in photogrammetric basics. The images processing stages are shown in Fig. 9 [11].

Fig. 9PhotoMod image processing

Calibration parameters are entered to PhotoMod manually (photo camera have to be calibrated before) meanwhile Pix4d Mapper has the self-calibration system. Photo camera Canon S100 was calibrated in 2012 by the special TCC software (Germany) according to the calibration stand-cube [5, 6]. The test field – cube was photographed from different position: right, center and left. The focal length was fixed at the widest angle with the focus attached to infinite. The following distortions – parameters of the camera were determined: $c_k$ – the distance of the focal length; $s_{xy}$ – the coefficient of the image scale; $x_h$ – the correction of the principle image point of the coordinate $x_0$; $y_h$ – the correction of the principal image point of the coordinate $y_0$; $K_1$, $K_2$ – a camera lens radial – symmetric distortion; $T_1$ – a camera lens radial – asymmetric distortion. The obtained results are presented in Table 7.

The accuracy of the camera calibration is characterized by the standardized deflection ($\delta$) of the weight unit, which indicates the accuracy of the identification and measurement of the points on the calibration stand. Theoretically, this value does not have to exceed 15 µm.

After calibration of camera Canon S100, the standard deflection of the weight unit was 3.16 µm; the acquired result is very good and acceptable. The size of the digital matrix of the image taken by the camera Canon S100 is 7.44×5.58 mm, thus, the smallest size of the image point (pixel) is 0.00186 mm. The distance of the focal length is $c_k=$2882.323 pix (5.361 mm). A small distance of the focal length indicate, that the camera has a wide field of view, therefore, when taking images with this camera a large area of the location could be covered. A difference of focal length of photo camera Canon S100 was got from different softwares is 0.371 mm.

Proceeding images interior orientations PhotoMod manually measure principal points, which coordinates is half of all the photo matrix size (2000×1500 pix). Principal point location is corrected automatically for photo camera calibration parameters. Next Tie and GCP points are measured. Without GCP points PhotoMod cannot generate ortho-photo mosaic, though Pix4D can do it. Tie and GCP points at this test project were measured manually. There were 100 Tie and 11 GCP points. GCP points were selected at 1:5000 scale orthographical map (www.geoportal.lt). There was determining rectangular coordinates with 1-5 m accuracy. Ortho-photo mosaic should be more accurate if GCP coordinates be determined by geodetic method. Geometric model used in village Kazbiejai and city Taurage projects are show at Fig. 10.

Fig. 10Geometric model of village Kazbiejai object and city Taurage

When relative orientation was made Tie point’s transformation to geodetic coordinate systems was started. Result of point transformation is shown in Fig. 11.

Fig. 11Result of point transformation

Root mean square of GCP points are 1.5 m and Tie points – 0.5 m. In case accuracy is sufficient and satisfies the requirements, it is possible to generate ortho-mosaic (Fig. 12.), surface model DTM and later according to this data render topographical map. These processes require some special modules with semi-automatic functions in PhotoMod.

Fig. 12Ortho-mosaic made by PhotoMod

a) Project of village Kazbiejai

b) Project of city Taurage

All output data can be exported to a selection of popular exchange formats. Ortho-mosaic may be saved into the following formats: BMP, DGN, ERDAS Img, GeoTIFF, JPEG, NITF, PCIDSK, PNG, TIFF. Result of comparison between two analyzed softwares was shown in the Table 8.

5. Conclusions

The images acquired by UAVs are suitable for proceeding by different software packages: software like Pix4d Mapper special used for UAV image processing and PhotoMod photogrammetric software were analyzed in this paper.

The most important differences between software’s are the following:

1) Photo camera mounted on the UAV has to be calibrated. Camera implemented can be calibrated in laboratory conditions or images can be calibrated during processing. Pix4d Mapper software has some advantages, it has self-calibration and there is no need to worry about cameras parameters before taking images. PhotoMod software does not have such function, so photo camera parameters have to be known. These parameters have to be entered to the software first of all.

2) Images got from the UAV have GPS coordinates of image center. These coordinates allow uploading images to Google Map. There is special software to do that but Pix4d Mapper also has that function.

3) Image orientation is fully automated in Pix4d Mapper. User may even be unaware of photogrammetry processes running inside the program. Meanwhile in PhotoMod software some processes have to be done manually or semi automatically, for example image center fixation, so user have to know photogrammetry basics very well.

4) Pix4d Mapper generates ortho-mosaic automatically, meanwhile in PhotoMod user has to select images, define area, enter data semi automatically. The time consumption for these processes is very different.

5) PhotoMod software can render topographical map manually and in stereo mode. User sees object data in 3D at the computer screen. Pix4d Mapper does not have that function. Topographical map made in stereo mode is more accurate.

6) Pix4d Mapper software can process qualitatively huge amount of images. It is difficult to do that with specialized software because photogrammetry needs high quality images. So if there are lots of images it is better to use specialized UAV processing software instead of just specialized photogrammetry software.