Computer Vision applications in road data collection

There are many applications of computer vision in road maintenance, which utilize the ability to detect visual defects automatically. The visual detections are beneficial especially in monitoring the vertical and horizontal civil infrastructure.

The common procedure is still by now, that the specialized inspectors are monitoring the road infrastructure in the regular basis, since the condition of the road infrastructure has a direct influence on the safety of the drivers. (Koch et al.

2015)

The computer vision is becoming increasingly important topic in road maintenance and the application has been increasing the whole first decade of 21^st century because of the low-cost and high-quality sensors, such as digital cameras. The main computer vision methods in road infrastructure detection are divided into the different categories: pre-processing method, feature-based method, model-based

method, pattern-based method and 3D reconstruction. The list of methods is not explicit, but the they are built on top of each other. (Koch et al. 2015)

The computer vision applications have already been succeeding. The bridge condition monitoring can be done effectively with computer vision based supervision. The defects can be localized, quantified and the changes can be recognized. In bridge monitoring the computer vision is fully comparable with the laser-scanner methods. Also considering the long horizontal civil infrastructure, such as paved roads, the respective data collection methods are fully automated. All the defects, for instance cracks, open joints and holes, can be already detected and mapped. The individual cracks can be measured and based on the measurement, some of the systems are already able to create the road condition index for the certain part of the road. (Koch et al. 2015)

The challenges of the computer vision roots to the accuracy of the detection and the influence of the recording conditions, the pose of the camera and so on. Moreover, the computer vision still requires quite much manual work and they fail to detect diverse geometries and material. Other challenges are caused by bad lighting conditions, different backgrounds of the images and the limited data quality and quantity; for example, not all the detects follow the same patterns and therefore the detection is never perfect. Though, if the detection is based on many parameters, it is more accurate, but it cannot be generalized in the different environments. The training of the computer vision is also error-sensitive, since it is still mostly manual work, demanding a great input from the labour. Because of that, the comprehensive and automated data collection from the road infrastructure is still under the development. (Koch et al. 2015)

For the reason that the computer vision still lacks the needed quality and is not able to comprehensively detect the defects in the civil infrastructure, Koch et al. have concluded, that visual condition monitoring is not sufficient for road condition monitoring, but the visual detection has to be complemented with the additional in-depth techniques, such as radar, magnetic or electrical technologies. The changing

environment set challenges for pure visual inspection and hinders its reliability.

Finally, the risks related to the visual inspections of the road infrastructure have to minimized with the safety data collection methods. And not only the risks of personal damages, but the risk correlates also with the privacy regulations, which do not allow that any personal information is saved in the collected material. (Koch et al. 2015)

Koch et al. (2015) concentrated in their article on the computer vision applications related to the concrete horizontal and vertical civil infrastructure but did not consider other road assets, such as traffic signs, guard rails and road markings.

Though, there exists diverse literature about the computer vision applications in each of those fields, but there is not found any comprehensive literature combining the current practices. For the traffic sign detection, there are developed many methods detecting the sign itself and different attributes. According to the summary of Hu and Li (2016) most of the computer vision methods are based on the classifier which is built by using the pre-determined features. For the road markings, there are, as well, multiple technical approaches to detect the defects in road markings (McCall, Trivedi 2006; Danescu, Nedevschi 2010), but according to the Soilan et al (2017), the problems are same as with the detection of concrete defects and that requires other complementary data sources to generate reliable data.

4.7 Summary

The road infrastructure is recognized as one of the most important assets in the society today. To keep the roads in order, it is essential to have the up-to-date information about the whole road network. The constant awareness of the road condition enables the decision makers to effectively allocate the scarce resources to the targets, which requires urgently the repairing. The road data collection is still mostly manual work done by specialized road inspectors.

In Germany, the road infrastructure was built after the second world war and was planned to carry much less traffic compared to the current situation. Therefore, the

road network is getting worse and causing severe safety risks and force the road authorities to lower speed limits. One of the biggest challenges in Germany is to generate objective estimations about the road condition of the national road network, since the condition is evaluated by the inspectors and due to the subjective bias, the results are not comparable. The Committee of the road and Transportation Research Association (FGSV) in Germany is aiming to conduct guidelines for road inspections and how the roads should be monitored and what features should be documented. However, FGSV’s guidelines are only recommendation and they do not force the road authorities to follow the certain procedures. FGSV has been studying the information technology for the road maintenance. The breakthrough of information technology in road maintenance has not been experienced in Germany, but the new technology is already assisting the road inspectors.

Worldwide there are multiple applications of computer vision in detection of road infrastructure defects. Computer vision does not require expensive hardware but cheap digital cameras are sufficient. However, the computer vision is suffering from the reliability, since the detections are built upon the certain parameters and all defects do not follow the same parameters. Moreover, the changing recording condition might not match with the parameters which applied in the first place. The training of the computer vision is also error-sensitive and finally the authors concluded that solely computer vision is not reliable technique to detect the road damages, but it requires complementary sensor data.