Road Surface Inspection System

As social infrastructure ages, its maintenance and management are becoming increasingly important. Consequently, they are expected to require more frequent inspections in the years to come.
Roads are an integral part of the social infrastructure. Keeping them safe as they age requires proper inspection of all essential elements—the road surfaces, tunnels, and roadside slopes. Ricoh has automated the inspection systems by combining its original optical systems with AI digital technologies, offering cost-effective, efficient inspection solutions.

Background

Roads continue to deteriorate on a daily basis because of the wear and tear associated with the variety of vehicles driven on them. The deterioration of road surface quality impairs not only the comfort of drivers, but also impacts the economic efficiency.
To maintain roads as part of the social infrastructure, it is necessary to repair them at the right time.
At present, inspection of road conditions is mainly done using a specialist vehicle. However, this is expensive and regularly doesn't allow access to residential roads.

Solutions

Ricoh has developed a camera system that can measure the road surface conditions using a standard motor vehicle rather than a specialist vehicle. This system enables measurement on residential roads that have prior to now been difficult to measure with a specialist vehicle.
It is also possible to automate the work processes from capturing images to the creation of an inspection record.
With this system, we will aim to achieve safe and secure road environments through more efficient maintenance.

Technical highlights

1. Capturing images with stereo cameras

With an imaging system using multiple stereo cameras, the measurement of three factors is achieved: the rate of cracks, the rutting depth, and flatness (σ). A stereo camera detects depth information of a subject using the parallax of two cameras aligned left and right.

Camera system using multiple stereo cameras that can be mounted on a standard vehicle

Based on the results obtained by combining the images in the width direction that are taken with multiple stereo cameras, the rutting depth can be calculated.
Further, based on the results obtained by combining the images taken by the same multiple stereo cameras in the traveling direction, flatness can be measured.
Image combining enables not only a 3D combined image to be obtained, but also a combined brightness image to be generated by combining the feature points in images (brightness images). It is also possible to measure the rate of cracks using this combined brightness image.

In this way, using only the information obtained from the stereo cameras, the measurement of three factors can be achieved: rate of cracks, rutting depth, and flatness (amount of longitudinal unevenness).

2. Using artificial intelligence

Recently, much effort has been put into research and development of artificial intelligence (AI), enabling machines to have a learning function. Machine learning is one of the methods used in image recognition and analysis.

In machine learning, a computer receives certain inputs, learns from them, and predicts useful outputs. As a preliminary step, the system needs to have learned combinations of certain input images and corresponding correct-answer outputs (correct-answer labels). The learning step is followed by the decision step, where the system receives a certain image input and predicts and outputs the correct answer based on what it has learned.

By replacing the visual inspection process with the machine reading based on a model obtained through machine learning, man-hours can be reduced compared to conventional methods.

Grids with a length of 50 cm are automatically created, and the number of cracks is determined by AI

3. Visualization of results

Based on the three data sets obtained, the system calculates a Maintenance Control Index (MCI), which is a comprehensive index for the judgment of maintenance and repair. This supports the creation of an inspection record.
The system also visualizes the road surface condition by mapping the results.

Image of visualized paved status

Ricoh's vision

This new technology can also be applied to the maintenance of diverse social infrastructures, such as tunnels and bridges, in addition to the road surface.
In developing a system that monitors social infrastructure, Ricoh is striving to resolve a range of social issues, including aging infrastructure and the labor shortage and motor efficiency.

Related information

• Tunnel Monitoring System