Towards automatic detection of road features with deep learning

In Japan, a substantial amount of point cloud data–a set of data points in space–has been measured and accumulated for public works using mobile mapping systems and terrestrial laser scanners. However, this vast amount of data is of limited use in an unprocessed and unstructured state. Fortunately, it can be structured by automatically extracting a feature using a "plan of completion drawing" that shows the completed geometry of a construction object.

Earlier this year, researchers from Japan, led by Professor Ryuichi Imai of Hosei University, Japan proposed another method for extracting road features using high-precision 3D (HD) map data. However, the applicability of their approach is limited to the developed sections of road maps. While the issue can be solved with deep learning-based identification, they require a large amount of manually-prepared, high-quality training data.

Recently, Prof. Imai and his collaborators, Kenji Nakamura of Osaka University of Economics, Yoshinori Tsukada of Setsunan University, Noriko Aso of Dynamic Map Platform, and Jin Yamamoto of Hosei University developed an algorithm to automate the process of training data generation and constructed a road feature identification model from point cloud data extracted automatically from HD maps.

"Currently, people need to visually check the point cloud data to identify road features as computers cannot recognize them. But with our proposed method, the feature extraction can be done automatically, including the features at undeveloped road map sections," explains Prof. Imai. Their work was presented at the Joint 12th International Conference on Soft Computing and Intelligent Systems and 23rd International Symposium on Advanced Intelligent Systems on November 29, 2022.

In their study, the researchers first separated the ground surface from the point cloud data using the CloudCompare software. Next, they generated area data from the HD map and extracted component points of features. While these points were assigned as either road signs or traffic lights, other labels were provided for the remaining data.

Then, the area data corresponding to the component points was extended to generate the training data. Using this, the researchers further generated the point cloud projection images. Lastly, they used the training data to construct the identification model using a YOLOv3 object-detection algorithm. The model could detect road features based on clustering points other than those identified for the ground surface using CloudCompare.

Having established the computational framework, the researchers performed demonstration experiments in the Shizuoka Prefecture on a road with 65 road signs, 46 traffic lights, and noise features over a distance of 1.5 kilometers. They used 258 road signs and 168 traffic lights to train their identification model and used 36 and 24 images, respectively, to calculate the algorithm determination accuracy.

The researchers found that the precision, recall, and F-measure were 0.84, 0.75, and 0.79, respectively, for the road signs and 1.00, 0.75, and 0.86, respectively for the traffic lights, indicating zero false determinations. The precision of the proposed model was shown to be higher than the existing models.

Prof. Imai concludes by highlighting the future implications of the work. "A product model constructed from point cloud data will enable the realization of a digital twin environment for urban space with regularly updated road maps. It will be indispensable for managing and reducing traffic restrictions and road closures during road inspections. The technology is expected to reduce time costs for people using roads, cities, and other infrastructures in their daily lives."