Autonomous UAV-based Multi-Model High-Resolution Reconstruction for Aging Bridge Inspection

Autonomous UAV-based Multi-Model High-Resolution Reconstruction for Aging Bridge Inspection

Published: by
Weikun Zhen

Autonomous UAV-based Multi-Model High-Resolution Reconstruction for Aging Bridge Inspection

The conventional way of aging infrastructure (e.g. bridges and tunnels) inspection can be time-consuming and dangerous for humans. Therefore we are developing technologies to let UAVs help collect, process and analyze data automatically.

A customized DJI M210 drone carrying the designed payload, which contains stereo pair, rotating LiDAR, IMU and thermal camera.

To achieve autonomy, we build UAVs with customized sensing payload (cameras, LiDAR, IMU, thermal sensors, etc.). Visual and LiDAR-based SLAM methods are applied to achieve real-time robust state estimation. Coverage planning algorithms ensure visual coverage of the surface being inspected.

LiDAR-enhanced Structure-from-Motion: We augmented the traditional Structure-from-Motion pipeline with LiDAR information. Observations from both sensors are fused into a single cost function and optimize for all parameters simultaneously. Take a look at the reconstructed model of the Pearl Harbor Memorial Bridge Tunnel in Connecticut (link).

DL aided Stereo: To improve the robustness of stereo matching algorithms in low textured circumstances, we trained a deep neural network to predict the disparity uncertainity, which provide a guide for correspondance searching.

2D-3D Localization: We proposed a global 2D-3D registration method to estimate 2D-3D line correspondences and the camera pose in untextured point clouds of structured environments.

On the inspection and analysis side, we are collaborating with Professor Kenji Shimada and Kris Kitani for FEA and image-based crack detection.

This project is sponsored by Shimizu Institute of Technology in Tokyo, Japan.

Publications

Zhen, W., Hu, Y., Liu, J. and Scherer, S.. A joint optimization approach of lidar-camera fusion for accurate dense 3-d reconstructions. IEEE Robotics and Automation Letters, 4(4), pp.3585-3592. [PDF] [Video]

Zhen, W., Hu, Y., Yu, H. and Scherer, S.. LiDAR Enhanced Structure-from-Motion. To appear in 2020 International Conference on Robotics and Automation (ICRA). IEEE. [PDF] [Video]

Yu, H., Zhen, W., Yang, W. and Scherer, S.. Line-based Camera Pose Estimation in Point Cloud of Structured Environments. arXiv preprint arXiv:1912.05013. [PDF] [Video]

Hu, Y., Zhen, W. and Scherer, S.. Deep-Learning Assisted High-Resolution Binocular Stereo Depth Reconstruction. To appear in 2020 International Conference on Robotics and Automation (ICRA). IEEE. [PDF] [Video]