Autonomous UAV-based Multi-Model High-Resolution Reconstruction for Aging Bridge Inspection

Autonomous UAV-based Multi-Model High-Resolution Reconstruction for Aging Bridge Inspection

Published: by

Autonomous UAV-based Multi-Model High-Resolution Reconstruction for Aging Bridge Inspection

The conventional way of aging infrastructure (e.g. bridges and tunnels) inspection can be time-consuming and dangerous for humans. Therefore we are developing technologies to let UAVs help collect, process and analyze data automatically.

A customized DJI M210 drone carrying the designed payload, which contains stereo pair, rotating LiDAR, IMU and thermal camera.

To achieve autonomy, we build UAVs with customized sensing payload (cameras, LiDAR, IMU, thermal sensors, etc.). Visual and LiDAR-based SLAM methods are applied to achieve real-time robust state estimation. Coverage planning algorithms ensure visual coverage of the surface being inspected.

LiDAR-enhanced Structure-from-Motion: We augmented the traditional Structure-from-Motion pipeline with LiDAR information. Observations from both sensors are fused into a single cost function and optimize for all parameters simultaneously. Take a look at the reconstructed model of the Pearl Harbor Memorial Bridge Tunnel in Connecticut (link).

DL aided Stereo: To improve the robustness of stereo matching algorithms in low textured circumstances, we trained a deep neural network to predict the disparity uncertainity, which provide a guide for correspondance searching.

2D-3D Localization: We proposed a global 2D-3D registration method to estimate 2D-3D line correspondences and the camera pose in untextured point clouds of structured environments.

On the inspection and analysis side, we are collaborating with Professor Kenji Shimada and Kris Kitani for FEA and image-based crack detection.

This project is sponsored by Shimizu Institute of Technology in Tokyo, Japan.

Publications

Zhen, W., Hu, Y., Liu, J. and Scherer, S.. A joint optimization approach of lidar-camera fusion for accurate dense 3-d reconstructions. IEEE Robotics and Automation Letters, 4(4), pp.3585-3592. [PDF] [Video]

Zhen, W., Hu, Y., Yu, H. and Scherer, S.. LiDAR Enhanced Structure-from-Motion. To appear in 2020 International Conference on Robotics and Automation (ICRA). IEEE. [PDF] [Video]

Yu, H., Zhen, W., Yang, W. and Scherer, S.. Line-based Camera Pose Estimation in Point Cloud of Structured Environments. arXiv preprint arXiv:1912.05013. [PDF] [Video]

Hu, Y., Zhen, W. and Scherer, S.. Deep-Learning Assisted High-Resolution Binocular Stereo Depth Reconstruction. To appear in 2020 International Conference on Robotics and Automation (ICRA). IEEE. [PDF] [Video]

Perception

Latest Research

MapEx: Indoor Structure Exploration with Probabilistic Information Gain from Global Map Predictions
MapEx: Indoor Structure Exploration with Probabilistic Information Gain from Global Map Predictions

We propose MapEx, a new exploration framework that uses map predictions to form probabilistic sen...

AirIO: Learning Inertial Odometry with Enhanced IMU Feature Observability
AirIO: Learning Inertial Odometry with Enhanced IMU Feature Observability

The key for deploying learning-based IO on UAVs is the input coordinate frame.

Flying Calligrapher: Contact-Aware Motion and Force Planning and Control for Aerial Manipulation
Flying Calligrapher: Contact-Aware Motion and Force Planning and Control for Aerial Manipulation

Flying calligrapher enables precise hybrid motion and contact force control for an aerial manipul...