Semantic & Dynamic SLAM

Semantic & Dynamic SLAM

Published: by

Semantic & Dynamic SLAM


We developed a method for real-time semantic understanding and 3D mapping in both static and dynamic environments, and demonstrate that the two parts can benefit each other. Semantic mapping has been used for many applications such as autonomous robots and AR/VR. However, most existing approaches treat semantic understanding and 3D mapping as separate problems. In addition, dynamic objects are often treated as outliers in the SLAM mapping of current approaches.

In this work, we first propose a consistent semantic 3D mapping system. The grid labels are jointly optimized in 3D space over multiple views instead of treating each frame independently. Secondly, we jointly optimize the camera’s ego-motion and the dynamic objects’ trajectories over time. Multi-view bundle adjustment with new object measurements is proposed to jointly optimize poses of cameras, objects and points.

Dynamic SLAM
Fig 1: Dynamic SLAM

Instead of treating dynamic objects as an outlier, we proposed to track the dynamic object in the scene and estimate the motion model of the dynamic object. By tracking the motion at object level, we can utilize the feature points on dynamic objects. In this method, the camera pose is partially constrained by dynamic points and dynamic objects. The rigid body assumption indicates that a dynamic point’s position on an object doesn’t change over time shown as the red diamonds in Figure 1. This allows us to utilize the standard 3D map point re-projection error to optimize the position shown as the black squares in the graph. The second assumption is that the object motion model is constrained to be physically feasible.

Project Members

  • Wenshan Wang
  • Chen Wang
  • Yafei Hu
  • Yuheng Qiu
SLAM

Latest Research

SubT-MRS: Pushing SLAM Towards All-weather Environments
SubT-MRS: Pushing SLAM Towards All-weather Environments

Simultaneous localization and mapping (SLAM) is a fundamental task for numerous applications such...

TartanDrive 2.0: More Modalities and Better Infrastructure to Further Self-Supervised Learning Research in Off-Road Driving Tasks
TartanDrive 2.0: More Modalities and Better Infrastructure to Further Self-Supervised Learning Research in Off-Road Driving Tasks

TartanAviation: Image, Speech, and Trajectory Datasets for Terminal Airspace Operations
TartanAviation: Image, Speech, and Trajectory Datasets for Terminal Airspace Operations

We introduce TartanAviation, an open-source multi-modal dataset focused on terminal-area airspace...