Outline:

• Overview of SLAM
• Learning-based methods for SLAM
• How do we handle the hard cases in SLAM? What are the challenges ahead?
• Bridging the gap between dataset validation and real-world system deployment

Factor graphs have become a popular tool for modeling robot perception problems. Not only can they model the bipartite relationship between sensor measurements and variables of interest for inference, but they have also been instrumental in devising novel inference algorithms that exploit the spatial and temporal structure inherent in these problems.

I will start with a brief history of these inference algorithms and relevant applications. I will then discuss open challenges in particular related to robustness from the inference perspective and discuss some recent steps towards more robust perception algorithms.

In this session, Prof. Michael Milford discusses five key questions and open research areas in the bio-inspired mapping, navigation, and SLAM area, linking into past and recent neuroscience and biological discoveries:

• The Loop Closure Question
• The 3D Question
• The Probabilistic Question
• The Multi-Scale Question
• The Behavioural Question

Prof. Michael also presents an objective take on opportunities and mysteries in the area that also recognizes the practical realities and requirements of modern-day SLAM applications.

To enable the next generation of smart robots and devices which can truly interact with their environments, Simultaneous Localisation and Mapping (SLAM) will progressively develop into a general real-time geometric and semantic 'Spatial AI' perception capability.

Andrew will give many examples from their work on gradually increasing visual SLAM capability over the years. However, much research must still be done to achieve true Spatial AI performance. A key issue is how estimation and machine learning components can be used and trained together as we continue to search for the best long-term scene representations to enable intelligent interaction. Further, to enable the performance and efficiency required by real products, computer vision algorithms must be developed together with the sensors and processors which form full systems, and Andrew will cover research on vision algorithms for non-standard visual sensors and graph-based computing architectures.

Enabling centimeter-accuracy positioning and human-like scene understanding for autonomous vehicles and mobile devices, holds potentially huge implications for practical applications. Optimal fusion of visual and inertial sensors provides a popular means of navigating in 3D, in part because of their complementary sensing modalities and their reduced cost and size.

In this talk, I will present our recent research efforts on visual-inertial estimation and perception. I will first discuss the observability-based methodology for consistent state estimation in the context of simultaneous localization and mapping (SLAM) and visual-inertial navigation system (VINS), and then will highlight some of our recent results on visual-inertial estimation, including OpenVINS, inertial preintegration for graph-based VINS, robocentric visual-inertial odometry, Schmidt-EKF for visual-inertial SLAM with deep loop closures, visual-inertial moving object tracking and many others.

Spatial perception has witnessed unprecedented progress in the last decade. Robots are now able to detect objects and create large-scale maps of an unknown environment, which are crucial capabilities for navigation and manipulation. Despite these advances, both researchers and practitioners are well aware of the brittleness of current perception systems, and a large gap still separates robot and human perception. For instance, while humans are able to quickly grasp both geometric and semantic aspects of a scene, high-level scene understanding remains a challenge for robotics.

This talk discusses recent efforts targeted at bridging this gap. I present our recent work on high-level scene understanding and hierarchical representations, including Kimera and 3D Dynamic Scene Graphs, and discuss their potential impact on planning and decision-making, human-robot interaction, long-term autonomy, and scene prediction. The creation of a 3D Dynamic Scene Graph requires a variety of algorithms, ranging from model-based estimation to deep learning, and offers new opportunities to both researchers and practitioners. Similar to the role played by occupancy grid maps or landmark-based maps in the past, 3D Dynamic Scene Graphs offer a new, general, and powerful representation, and the grand challenge of designing Spatial Perception Engines that can estimate 3D Scene Graphs in real-time from sensor data has the potential to spark new research ideas and can push the community outside the “SLAM comfort zone”.

Over the last four decades, research in SLAM and spatial AI has revolved around the question of map "representation". Where "classical" techniques in the SLAM community have focused on building general-purpose---but handcrafted---representations; modern gradient-based learning techniques have focused on building representations specialized to a set of downstream tasks of interest. Krishna postulates that a flexible blend of "classical" and learned methods is the most promising path to developing flexible, interpretable, and actionable models of the world: a necessity for intelligent embodied agents.

In this talk, Krishna will present two recent research efforts that tightly integrate spatial representations with gradient based learning.

• 1. GradSLAM - a fully differentiable dense SLAM system that can be plugged as a "layer" into neural networks
• 2. Taskography - a differentiable sparsification mechanism to build relational abstractions for enabling efficient task planning over large 3D scene graphs