Autonomous Aerial Cinematography

Aerial cinematography is revolutionizing industries that require live and dynamic camera viewpoints such as entertainment, sports, and security. However, safely piloting a drone while filming a moving target in the presence of obstacles is immensely taxing, often requiring multiple expert human operators.

Commercial drone products can tackle some automated tasks, but one thing those systems don’t address is filming artistically. Our team of researchers has proposed a complete system for aerial cinematography that learns humans’ visual preferences. The fully autonomous system does not require scripted scenes, GPS tags to localize targets or prior maps of the environment.

We address the problem in its entirety and propose a complete system for real‐time aerial cinematography that for the first time combines: (a) vision‐based target estimation; (b) 3D signed‐distance mapping for occlusion estimation; (c) efficient trajectory optimization for long time horizon camera motion; and (d) learning‐based artistic shot selection. We extensively evaluate our system both in simulation and in field experiments by filming dynamic targets moving through unstructured environments. Our results indicate that our system can operate reliably in the real world without restrictive assumptions.

Project Members

• Rogerio Bonatti
• Wenshan Wang
• Cherie Ho
• Aayush Ahuja
• Mirko Gaschwindt
• Efe Camci

Publications

• R. Bonatti, W. Wang, C. Ho, A. Ahuja, M. Gschwindt, E. Camci, E. Kayacan, S. Choudhury, S. Scherer, “Autonomous Aerial Cinematography Among Unstructured Environments With Learned Artistic Decision-Making”. Journal of Field Robotics, 2019 [Link][PDF][Video]
• R. Bonatti, H. Cherie, W. Wenshan, S. Choudhury, S. Scherer, “Towards a Robust Aerial Cinematography Platform: Localizing and Tracking Moving Targets in Unstructured Environments”. International Conference on Intelligent Robots and Systems (IROS), 2019 [PDF][Video]
• M. Gschwindt, E. Camci, R. Bonatti, W. Wenshan, S. Scherer, “Can a Robot Become a Movie Director? Learning Artistic Principles for Aerial Cinematography”. International Conference on Intelligent Robots and Systems (IROS), 2019 [PDF][Video]
• W. Wang, A. Ahuja, Y. Zhang, R. Bonatti, S. Scherer, “Improved Generalization of Heading Direction Estimation for Aerial Filming Using Semi-supervised Regression”. International Conference on Robotics and Automation (ICRA), 2019 [PDF][Video]
• R. Bonatti, Y.Zhang, S. Choudhury, W. Wang, S. Scherer, “Autonomous drone cinematographer: Using artistic principles to create smooth, safe, occlusion-free trajectories for aerial filming”. International Symposium on Experimental Robotics (ISER), 2018 [PDF][Video 1][Video 2][Video 3]
• Y.Zhang, W. Wang, R. Bonatti, D. Maturana, S. Scherer, “Integrating kinematics and environment context into deep inverse reinforcement learning for predicting off-road vehicle trajectories”. Conference on Robot Learning (CoRL), 2018 [PDF][Video]

Video Data

We release a few videos of the field experiments. These videos were taken from different stages of our tests containing multiple types of actors in various seasons.