MaLGa logoMaLGa black extendedMaLGa white extendedUniGe ¦ MaLGaUniGe ¦ MaLGaUniversita di Genova | MaLGaUniversita di Genova

3D scene and object understanding in the era of deep learning

dgt book


3D scene and object understanding in the era of deep learning


Federico Tombari


In the deep learning era, to address common 3D scene and object understanding applications we delegate a neural network the task of extracting feature representations from data and to learn intermediate subspaces and embeddings. This dangerously exposes our algorithms to known limitations of neural networks, in particular the need for huge amount of labeled data and the domain shift. In this talk, I will walk through current approaches for using deep learning for 3D reconstruction and 3D recognition tasks, and highlight novel directions to solve these limitations. In the first part of the talk, I will give an overview of methods aimed at monocular 6D object pose estimation and discuss how current techniques are tackling the problem of domain shift. In the second part of the talk, I will focus instead on 3D scene understanding and monocular 3D reconstruction, discussing the use of monocular depth prediction for monocular SLAM and related applications, as well as looking at approaches for unsupervised and self-supervised learning.


Federico Tombari is a research scientist and manager at Google and a lecturer at the Technical University of Munich (TUM). He has more than 180 refereed papers in the field of computer vision, machine learning and robotic perception. He got his PhD in 2009 from the University of Bologna, at the same institution he was Assistant Professor from 2013 to 2016. In 2008 and 2009 he was an intern and consultant at Willow Garage, California. Since 2014 he has been leading a team of PhD students at TUM on computer vision and deep learning. In 2017-2018, he was co-founder and managing director of Pointu3D Gmbh, a Munich-based startup on 3D perception for AR and robotics. He was the recipient of two Google Faculty Research Award (in 2015 and 2018) and an Amazon Research Award (in 2017). He has been a research partner, among others, of Google, Toyota, BMW, Audi, Amazon, Stanford and JHU. His works have been awarded at conferences and workshops such as 3DIMPVT'11, MICCAI'15, ECCV-R6D'16, AE-CAI'16, ISMAR '17


2020-05-05 at 3:00 pm