MaLGa logoMaLGa black extendedMaLGa white extendedUniGe ¦ MaLGaUniGe ¦ MaLGaUniversita di Genova | MaLGaUniversita di Genova

Human Pose and Motion Understanding


dgt book

Computational models and cognitive science

We consider motion understanding tasks in the general domain of Cognitive Computer Vision (or bio-inspired Computer Vision). To this purpose, we design strategies characterized by a strong interplay between visual computation and cognitive science, to address tasks including motion detection and recognition, action categorization and anticipation. Furthermore, we are interested in the understanding of motion qualities that may be not directly visible, as the style, the emotional load, and the goals or intentions. We apply our research in particular to HRI.


Collaboration with


References

A Vignolo, N Noceti, F Rea, A Sciutti, F Odone, G Sandini "Detecting biological motion for human–robot interaction: A link between perception and action" Frontiers in Robotics and AI 4, 14, 2017


Marker-less motion analysis

We tackle marker-less motion analysis to describe the motion evolution in time and to provide a quantitative analysis of human behavior in supervised or unsupervised way. Our goal is to understand the quality of motion, derive information on functional impairments, possibly also assessing the benefits of a rehabilitation procedure. Long term objectives of our research are ecological, non invasive, unbiased motion analysis methods to be adopted in the clinical practice.We both consider full-body movements and gestures, and data provided by cameras, RGB-D sensors, and graphical tablets. We address different application: motion analysis in Multiple Sclerosis patients (with FISM) and Stroke survivors, General Movements analysis in premature infants (with Gaslini Hospital, Genova), analysis of motor learning in instruments players (with Conservatorio Nicolò Paganini, Marquette University, Music Institute of Chicago).


Collaboration with


Gaze estimation

We study methods for apparent gaze (or heading) estimation in video sequences containing multiple individuals. Our goal is to rely on multiple information gathered from the scene under analysis, starting from the outputs of 2D pose estimation methods. We address the challenges of occlusions and partial information with a methodology able to provide a gaze direction estimate associated with an uncertainty prediction.


Collaboration with


References

P A Dias, D Malafronte, H Medeiros and F Odone “Gaze Estimation for Assisted Living Environments” WACV 2020


Cross-view action recognition

Cross-view action recognition is a natural task for humans, while it is well known that view-point changes are a major challenge for computer vision algorithms, which have to deal with signal variations in geometry and overall appearance. To this end, we explore the appropriateness of deep learning approaches to implicitly learn view-invariant features, as well as other dynamic and appearance information. We also study the general transferability of a learnt model: how well an extensively learnt spatio-temporal representation fares when used for different types of action datasets, varying from full-body actions with large movements and variations to focused upper-body movements with subtle, fine grain difference in actions.


The MoCa project

The goal of the project is to acquire and maintain a multi-modal multi-view dataset in which we collect MoCap data and video sequences from multiple views of upper body actions in a cooking scenario. The acquisition has the specific purpose of investigating view-invariant action properties in both biological and artificial systems. Beside addressing classical action recognition tasks, the dataset enables research on different nuances of action understanding, from the segmentation of action primitives robust across different sensors and viewpoints, to the detection of actions categories depending on their dynamic evolution or the goal.


Collaboration with


References

The cooking dataset (on GitHub)