Designing non-parametric activation functions: recent advances

Title

Designing non-parametric activation functions: recent advances

Speaker

Simone Scardapane - Sapienza University of Rome

Abstract

Recently, the design of flexible nonlinearities has become an important line of research in the deep learning community. In the first part of the talk we will review how to tackle this problem, both in the context of simple parameterizations of known functions (e.g., the parametric ReLU), and with the definition of more advanced, non-parametric models (e.g., the Maxout network). The second part of the talk will focus on a recent proposal, the kernel activation function, which is based on an kernel expansion of its input. We will show its core idea and some recent extensions, involving its use in the context of other types of nonlinearities, such as gates (as in LSTMs), and attention models. The talk is concluded with some open challenges and possible lines of research.

Bio

Simone Scardapane is an assistant professor at Sapienza, where he was previously a post-doctoral fellow, with a focus on deep learning. Previously, he was a fellow researcher at Stirling University (UK) and a visiting student in La Trobe University in Melbourne. He also has a strong interest in promoting machine learning in Italy. He is a co-founder and chairman of the Italian Association for Machine Learning, co-organizer of the Rome Machine Learning and Data Science Meetup, and a current Google Developer Expert for Machine Learning.

When

2019-04-17 at 3:00 pm

Where

Genova