Towards Safe Reinforcement Learning

Title

Towards Safe Reinforcement Learning

Speaker

Andreas Krause - ETH Zürich

Abstract

More and more machine learning systems make data-driven decisions in the real work, in increasingly higher-stakes applications. This has caused substantial interest in reinforcement learning -- the field of learning to make decisions from data -- which has seen stunning recent empirical breakthroughs. At its heart is the challenge of trading exploration -- collecting data for learning better models -- and exploitation -- using the estimate to make decisions. In many applications, however, exploration is a potentially dangerous proposition, as it requires experimenting with actions that have unknown consequences. Hence, most prior work has confined exploration to simulated environments. In this talk, I will present our work towards rigorously reasoning about safety of exploration in reinforcement learning. I will discuss a model-free approach, where we seek to optimize an unknown reward function subject to unknown constraints. Both reward and constraints are revealed through noisy experiments, and safety requires that no infeasible action is chosen at any point. I will also discuss model-based approaches, where we learn about system dynamics through exploration, yet need to verify safety of the estimated policy. Our approaches use Bayesian inference over the objective, constraints and dynamics, and -- under some regularity conditions -- are guaranteed to be both safe and complete, i.e., converge to a natural notion of reachable optimum. I will also show experiments on safely tuning cyber-physical systems in a data-driven manner.

Bio

Andreas Krause is a Professor of Computer Science at ETH Zürich, where he leads the Learning & Adaptive Systems Group. He also serves as Academic Co-Director of the Swiss Data Science Center. Before that he was an Assistant Professor of Computer Science at Caltech. He received his Ph.D. in Computer Science from Carnegie Mellon University (2008) and his Diplom in Computer Science and Mathematics from the Technical University of Munich, Germany (2004). He is a Microsoft Research Faculty Fellow and a Kavli Frontiers Fellow of the US National Academy of Sciences. He received ERC Starting Investigator and ERC Consolidator grants, the Deutscher Mustererkennungspreis, an NSF CAREER award, the Okawa Foundation Research Grant recognizing top young researchers in telecommunications as well as the ETH Golden Owl teaching award. His research on machine learning and adaptive systems has received awards at several premier conferences and journals. Andreas Krause served as Program Chair for ICML 2018, and is serving as Action Editor for the Journal of Machine Learning Research.

When

2020-02-04 at 3:00 pm (subject to variability)

Where

Genova