Seminar: Machine Learning Seminar
RL in the Wild: Risk, Robustness and Awareness
Applying reinforcement learning (RL) to real-world problems brings up significant challenges. In this talk, we will cover two of these challenges.
First, real-world applications are often sensitive to risks and uncertainties. Risk aversion is often handled by optimization of a risk measure of the returns, instead of their expectation. We will discuss the surprising challenges raised by this mere change of objective, and will consider solutions. Another key requirement in real-world systems is performance awareness: if an autonomous vehicle begins to falter, for example, it is critical to alert to the misbehavior before any harm is done. A primary challenge is the non-i.i.d nature of the reward feedback. Indeed, common solutions bypass this problem by only considering the average reward. We will discuss the loss of information in this approach, and will show that breaking down the reward structure may increase the statistical power by orders of magnitude. |
Ido Greenberg is a PhD candidate at the Technion under the supervision of Prof. Shie Mannor. His PhD focuses on making the extraordinary achievements in the RL literature more applicable to real-world problems. His research includes fundamental works about risk-aversion and awareness in RL, as well as practical applications for conversational planning (at Google) and NP-hard routing problems (at Nvidia). His research interests also include medical and biological forecasting and filtering.
|