Seminar: Machine Learning Seminar
A Bayesian Approach to Online Planning
Date:
March,26,2025
Start Time:
11:30 - 12:30
Location:
506, Zisapel Building
Zoom:
Zoom link
Add to:
Lecturer:
Nir Greshler
Research Areas:
| The combination of Monte Carlo tree search and neural networks has revolutionized online planning. As neural network approximations are often imperfect, we ask whether uncertainty estimates about the network outputs could be used to improve planning. We develop a Bayesian planning approach that facilitates such uncertainty quantification, inspired by classical ideas from the meta-reasoning literature. We propose a Thompson sampling-based algorithm for searching the tree of possible actions, for which we prove the first (to our knowledge) finite time Bayesian regret bound and propose an efficient implementation for a restricted family of posterior distributions. In addition, we propose a variant of the Bayes-UCB method applied to trees. Empirically, we demonstrate that on the ProcGen Maze and Leaper environments, when the uncertainty estimates are accurate, but the neural network output is inaccurate, our Bayesian approach searches the tree much more effectively. In addition, we investigate whether popular uncertainty estimation methods are accurate enough to yield significant gains in planning. |
| Nir is an applied AI researcher in General Motors for the past 3.5 years, working on motion planning, decision making and perception algorithms for ADAS and AV features. Before that Nir received is M.Sc. and B.Sc. in Electrical and Computers Engineering from the Technion and worked for 10 years in Rafael in the fields of operational research, multi-objective optimization, and missing planning.
|