Seminar: Graduate Seminar
Representation Driven Exploration in Reinforcement Learning
Date:
January,18,2026
Start Time:
11:00 - 12:00
Location:
608, Zisapel Building
Add to:
Lecturer:
Ofir Nabati
Research Areas:
| Efficient exploration remains a primary challenge in Reinforcement Learning (RL), particularly in environments with large state spaces or sparse rewards. Standard methods often fail to distinguish between visited and unvisited regions effectively, leading to sample inefficiency. This seminar argues that the key to solving the exploration-exploitation dilemma lies in learning representations that are specifically aligned with exploration objectives. I present three novel frameworks that establish this connection. First, I introduce a method for neural-linear bandits that resolves the problem of catastrophic forgetting, enabling agents to maintain accurate uncertainty estimates online by matching the likelihood of past data. Second, I present “RepRL,” a framework that shifts exploration from the state space to a learned latent space of policies, allowing for efficient exploration-exploitation using linear bandit algorithms. Finally, I discuss the Spectral Bellman Method, which derives a representation learning objective directly from the spectral properties of the optimal Bellman operator. This approach learns features whose covariance structure naturally facilitates structured exploration, leading to improved performance in hard exploration tasks. |
| Ofir Nabati is a PhD candidate, under the supervision of Prof. Shie Mannor. Ofir is in the final stages of his PhD at the Technion, after completing his B.Sc. and M.Sc. in Electrical Engineering at Tel Aviv University. His research focuses on exploration in reinforcement learning through representation learning, as well as the integration of diffusion models into reinforcement learning frameworks.
PhD candidate, under the supervision of Prof. Shie Mannor.
|