Seminar: Graduate Seminar
Predictive Signature Diversity for Model-Based Offline Reinforcement Learning
Date:
October,19,2026
Start Time:
16:00 - 17:00
Location:
1061, Meyer Building
Zoom:
Zoom link
Add to:
Lecturer:
Hagar Baer
Research Areas:
| Offline reinforcement learning (offline RL) aims to learn policies from static datasets, but suffers from distributional shift when reaching states not represented by the data. Model-based approaches mitigate this issue using learned dynamics models, but are highly sensitive to model uncertainty, especially in regions less covered by the dataset. Existing approaches fail to adequately capture such uncertainty, particularly in terms of worst-case discrepancies between potential models. We propose a predictive, policy-independent framework for model selection based on predictive signature diversity. Models are differentiated by their predictions on shared anchor inputs, enabling comparison of behaviors in a common space. We encourage diversity during model selection by selecting a subset that maximizes worst-case separation of their predictions, yielding a dynamically updated ensemble for policy learning via diverse rollout generation. This decouples diversity estimation from policy rollouts and directly captures model disagreement in uncertain regions. Our empirical evaluation shows improved robustness to model uncertainty over a broad range of tasks compared to state-of-the-art baselines.
M.Sc. student under the supervision of Assistant Prof. Sarah Eisenstein Keren.
|