Generative Models for Planning and Control in Robotic Domains - Faculty of Electrical And Computer Engineering - Technion Skip to content

Seminar: Machine Learning Seminar

ECE Women Community

Generative Models for Planning and Control in Robotic Domains

Date: December,11,2024 Start Time: 15:30 - 16:30
Location: 1061, Meyer Building
Add to:
Lecturer: Orr Krupnik
Model-based reinforcement learning is the practice of utilizing prior information and domain knowledge to learn models of the environment and use them to expedite the training process of reinforcement learning agents. With the recent advancement of deep generative models such as diffusion models and transformers, models have become more powerful and expressive. Consequently, model-based RL has grown in popularity and found many use cases. However, many open questions remain in the training and usage of model-based approaches. How can we efficiently use data to train models for a variety of downstream tasks? How can we learn models that are dynamic and adaptive to novel scenarios? What is the best way to use the model to improve agent performance?

In this talk, I will present some of my work attempting to tackle these challenges.First, we will discuss multi-agent settings where two players can compete or collaborate to obtain goals. The central challenge in these scenarios is to model the interactions between agents while still allowing for optimization of each agent behavior separately. Our approach is based on a Variational Autoencoder which generates agent trajectories and can be optimized for cooperative or competitive behaviors while trained on a single dataset. Next, we will examine a method allowing learned models to adapt to new scenarios at inference time. We propose a simple approach based on the cross-entropy method which enables fine-tuning of generative models given test-time observations. While we present results on robotic grasping and inverse kinematics tasks, the approach is general and can be used with a variety of generative models. Finally, high-fidelity models and simulators are evolving and improving both in their representation capabilities and their speed. We study how to use a fast state-based simulator for a realistic 3D bin packing task, by applying planning and search techniques to improve exploration and policy learning.

Ph.D. Under the supervision of Prof. Aviv Tamar.

 

All Seminars