Seminar: Pixel Club
IR4RL: Reinforcement Learning from Intermediate Renders for Image-to-Code Generation
Date:
November,17,2026
Start Time:
11:30 - 12:30
Location:
506, New Zisapel Building
Add to:
Lecturer:
Kate Feingold
Research Areas:
| Reinforcement learning is increasingly used to post-train vision-language models for image-to-code generation, such as generating SVG code from a reference image, by optimizing rewards computed from the final rendered output. However, relying on a single terminal reward provides sparse feedback that is poorly aligned with the contribution of individual tokens. A generated program may contain operations that accurately reproduce some parts of the target image alongside others that introduce errors, yet all tokens are trained from the same final outcome. We observe that many intermediate code prefixes are not only executable, but already produce meaningful partial renders that reflect progress toward the target. This property provides a natural source of denser supervision during generation. Based on this observation, we introduce IR4RL, an RL framework with a token-level render-progress reward that turns changes between intermediate renders into localized feedback for the generated sequence. We evaluate our approach on Image-to-SVG and Image-to-TikZ generation. Across both tasks, our method improves over supervised fine-tuning and standard GRPO, yielding new state-of-the-art open-source models. This shows that intermediate rendering provides a simple and effective source of process supervision for RL post-training of image-to-code models. |
| Kate Feingold is a PhD student at the Weizmann Institute of Science, advised by Prof. Tali Dekel, with whom she also completed her MSc. Her research spans multimodal learning, generative models, and 3D/4D perception. Previously, she had over six years of industry experience working on NAS, UAV navigation, and video understanding.
|