Seminar: Graduate Seminar
State-Space Analysis of Transformer Generation Dynamics
Date:
October,13,2026
Start Time:
15:00 - 16:00
Location:
Fishbach 430
Add to:
Lecturer:
Gal Kinberg
Research Areas:
| Transformer-based large language models (LLMs) are inherently feed-forward networks, yet they are deployed in practice in an autoregressive setting. This can give rise to rich dynamical phenomena, ranging from divergence to repetition loops. However, analyzing these phenomena with standard dynamical systems tools, such as fixed points and their stability, faces a basic obstacle: due to their growing memory, these systems lack a fixed-size state, preventing straightforward application of state-space analysis. Here, we take the first steps towards enabling and demonstrating this analysis for growing-memory transformers. We first show that an attention head without positional embeddings is exactly described by a non-linear self-interacting Markov chain with a fixed-size state, and use its timescale separation property to derive a self-consistency condition for fixed points. Building on this condition, we develop an optimization method that finds both stable and unstable fixed points. Applied to individual position-ablated heads of a pretrained 1-layer transformer, it reveals stable repetition attractors, asymptotically unstable yet persistent semantic clusters, and saddle states between neighboring attractors. We then extend this notion to a multi-timescale mean-field approximation for general attention layers, which closely approximates the full 1-layer model, and use it to demonstrate our approach by finding an almost-fixed yet unstable repetition of the real model. Our work opens the door to applying dynamical systems tools to understand LLM generation dynamics as a new approach for transformer interpretability research.
M.Sc. student under the supervision of Prof. Omri Barak.
|