Anne and Peter Costa Distinguished Lecture in Applied Mathematics
This annual lecture is for the benefit of the faculty and graduate students in Applied Mathematics at the University of Massachusetts Amherst.
Fall 2026
Philippe Rigollet
Cecil and Ida Green Distinguished Professor of Mathematics
Massachusetts Institute of Technology
Title: The Dynamics of Transformers
Abstract: Transformers are a class of neural network architectures behind the model AI models. Yet how signals propagate through their layers remains poorly understood. We study this question by viewing attention as a system of interacting particles, connecting Transformers to ideas from dynamical systems, probability, optimization, and geometry.
Even simplified models exhibit rich behavior. Tokens can cluster, and distinct groups can persist across many layers before eventually merging. A key insight is that these contractive properties of attention can be understood from a variational perspective, with the token dynamics arising from an underlying energy minimization principle.
This variational perspective also provides a way to design new architectures. Some of these significantly outperform existing ones, illustrating how mathematics can serve not only to analyze models, but also to guide their design.
Past Speakers
Fall 2025
Laure Zanna
Joseph B. Keller and Herbert B. Keller Professor in Applied Mathematics
Courant Institute, NYU
Fall 2023
Anne Gelb
John G. Kemeny Parents Professor
Dartmouth College