Martin Wainwright: The Information Geometry of Diffusion Sampling: A Mathematical View of Generative AI

Massachusetts Institute of Technology

Diffusion models have emerged as one of the most successful approaches to generative AI, producing strikingly realistic images and video, and finding growing use in areas including biology, materials science, and robotics.  The basic idea is deceptively simple: gradually destroy the structure in data by adding noise, learn how to undo this corruption, and then run the reverse process to generate new samples.

But how difficult is this reverse journey from noise to data? In this lecture, I will describe a mathematical perspective in which the answer is governed by how information and geometry interact along the diffusion path.  This viewpoint brings together Shannon's information theory, data geometry, and the design of efficient algorithms.