You are here

Honors thesis defense: A mean-field games approach to score-based generative modeling

Event Category:
Learning Learning
Speaker:
Benjamin Burns
Institution:
University of Massachusetts Amherst

Advancements in diffusion-based generative modeling have made high-performing, image-based generative models, such as Stable Diffusion, accessible to anyone with an internet connection. It was shown early last year that diffusion-based generative models can be formulated in terms of a mean-field game (MFG). For score-based generative modeling (SGM) in particular, this MFG formulation admits alternate characterizations of the SGM based on its optimality conditions, which are a set of coupled nonlinear partial differential equations (PDE). The first PDE is a controlled Fokker-Planck equation which is equivalent to the denoising process in SGM, while the second PDE is a HJB equation that characterizes the optimal controller, whose solution is related to the score function. Based on this mathematically rigorous connection, we develop a PDE-based theory for explaining and understanding the role of latent space in SGM. In addition, we introduce two regularizers, the first based on measuring the discrepancy in the HJB equation, the second based on measuring discrepancy in satisfying the terminal condition. The terminal condition is determined by the training data. By including these regularizers, we are informing the structure of the score function, thereby constraining the search space. This strategy is well-grounded through the theory of mean-field games and HJB equations. Through experiments we will show that the HJB regularizer helps learn the score function in a more stable way, while the terminal condition regularizer is associated with higher quality samples.

Thursday, May 9, 2024 - 11:30am
LGRT 1621