What is one way to model "feature learning" in neural networks?

Speaker: Matthew Li

Abstract: Recent advances have significantly deepened our understanding of what
structural properties, or "features", can be learned by neural
networks when trained via (stochastic) gradient descent. In this talk,
I'll focus on an idealized mathematical model of neural network
training known as "correlational statistical query (CSQ)" learners. In
particular, I aim to sketch out the foundational ideas behind the
computational-statistical gaps identified in these learners by Damian,
Lee, and Soltanolkotabi [1]. Minimal prior knowledge of machine
learning will be required to understand these ideas.

[1] A. Damian, J. Lee, and M. Soltanolkotabi, “Neural Networks can
Learn Representations with Gradient Descent,” in Proceedings of Thirty
Fifth Conference on Learning Theory, PMLR, Jun. 2022, pp. 5413–5452.
Available: https://proceedings.mlr.press/v178/damian22a.html