Symmetry and Generalisation in Machine Learning

It is widely believed that engineering a model to be invariant/equivariant improves generalisation. Despite the growing popularity of this approach, a precise characterisation of the generalisation benefit is lacking. By considering the action of certain averaging operators, we provide the first provably non-zero improvement in generalisation for invariant/equivariant models when the target distribution is invariant/equivariant to the action of a compact group. Our results are general and hold for almost any predictor used in machine learning. Additionally, we specialise our results to linear models and kernel methods to provide a precise characterisation of the generalisation benefit in terms of the number of training examples and properties of the group action.