Equivariance and algebraic priors in neural networks

Blackboard talk presented in the Sandwich seminar series at the Cambridge Logic And Semantics Hub (CLASH).

Abstract

Neural networks are “black boxes” that progressively transform an input into vector representations that somehow preserve the main semantic information of the data. In this talk we will discuss how equivariance is used in neural network design as a principled way to introduce “inductive biases” (beliefs we have about the problem at hand). Then, we will look at approximate equivariance methods and how they interact with optimisation.

Finally, I will present a recent work that uses these concepts to study the algebraic structure of neural network representations.