Publications

We use techniques from representation theory to uncover what structure neural networks tend to learn, finding a strong preference for the regular representation. Building on this insight, we propose a simple method to enforce approximate equivariance with strong experimental results.

We track the Shannon entropy of intermediate predictions in transformers and use them to uncover consistent computational patterns.

We propose a metric learning scheme to enhance Clifford Group Equivariant Neural Networks, motivated using insights from category theory to guarantee its soundness.

We establish a formal connection between the two seminal decompositions using ideas from statistical learning theory and information geometry.