The Geometry of Manifold Learning: Singular foliations via Deep Learning models

Abstract

Deep learning algorithms have become widely used and successful across a wide range of tasks. A key challenge is determining whether these algorithms truly capture the intrinsic geometry of their training data in order to better understand what they learn from data. In this theoretical work, we explore a new geometric structure defined on the data space by a trained neural network through the Data Information Matrix (DIM), a variation of the Fisher Information Matrix. This new geometric structure is a foliation and the main focus of this work is to explore the non-smooth case of ReLU activation functions, where traditional geometric tools often fail. The foliation is defined through a distribution spanned by the columns of the DIM. We first prove that this distribution is non-smooth in the sense of distributions, rendering the existence of an associated foliation much more complex. To circumvent this issue, we introduce a smoothed version of this distribution that differs from the original only on a set of measure zero. Finally, we prove that this smoothed distribution induces a singular foliation on the data space, which is based on the data the network has been trained with. Our theoretical contributions provide a geometric framework to better understand how neural networks organize their input space in non-smooth settings, bridging differential geometry and deep learning theory.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Differential Geometry

• arXiv preprint · 28 October 2026

  Einstein Metrics and Quantum Control on Symmetric Tensor Product Quotients

  Honghuai Fang
• arXiv preprint · 26 October 2026

  Geometric aspects of fault tolerance

  Jonathan Conrad
• arXiv preprint · 28 October 2026

  The geometry of Floquet states

  Vladimir Y. Cherynak and John R. Klein
• arXiv preprint · 22 October 2026

  SympFormer: Inertial Attention Blocks from Damped Hamiltonian Dynamics on Spaces of Probability Measures

  Viktor Stein, Levin Maier, Wuchen Li and 1 other
• arXiv preprint · 31 October 2026

  The Continuum Limit of the Deep Linear Network

  Catherine Eva Pfaff and Karen Uhlenbeck

All new papers in Differential Geometry