A Transfer Principle: Universal Approximators Between Metric Spaces From Euclidean Universal Approximators
share
We build universal approximators of continuous maps between arbitrary Polish metric spaces π³ and π΄ using universal approximators between Euclidean spaces as building blocks. Earlier results assume that the output space π΄ is a topological vector space. We overcome this limitation by "randomization": our approximators output discrete probability measures over π΄. When π³ and π΄ are Polish without additional structure, we prove very general qualitative guarantees; when they have suitable combinatorial structure, we prove quantitative guarantees for HΓΆlder-like maps, including maps between finite graphs, solution operators to rough differential equations between certain Carnot groups, and continuous non-linear operators between Banach spaces arising in inverse problems. In particular, we show that the required number of Dirac measures is determined by the combinatorial structure of π³ and π΄. For barycentric π΄, including Banach spaces, β-trees, Hadamard manifolds, or Wasserstein spaces on Polish metric spaces, our approximators reduce to π΄-valued functions. When the Euclidean approximators are neural networks, our constructions generalize transformer networks, providing a new probabilistic viewpoint of geometric deep learning.
READ FULL TEXT