Two-layer neural networks with values in a Banach space

05/05/2021
by   Yury Korolev, et al.
0

We study two-layer neural networks whose domain and range are Banach spaces with separable preduals. In addition, we assume that the image space is equipped with a partial order, i.e. it is a Riesz space. As the nonlinearity we choose the lattice operation of taking the positive part; in case of ℝ^d-valued neural networks this corresponds to the ReLU activation function. We prove inverse and direct approximation theorems with Monte-Carlo rates, extending existing results for the finite-dimensional case. In the second part of the paper, we consider training such networks using a finite amount of noisy observations from the regularisation theory viewpoint. We discuss regularity conditions known as source conditions and obtain convergence rates in a Bregman distance in the regime when both the noise level goes to zero and the number of samples goes to infinity at appropriate rates.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset