Supervised learning with probabilistic morphisms and kernel mean embeddings
share
In this paper I propose a concept of a correct loss function in a generative model of supervised learning for an input space π³ and a label space π΄, both of which are measurable spaces. A correct loss function in a generative model of supervised learning must accurately measure the discrepancy between elements of a hypothesis space β of possible predictors and the supervisor operator, even when the supervisor operator does not belong to β. To define correct loss functions, I propose a characterization of a regular conditional probability measure ΞΌ_π΄|π³ for a probability measure ΞΌ on π³Γπ΄ relative to the projection Ξ _π³: π³Γπ΄βπ³ as a solution of a linear operator equation. If π΄ is a separable metrizable topological space with the Borel Ο-algebra β¬ (π΄), I propose an additional characterization of a regular conditional probability measure ΞΌ_π΄|π³ as a minimizer of mean square error on the space of Markov kernels, referred to as probabilistic morphisms, from π³ to π΄. This characterization utilizes kernel mean embeddings. Building upon these results and employing inner measure to quantify the generalizability of a learning algorithm, I extend a result due to Cucker-Smale, which addresses the learnability of a regression model, to the setting of a conditional probability estimation problem. Additionally, I present a variant of Vapnik's regularization method for solving stochastic ill-posed problems, incorporating inner measure, and showcase its applications.
READ FULL TEXT
