Active Distribution Learning from Indirect Samples
share
This paper studies the problem of learning the probability distribution P_X of a discrete random variable X using indirect and sequential samples. At each time step, we choose one of the possible K functions, g_1, ..., g_K and observe the corresponding sample g_i(X). The goal is to estimate the probability distribution of X by using a minimum number of such sequential samples. This problem has several real-world applications including inference under non-precise information and privacy-preserving statistical estimation. We establish necessary and sufficient conditions on the functions g_1, ..., g_K under which asymptotically consistent estimation is possible. We also derive lower bounds on the estimation error as a function of total samples and show that it is order-wise achievable. Leveraging these results, we propose an iterative algorithm that i) chooses the function to observe at each step based on past observations; and ii) combines the obtained samples to estimate p_X. The performance of this algorithm is investigated numerically under various scenarios, and shown to outperform baseline approaches.
READ FULL TEXT