Towards Mixture Proportion Estimation without Irreducibility
share
Mixture proportion estimation (MPE) is a fundamental problem of practical significance, where we are given data from only a mixture and one of its two components to identify the proportion of each component. All existing MPE methods that are distribution-independent explicitly or implicitly rely on the irreducible assumption—the unobserved component is not a mixture containing the observable component. If this is not satisfied, those methods will lead to a critical estimation bias. In this paper, we propose Regrouping-MPE that works without irreducible assumption: it builds a new irreducible MPE problem and solves the new problem. It is worthwhile to change the problem: we prove that if the assumption holds, our method will not affect anything; if the assumption does not hold, the bias from problem changing is less than the bias from violation of the irreducible assumption in the original problem. Experiments show that our method outperforms all state-of-the-art MPE methods on various real-world datasets.
READ FULL TEXT