Intervention Harvesting for Context-Dependent Examination-Bias Estimation
share
Accurate estimates of examination bias are crucial for unbiased learning-to-rank from implicit feedback in search engines and recommender systems, since they enable the use of Inverse Propensity Score (IPS) weighting techniques to address selection biases and missing data Joachims/etal/17a. Unfortunately, existing examination-bias estimators Agarwal/etal/18c, wang2018position are limited to the Position-Based Model (PBM) chuklin2015click, where the examination bias may only depend on the rank of the document. To overcome this limitation, we propose a Contextual Position-Based Model (CPBM) where the examination bias may also depend on a context vector describing the query and the user. Furthermore, we propose an effective estimator for the CPBM based on intervention harvesting Agarwal/etal/18c. A key feature of the estimator is that it does not require disruptive interventions but merely exploits natural variation resulting from the use of multiple historic ranking functions. Semi-synthetic experiments on the Yahoo Learning-To-Rank dataset demonstrate the superior effectiveness of the new approach.
READ FULL TEXT