On the consistency of the Kozachenko-Leonenko entropy estimate
share
We revisit the problem of the estimation of the differential entropy H(f) of a random vector X in R^d with density f, assuming that H(f) exists and is finite. In this note, we study the consistency of the popular nearest neighbor estimate H_n of Kozachenko and Leonenko. Without any smoothness condition we show that the estimate is consistent (E{|H_n - H(f)|}→ 0 as n →∞) if and only if 𝔼{log ( X + 1 )} < ∞. Furthermore, if X has compact support, then H_n → H(f) almost surely.
READ FULL TEXT