Estimating divergence functionals and the likelihood ratio by convex risk minimization - Mathematics > Statistics TheoryReport as inadecuate




Estimating divergence functionals and the likelihood ratio by convex risk minimization - Mathematics > Statistics Theory - Download this document for free, or read online. Document in PDF available to download.

Abstract: We develop and analyze $M$-estimation methods for divergence functionals andthe likelihood ratios of two probability distributions. Our method is based ona non-asymptotic variational characterization of $f$-divergences, which allowsthe problem of estimating divergences to be tackled via convex empirical riskoptimization. The resulting estimators are simple to implement, requiring onlythe solution of standard convex programs. We present an analysis of consistencyand convergence for these estimators. Given conditions only on the ratios ofdensities, we show that our estimators can achieve optimal minimax rates forthe likelihood ratio and the divergence functionals in certain regimes. Wederive an efficient optimization algorithm for computing our estimates, andillustrate their convergence behavior and practical viability by simulations.



Author: XuanLong Nguyen, Martin J. Wainwright, Michael I. Jordan

Source: https://arxiv.org/







Related documents