High dimensional robust M-estimation: asymptotic variance via approximate message passingReport as inadecuate

High dimensional robust M-estimation: asymptotic variance via approximate message passing - Download this document for free, or read online. Document in PDF available to download.

Probability Theory and Related Fields

, Volume 166, Issue 3–4, pp 935–969

First Online: 07 November 2015Received: 14 November 2013Revised: 28 May 2015


In a recent article, El Karoui et al. Proc Natl Acad Sci 11036:14557–14562, 2013 study the distribution of robust regression estimators in the regime in which the number of parameters p is of the same order as the number of samples n. Using numerical simulations and ‘highly plausible’ heuristic arguments, they unveil a striking new phenomenon. Namely, the regression coefficients contain an extra Gaussian noise component that is not explained by classical concepts such as the Fisher information matrix. We show here that that this phenomenon can be characterized rigorously using techniques that were developed by the authors for analyzing the Lasso estimator under high-dimensional asymptotics. We introduce an approximate message passing AMP algorithm to compute M-estimators and deploy state evolution to evaluate the operating characteristics of AMP and so also M-estimates. Our analysis clarifies that the ‘extra Gaussian noise’ encountered in this problem is fundamentally similar to phenomena already studied for regularized least squares in the setting
Mathematics Subject Classification62F10 62F12 60F99  Download fulltext PDF

Author: David Donoho - Andrea Montanari

Source: https://link.springer.com/

Related documents