Exponential Screening and optimal rates of sparse estimation - Mathematics > Statistics TheoryReport as inadecuate

Exponential Screening and optimal rates of sparse estimation - Mathematics > Statistics Theory - Download this document for free, or read online. Document in PDF available to download.

Abstract: In high-dimensional linear regression, the goal pursued here is to estimatean unknown regression function using linear combinations of a suitable set ofcovariates. One of the key assumptions for the success of any statisticalprocedure in this setup is to assume that the linear combination is sparse insome sense, for example, that it involves only few covariates. We consider ageneral, non necessarily linear, regression with Gaussian noise and study arelated question that is to find a linear combination of approximatingfunctions, which is at the same time sparse and has small mean squared errorMSE. We introduce a new estimation procedure, called Exponential Screeningthat shows remarkable adaptation properties. It adapts to the linearcombination that optimally balances MSE and sparsity, whether the latter ismeasured in terms of the number of non-zero entries in the combination$\ell 0$ norm or in terms of the global weight of the combination $\ell 1$norm. The power of this adaptation result is illustrated by showing thatExponential Screening solves optimally and simultaneously all the problems ofaggregation in Gaussian regression that have been discussed in the literature.Moreover, we show that the performance of the Exponential Screening estimatorcannot be improved in a minimax sense, even if the optimal sparsity is known inadvance. The theoretical and numerical superiority of Exponential Screeningcompared to state-of-the-art sparse procedures is also discussed.

Author: Philippe Rigollet, Alexandre Tsybakov

Source: https://arxiv.org/


Related documents