Structured variable selection in support vector machines - Statistics > Machine LearningReport as inadecuate

Structured variable selection in support vector machines - Statistics > Machine Learning - Download this document for free, or read online. Document in PDF available to download.

Abstract: When applying the support vector machine SVM to high-dimensionalclassification problems, we often impose a sparse structure in the SVM toeliminate the influences of the irrelevant predictors. The lasso and othervariable selection techniques have been successfully used in the SVM to performautomatic variable selection. In some problems, there is a natural hierarchicalstructure among the variables. Thus, in order to have an interpretable SVMclassifier, it is important to respect the heredity principle when enforcingthe sparsity in the SVM. Many variable selection methods, however, do notrespect the heredity principle. In this paper we enforce both sparsity and theheredity principle in the SVM by using the so-called structured variableselection SVS framework originally proposed in Yuan, Joseph and Zou 2007.We minimize the empirical hinge loss under a set of linear inequalityconstraints and a lasso-type penalty. The solution always obeys the desiredheredity principle and enjoys sparsity. The new SVM classifier can beefficiently fitted, because the optimization problem is a linear program.Another contribution of this work is to present a nonparametric extension ofthe SVS framework, and we propose nonparametric heredity SVMs. Simulated andreal data are used to illustrate the merits of the proposed method.

Author: Seongho Wu, Hui Zou, Ming Yuan


Related documents