Skip to Main content Skip to Navigation
New interface
Preprints, Working Papers, ...

A Sparse Generative Model and its EM Algorithm for Variable Selection in High-Dimensional Regression

Abstract : We address the problem of Bayesian variable selection for high-dimensional linear regression. We consider a generative model that uses a spike-and-slab like prior distribution obtained by multiplying a deterministic binary vector, which traduces the sparsity of the problem, with a random Gaussian parameter vector. Such a model allows an expectation-maximization algorithm, optimizing a type-II log-likelihood, to be derived. This marginal log-likelihood involves an Occam's razor term, automatically penalizing the complexity, which is used for model selection. Albeit NP-hard, the algorithm we propose can be relaxed in order to infer a family of models. Model selection is eventually performed afterwards based on Occam's razor. We report numerical comparisons between our method, called spinyReg, and the most recent variable selection algorithms, including lasso, adaptive lasso and stability selection. SpinyReg turns out to perform well compared to those algorithms, especially regarding false detection rates.
Document type :
Preprints, Working Papers, ...
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-01003395
Contributor : Charles Bouveyron Connect in order to contact the contributor
Submitted on : Wednesday, September 10, 2014 - 7:00:07 AM
Last modification on : Friday, May 6, 2022 - 4:50:07 PM
Long-term archiving on: : Thursday, December 11, 2014 - 11:40:22 AM

File

SpinyReg.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01003395, version 1

Citation

Charles Bouveyron, Julien Chiquet, Pierre Latouche, Pierre-Alexandre Mattei. A Sparse Generative Model and its EM Algorithm for Variable Selection in High-Dimensional Regression. 2014. ⟨hal-01003395v1⟩

Share

Metrics

Record views

889

Files downloads

929