WorkshopsGrouped variable selection via nested spike and slab Priors
59
reads
We study grouped variable selection problems by proposing a specified prior, called the nested spike and slab prior, for modeling collective behavior of the regression coefficients. At the group level, the nested spike and slab prior puts positive mass on the event that the $l_{2}$ norm of the grouped coefficients equals to zero. At the individual level, each coefficient is further assumed to follow a spike and slab prior. We carry out maximum a posteriori estimation for the model by applying a blockwise coordinate descent algorithm to solve an optimization problem involving an approximate objective modified by majorization-minimization techniques. Simulation study shows that the proposed MAP estimator performs relatively well in the situations in which the true and redundant covariates are both covered by the same group. Asymptotic analysis under a frequentist's framework shows that the proposed estimator can enjoy the benefit of group sparsity as its $l_{2}$ estimation error can have an order of magnitude smaller than the one of the benchmark lasso estimator. In addition, given some regular conditions hold, the proposed estimator is asymptotically invariant to group structures, and its model selection consistency can be established without irrepresentable-type conditions being imposed.
reads
Tso-Jung Yen
2011-06-22
16:55:00 - 17:30:00
308 , Mathematics Research Center Building (ori. New Math. Bldg.)
We study grouped variable selection problems by proposing a specified prior, called the nested spike and slab prior, for modeling collective behavior of the regression coefficients. At the group level, the nested spike and slab prior puts positive mass on the event that the $l_{2}$ norm of the grouped coefficients equals to zero. At the individual level, each coefficient is further assumed to follow a spike and slab prior. We carry out maximum a posteriori estimation for the model by applying a blockwise coordinate descent algorithm to solve an optimization problem involving an approximate objective modified by majorization-minimization techniques. Simulation study shows that the proposed MAP estimator performs relatively well in the situations in which the true and redundant covariates are both covered by the same group. Asymptotic analysis under a frequentist's framework shows that the proposed estimator can enjoy the benefit of group sparsity as its $l_{2}$ estimation error can have an order of magnitude smaller than the one of the benchmark lasso estimator. In addition, given some regular conditions hold, the proposed estimator is asymptotically invariant to group structures, and its model selection consistency can be established without irrepresentable-type conditions being imposed.