Aspects of Bayesian model and variable selection using MCMC

2017

2001

In principle, the Bayesian approach to model selection is straightforward. Prior probability distributions are used to describe the uncertainty surrounding all unknowns. After observing the data, the posterior distribution provides a coherent post data summary of the remaining uncertainty which is relevant for model selection. However, the practical implementation of this approach often requires carefully tailored priors and novel posterior calculation methods. In this article, we illustrate some of the fundamental practical issues that arise for two different model selection problems: the variable selection problem for the linear model and the CART model selection problem.

Environmetrics, 2001

We show how the Full Bayesian Signi®cance Test (FBST) can be used as a model selection criterion. The FBST was presented in Pereira and Stern as a coherent Bayesian signi®cance test.

Statistics and Computing, 2000

p(mj D)D p(m) p(Dj m) 6m0 p(m0) p(Dj m0) p(µmj D; m)D p(µmj m) p(Djµm; m)

Institute of Mathematical Statistics Lecture Notes - Monograph Series, 2001

The basics of the Bayesian approach to model selection are first presented, as well as the motivations for the Bayesian approach. We then review four methods of developing default Bayesian procedures that have undergone considerable recent development, the Conventional Prior approach, the Bayes Information Criterion, the Intrinsic Bayes Factor, and the Fractional Bayes Factor. As part of the review, these methods are illustrated on examples involving the normal linear model. The later part of the chapter focuses on comparison of the four approaches, and includes an extensive discussion of criteria for judging model selection procedures.

The Annals of Statistics, 2012

In objective Bayesian model selection, no single criterion has emerged as dominant in defining objective prior distributions. Indeed, many criteria have been separately proposed and utilized to propose differing prior choices. We first formalize the most general and compelling of the various criteria that have been suggested, together with a new criterion. We then illustrate the potential of these criteria in determining objective model selection priors by considering their application to the problem of variable selection in normal linear models. This results in a new model selection objective prior with a number of compelling properties.

Most treatments of the model selection problem are either re- stricted to special situations (lag selection in AR, MA or ARMA models, re- gression selection, selection of a model out of a nested sequence) or to special selection methods (selection through testing or penalization). Our aim is to provide some basic tools for the analysis of model selection as a statistical deci- sion problem, independently of the situation and of the method used. In order to achieve this objective, we embed model selection in the theoretical decision framework oered by modern Decision Theory. This allows us to obtain sim- ple conditions under which pairwise comparison of models and penalization of objective functions arise naturally from preferences defined on the collection of statistical models under scrutiny. As a major application of our framework, we derive necessary and sucient conditions for an information criterion to satisfy in the case of independent and identically distributed realizations ...

2000

Given a set of possible models for variables X and a set of possible parameters for each model, the Bayesian "estimate" of the probability distribution for X given observed data is obtained by averaging over the possible models and their parameters. An often-used approximation for this estimate is obtained by selecting a single model and averaging over its parameters. The approximation is useful because it is computationally efficient, and because it provides a model that facilitates understanding of the domain. A common criterion for model selection is the posterior probability of the model. Another criterion for model selection, proposed by San Martini and Spezzafari (1984), is the predictive performance of a model for the next observation to be seen. From the standpoint of domain understanding, both criteria are useful, because one identifies the model that is most likely, whereas the other identifies the model that is the best predictor of the next observation. To highlight the difference, we refer to the posterior-probability and alternative criteria as the scientific criterion (SC) and engineering criterion (EC), respectively. When we are interested in predicting the next observation, the model-averaged estimate is at least as good as that produced by EC, which itself is at least as good as the estimate produced by SC. We show experimentally that, for Bayesian-network models containing discrete variables only, the predictive performance of the model average can be significantly better than those of single models selected by either criterion, and that differences between models selected by the two criterion can be substantial.

2005

Abstract: We propose a new approach for model selection in mathematical statistics that is based not on the probability but on the `waiting time' of a sample. By waiting time of a sample we call the average time of the first appearance of the sample in a sequence of independent identically distributed random variables. In the paper we consider a few simple examples to illustrate the main idea and further mathematical problems related to the new approach.

2002

Several MCMC methods have been proposed for estimating probabilities of models and associated'model-averaged'posterior distributions in the presence of model uncertainty. We discuss, compare, develop and illustrate several of these methods, focussing on connections between them.