Papers by Marcelo Pereyra
2018 26th European Signal Processing Conference (EUSIPCO)
We propose to perform Bayesian uncertainty quantification via convex optimization tools (BUQO), i... more We propose to perform Bayesian uncertainty quantification via convex optimization tools (BUQO), in the context of high dimensional inverse problems. We quantify the uncertainty associated with particular structures appearing in the maximum a posteriori estimate, obtained from a log-concave Bayesian model. A hypothesis test is defined, where the null hypothesis represents the non-existence of the structure of interest in the true image. To determine if this null hypothesis is rejected, we use the data and prior knowledge. Computing such test in the context of imaging problem is often intractable due to the high dimensionality involved. In this work, we propose to leverage probability concentration phenomena and the underlying convex geometry to formulate the Bayesian hypothesis test as a convex minimization problem. This problem is subsequently solved using a proximal primal-dual algorithm. The proposed method is applied to astronomical radio-interferometric imaging. Index Terms-Bayesian uncertainty quantification, hypothesis testing, astronomical imaging, inverse problem, proximal primaldual algorithm
SIAM Journal on Imaging Sciences
This paper proposes a new methodology for performing Bayesian inference in imaging inverse proble... more This paper proposes a new methodology for performing Bayesian inference in imaging inverse problems where the prior knowledge is available in the form of training data. Following the manifold hypothesis and adopting a generative modelling approach, we construct a data-driven prior that is supported on a sub-manifold of the ambient space, which we can learn from the training data by using a variational autoencoder or a generative adversarial network. We establish the existence and well-posedness of the associated posterior distribution and posterior moments under easily verifiable conditions, providing a rigorous underpinning for Bayesian estimators and uncertainty quantification analyses. Bayesian computation is performed by using a parallel tempered version of the preconditioned Crank-Nicolson algorithm on the manifold, which is shown to be ergodic and robust to the non-convex nature of these data-driven models. In addition to point estimators and uncertainty quantification analyses, we derive a model misspecification test to automatically detect situations where the data-driven prior is unreliable, and explain how to identify the dimension of the latent space directly from the training data. The proposed approach is illustrated with a range of experiments with the MNIST dataset, where it outperforms alternative image reconstruction approaches from the state of the art. A model accuracy analysis suggests that the Bayesian probabilities reported by the data-driven models are also remarkably accurate under a frequentist definition of probability.
2018 26th European Signal Processing Conference (EUSIPCO), 2018
This paper investigates different algorithms to perform image restoration from single-photon meas... more This paper investigates different algorithms to perform image restoration from single-photon measurements corrupted with Poisson noise. The restoration problem is formulated in a Bayesian framework and several state-of-the-art Monte Carlo samplers are considered to estimate the unknown image and quantify its uncertainty. The different samplers are compared through a series of experiments conducted with synthetic images. The results demonstrate the scaling properties of the proposed samplers as the dimensionality of the problem increases and the number of photons decreases. Moreover, our experiments show that for a certain photon budget (i.e., acquisition time of the imaging device), downsampling the observations can yield better reconstruction results.
Publication in the conference proceedings of EUSIPCO, Barcelona, Spain, 2011
This course presents an overview of modern Bayesian strategies for solving imaging inverse proble... more This course presents an overview of modern Bayesian strategies for solving imaging inverse problems. We will start by introducing the Bayesian statistical decision theory framework underpinning Bayesian analysis, and then explore efficient numerical methods for performing Bayesian computation in large-scale settings. We will pay special attention to high-dimensional imaging models that are log-concave w.r.t. the unknown image, related to so-called "convex imaging problems". This will provide an opportunity to establish connections with the convex optimisation and machine learning approaches to imaging, and to discuss some of their relative strengths and drawbacks. Examples of topics covered in the course include: efficient stochastic simulation and optimisation numerical methods that tightly combine proximal convex optimisation with Markov chain Monte Carlo techniques; strategies for estimating unknown model parameters and performing model selection, methods for calculatin...
This letter considers the problem of computing the Cramer-Rao bound for the parameters of a Marko... more This letter considers the problem of computing the Cramer-Rao bound for the parameters of a Markov random field. Computation of the exact bound is not feasible for most fields of interest because their likelihoods are intractable and have intractable derivatives. We show here how it is possible to formulate the computation of the bound as a statistical inference problem that can be solve approximately, but with arbitrarily high accuracy, by using a Monte Carlo method. The proposed methodology is successfully applied on the Ising and the Potts models.
This paper addresses the problem of estimating the Potts parameter β jointly with the unknown pa-... more This paper addresses the problem of estimating the Potts parameter β jointly with the unknown pa-rameters of a Bayesian model within a Markov chain Monte Carlo (MCMC) algorithm. Standard MCMC methods cannot be applied to this problem because performing inference on β requires computing the intractable normalizing constant of the Potts model. In the proposed MCMC method the estimation of β is conducted using a likelihood-free Metropolis-Hastings algorithm. Experimental results obtained for synthetic data show that estimating β jointly with the other unknown parameters leads to estimation results that are as good as those obtained with the actual value of β. On the other hand, assuming that the value of β is known can degrade estimation performance significantly if this value is incorrect. To illustrate the interest of this method, the proposed algorithm is successfully applied to real bidimensional SAR and tridimensional ultrasound images.
Imaging methods often rely on Bayesian statistical inference strategies to solve difficult imagin... more Imaging methods often rely on Bayesian statistical inference strategies to solve difficult imaging problems. Applying Bayesian methodology to imaging requires the specification of a likelihood function and a prior distribution, which define the Bayesian statistical model from which the posterior distribution of the image is derived. Specifying a suitable model for a specific application can be very challenging, particularly when there is no reliable ground truth data available. Bayesian model selection provides a framework for selecting the most appropriate model directly from the observed data, without reference to ground truth data. However, Bayesian model selection requires the computation of the marginal likelihood (Bayesian evidence), which is computationally challenging, prohibiting its use in high-dimensional imaging problems. In this work we present the proximal nested sampling methodology to objectively compare alternative Bayesian imaging models, without reference to groun...
2014 22nd European Signal Processing Conference (EUSIPCO), 2014
This paper presents a new approximate Bayesian estimator for hidden Potts-Markov random fields, w... more This paper presents a new approximate Bayesian estimator for hidden Potts-Markov random fields, with application to fast K-class image segmentation. The estimator is derived by conducting a small-variance-asymptotic analysis of an augmented Bayesian model in which the spatial regularisation and the integer-constrained terms of the Potts model are decoupled. This leads to a new image segmentation methodology that can be efficiently implemented in large 2D and 3D scenarios by using modern convex optimisation techniques. Experimental results on synthetic and real images as well as comparisons with state-of-the-art algorithms confirm that the proposed methodology converges extremely fast and produces accurate segmentation results in only few iterations.
2019 27th European Signal Processing Conference (EUSIPCO), Sep 1, 2019
Inverse problems play a key role in modern image/signal processing methods. However, since they a... more Inverse problems play a key role in modern image/signal processing methods. However, since they are generally ill-conditioned or ill-posed due to lack of observations, their solutions may have significant intrinsic uncertainty. Analysing and quantifying this uncertainty is very challenging, particularly in high-dimensional problems and problems with non-smooth objective functionals (e.g. sparsity-promoting priors). In this article, a series of strategies to visualise this uncertainty are presented, e.g. highest posterior density credible regions, and local credible intervals (cf. error bars) for individual pixels and superpixels. Our methods support non-smooth priors for inverse problems and can be scaled to high-dimensional settings. Moreover, we present strategies to automatically set regularisation parameters so that the proposed uncertainty quantification (UQ) strategies become much easier to use. Also, different kinds of dictionaries (complete and over-complete) are used to represent the image/signal and their performance in the proposed UQ methodology is investigated. Index Terms-Uncertainty quantification, image/signal processing, inverse problem, Bayesian inference, convex optimisation.
SIAM Journal on Imaging Sciences, 2019
Maximum-a-posteriori (MAP) estimation is the main Bayesian estimation methodology in imaging scie... more Maximum-a-posteriori (MAP) estimation is the main Bayesian estimation methodology in imaging sciences, where high dimensionality is often addressed by using Bayesian models that are log-concave and whose posterior mode can be computed efficiently by convex optimisation. However, despite its success and wide adoption, MAP estimation is not theoretically well understood yet. In particular, the prevalent view in the community is that MAP estimation is not proper Bayesian estimation in the sense of Bayesian decision theory because it does not minimise a meaningful expected loss function (unlike the minimum mean squared error (MMSE) estimator that minimises the mean squared loss). This paper addresses this theoretical gap by presenting a general decision-theoretic derivation of MAP estimation in Bayesian models that are log-concave. A main novelty is that our analysis is based on differential geometry, and proceeds as follows. First, we use the underlying convex geometry of the Bayesian model to induce a Riemannian geometry on the parameter space. We then use differential geometry to identify the so-called natural or canonical loss function to perform Bayesian point estimation in that Riemannian manifold. For log-concave models, this canonical loss coincides with the Bregman divergence associated with the negative log posterior density. Following on from this, we show that the MAP estimator is the only Bayesian estimator that minimises the expected canonical loss, and that the posterior mean or MMSE estimator minimises the dual canonical loss. We then study the question of MAP and MSSE estimation performance in high dimensions. Precisely, we establish a universal bound on the expected canonical error as a function of image dimension, providing new insights the good empirical performance observed in convex problems. Together, these results provide a new understanding of MAP and MMSE estimation in log-concave settings, and of the multiple beneficial roles that convex geometry plays in imaging problems. Finally, we illustrate this new theory by analysing the regularisation-by-denoising Bayesian models, a class of state-of-the-art imaging models where priors are defined implicitly through image denoising algorithms, and an image denoising model with a wavelet shrinkage prior.
SIAM Journal on Imaging Sciences, 2019
We propose a Bayesian uncertainty quantification method for large-scale imaging inverse problems.... more We propose a Bayesian uncertainty quantification method for large-scale imaging inverse problems. Our method applies to all Bayesian models that are log-concave, where maximum-aposteriori (MAP) estimation is a convex optimization problem. The method is a framework to analyse the confidence in specific structures observed in MAP estimates (e.g., lesions in medical imaging, celestial sources in astronomical imaging), to enable using them as evidence to inform decisions and conclusions. Precisely, following Bayesian decision theory, we seek to assert the structures under scrutiny by performing a Bayesian hypothesis test that proceeds as follows: firstly, it postulates that the structures are not present in the true image, and then seeks to use the data and prior knowledge to reject this null hypothesis with high probability. Computing such tests for imaging problems is generally very difficult because of the high dimensionality involved. A main feature of this work is to leverage probability concentration phenomena and the underlying convex geometry to formulate the Bayesian hypothesis test as a convex problem, that we then efficiently solve by using scalable optimization algorithms. This allows scaling to high-resolution and high-sensitivity imaging problems that are computationally unaffordable for other Bayesian computation approaches. We illustrate our methodology, dubbed BUQO (Bayesian Uncertainty Quantification by Optimization), on a range of challenging Fourier imaging problems arising in astronomy and medicine. Matlab code for the proposed uncertainty quantification method is available on GitHub.
Monthly Notices of the Royal Astronomical Society, 2018
Uncertainty quantification is a critical missing component in radio interferometric imaging that ... more Uncertainty quantification is a critical missing component in radio interferometric imaging that will only become increasingly important as the big-data era of radio interferometry emerges. Statistical sampling approaches to perform Bayesian inference, like Markov Chain Monte Carlo (MCMC) sampling, can in principle recover the full posterior distribution of the image, from which uncertainties can then be quantified. However, for massive data sizes, like those anticipated from the Square Kilometre Array, it will be difficult if not impossible to apply any MCMC technique due to its inherent computational cost. We formulate Bayesian inference problems with sparsity-promoting priors (motivated by compressive sensing), for which we recover maximum a posteriori (MAP) point estimators of radio interferometric images by convex optimization. Exploiting recent developments in the theory of probability concentration, we quantify uncertainties by post-processing the recovered MAP estimate. Three strategies to quantify uncertainties are developed: (i) highest posterior density credible regions, (ii) local credible intervals (cf. error bars) for individual pixels and superpixels, and (iii) hypothesis testing of image structure. These forms of uncertainty quantification provide rich information for analysing radio interferometric observations in a statistically robust manner. Our MAP-based methods are approximately 10 5 times faster computationally than state-of-theart MCMC methods and, in addition, support highly distributed and parallelized algorithmic structures. For the first time, our MAP-based techniques provide a means of quantifying uncertainties for radio interferometric imaging for realistic data volumes and practical use, and scale to the emerging big data era of radio astronomy.
SIAM Journal on Imaging Sciences, 2017
Solutions to inverse problems that are ill-conditioned or ill-posed may have significant intrinsi... more Solutions to inverse problems that are ill-conditioned or ill-posed may have significant intrinsic uncertainty. Unfortunately, analysing and quantifying this uncertainty is very challenging, particularly in high-dimensional problems. As a result, while most modern mathematical imaging methods produce impressive point estimation results, they are generally unable to quantify the uncertainty in the solutions delivered. This paper presents a new general methodology for approximating Bayesian high-posterior-density credibility regions in inverse problems that are convex and potentially very high-dimensional. The approximations are derived by using recent concentration of measure results related to information theory for log-concave random vectors. A remarkable property of the approximations is that they can be computed very efficiently, even in large-scale problems, by using standard convex optimisation techniques. In particular, they are available as a by-product in problems solved by maximum-a-posteriori estimation. The approximations also have favourable theoretical properties, namely they outer-bound the true high-posterior-density credibility regions, and they are stable with respect to model dimension. The proposed methodology is illustrated on two high-dimensional imaging inverse problems related to tomographic reconstruction and sparse deconvolution, where the approximations are used to perform Bayesian hypothesis tests and explore the uncertainty about the solutions, and where proximal Markov chain Monte Carlo algorithms are used as benchmark to compute exact credible regions and measure the approximation error.
IEEE transactions on image processing : a publication of the IEEE Signal Processing Society, 2017
This paper presents a new Bayesian estimation technique for hidden Potts-Markov random fields wit... more This paper presents a new Bayesian estimation technique for hidden Potts-Markov random fields with unknown regularisation parameters, with application to fast unsupervised K -class image segmentation. The technique is derived by first removing the regularisation parameter from the Bayesian model by marginalisation, followed by a small-variance-asymptotic (SVA) analysis in which the spatial regularisation and the integer-constrained terms of the Potts model are decoupled. The evaluation of this SVA Bayesian estimator is then relaxed into a problem that can be computed efficiently by iteratively solving a convex total-variation denoising problem and a least-squares clustering ( K -means) problem, both of which can be solved straightforwardly, even in high-dimensions, and with parallel computing techniques. This leads to a fast fully unsupervised Bayesian image segmentation methodology in which the strength of the spatial regularisation is adapted automatically to the observed image du...
Http Www Theses Fr, Jul 4, 2012
This thesis studies statistical image processing of high frequency ultrasound imaging, with appli... more This thesis studies statistical image processing of high frequency ultrasound imaging, with application to in-vivo exploration of human skin and noninvasive lesion assessment. More precisely, Bayesian methods are considered in order to perform tissue segmentation in ultrasound images of skin. It is established that ultrasound signals backscattered from skin tissues converge to a complex Levy Flight random process with non-Gaussian α-stable statistics. The envelope signal follows a generalized (heavy-tailed) Rayleigh distribution. Based on these results, it is proposed to model the distribution of multiple-tissue ultrasound images as a spatially coherent finite mixture of heavy-tailed Rayleigh distributions. Spatial coherence inherent to biological tissues is modeled by a Potts Markov random field. An original Bayesian algorithm combined with a Markov chain Monte Carlo method is then proposed to jointly estimate the mixture parameters and a label-vector associating each voxel to a tissue. The proposed method is successfully applied to the segmentation of in-vivo skin tumors in high frequency 2D and 3D ultrasound images. This method is subsequently extended by including the estimation of the Potts regularization parameter β within the Markov chain Monte Carlo (MCMC) algorithm. Standard MCMC methods cannot be applied to this problem because the likelihood of β is intractable. This difficulty is addressed by using a likelihood-free Metropolis-Hastings algorithm based on the sufficient statistic of the Potts model. The resulting unsupervised segmentation method is successfully applied to tridimensional ultrasound images. Finally, the problem of computing the Cramer-Rao bound (CRB) of β is studied. The CRB depends on the derivatives of the intractable normalizing constant of the Potts model. This is resolved by proposing an original Monte Carlo algorithm, which is successfully applied to compute the CRB of the Ising and Potts models.
IEEE Journal of Selected Topics in Signal Processing, 2016
IEEE Journal of Selected Topics in Signal Processing, 2016
Modern signal processing (SP) methods rely very heavily on probability and statistics to solve ch... more Modern signal processing (SP) methods rely very heavily on probability and statistics to solve challenging SP problems. SP methods are now expected to deal with ever more complex models, requiring ever more sophisticated computational inference techniques. This has driven the development of statistical SP methods based on stochastic simulation and optimization. Stochastic simulation and optimization algorithms are computationally intensive tools for performing statistical inference in models that are analytically intractable and beyond the scope of deterministic inference methods. They have been recently successfully applied to many difficult problems involving complex statistical models and sophisticated (often Bayesian) statistical inference techniques. This survey paper offers an introduction to stochastic simulation and optimization methods in signal and image processing. The paper addresses a variety of high-dimensional Markov chain Monte Carlo (MCMC) methods as well as deterministic surrogate methods, such as variational Bayes, the Bethe approach, belief and expectation propagation and approximate message passing algorithms. It also discusses a range of optimization methods that have been adopted to solve stochastic problems, as well as stochastic methods for deterministic optimization. Subsequently, areas of overlap between simulation and optimization, in particular optimization-within-MCMC and MCMC-driven optimization are discussed.
Statistics and Computing, 2015
This paper presents a new Metropolis-adjusted Langevin algorithm (MALA) that uses convex analysis... more This paper presents a new Metropolis-adjusted Langevin algorithm (MALA) that uses convex analysis to simulate efficiently from high-dimensional densities that are log-concave, a class of probability distributions that is widely used in modern high-dimensional statistics and data analysis. The method is based on a new first-order approximation for Langevin diffusions that exploits log-concavity to construct Markov chains with favourable convergence properties. This approximation is closely related to Moreau-Yoshida regularisations for convex functions and uses proximity mappings instead of gradient mappings to approximate the continuoustime process. The proposed method complements existing MALA methods in two ways. First, the method is shown to have very robust stability properties and to converge geometrically for many target densities for which other MALA are not geometric, or only if the step size is sufficiently small. Second, the method can be applied to high-dimensional target densities that are not continuously differentiable, a class of distributions that is increasingly used in image processing and machine learning and that is beyond the scope of existing MALA and HMC algorithms. To use this method it is necessary to compute or to approximate efficiently the proximity mappings of the logarithm of the target density. For several popular models, including many Bayesian models used in modern signal and image processing and machine learning, this can be achieved with convex optimisation algorithms and with approximations based on proximal splitting techniques, which can be implemented in parallel. The proposed method is demonstrated on two challenging highdimensional and non-differentiable models related to image
ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, 2011
This paper addresses the problem of estimating the statistical distribution of multiple-tissue no... more This paper addresses the problem of estimating the statistical distribution of multiple-tissue non-stationary ultrasound images of skin. The distribution of multiple-tissue images is modeled as a finite mixture of Heavy-Tailed Rayleigh distributions. An original Bayesian algorithm combined with a Markov chain Monte Carlo method is then derived to jointly estimate the mixture parameters and a label vector associating each voxel to a tissue. Precisely, a hybrid Metropolis-within-Gibbs sampler is proposed to draw samples that are asymptotically distributed according to the posterior distribution of the Bayesian model. These samples are then used to compute the Bayesian estimators of the model parameters. Simulation results are conducted on synthetic data to illustrate the performance of the proposed estimation strategy. The method is then successfully applied to the detection of an in-vivo skin lesion in a high frequency 3D ultrasound image.
Uploads
Papers by Marcelo Pereyra