Computer Science

Blind sampling is a sampling scheme which works without any knowledge about the image except for the measurements it obtains. An adaptive blind sampling scheme makes use of that knowledge to wisely choose the next sample. In this work we consider generalised sampling, where each measurement is obtained by an inner product between the image and some mask.
In the first part of this work, we discuss the reconstruction of images from arbitrary generalised samples, under second-order statistical models. We consider algebraic and statistical schemes for reconstruction and the connection between them. We then show an optimal nonadaptive method for choosing generalised sampling masks in this context.
In the second part, we limit ourselves to a particular set of wavelet masks and propose a blind adaptive progressive sampling scheme. The scheme estimates the magnitudes of the unsampled wavelet coefficients and samples the coefficients with larger estimated magnitude first. With this method, the selected masks extract the image information much more efficiently than nonadaptive schemes and get closer to the optimal, non-blind results.

We consider the task of sequential point sampling for three-dimensional structure reconstruction and focus on terrestrial topographic mapping using a laser range scanner. Both the sampling and the reconstruction rely on a stochastic model of the sampled object. We describe several algorithms for sequential point sampling including a new adaptive algorithm that is specifically designed for mechanical devices and produces grid-like sampling patterns. Experimental results verify that relying on the stochastic model indeed leads to efficient sampling associated with accurate surface reconstruction.

Blind sampling is a sampling scheme which uses no knowledge about the image except for the measurements it obtains. An adaptive blind sampling scheme makes use of that knowledge to wisely choose the next sample. In this work we consider generalised sampling, where each measurement is obtained by computing an inner product between the image and some mask. The context of this paper is second order statistical models for images. We discuss the reconstruction of images from arbitrary generalised sampling and present criteria for selecting an optimal mask from a dictionary (family) of masks, or from the set of all possible masks. The latter case leads to PCA. We further propose adaptive sampling schemes that produce different sets of sampling masks for different images, and experimentally verify the advantage of the adaptive schemes over nonadaptive ones.

Adaptive sampling schemes choose different sampling masks for different images. Blind adaptive sampling schemes use the measurements they obtain (without any additional or direct knowledge about the image) to wisely choose the next sample mask.
In this paper we present and discuss two blind adaptive sampling schemes. The first is a general scheme not restricted to a specific class of sampling functions. It is based on an underlying statistical model for the image, which is updated according to the available measurements. A second, less general but more practical method, uses the wavelet decomposition of an image. It estimates the magnitude of the unsampled wavelet coefficients and samples those with larger estimated magnitude first. Experimental results show the benefits of the proposed blind sampling schemes.

Among the many ways to model signals, a recent approach that draws considerable attention is sparse representation modeling. In this model, the signal is assumed to be generated as a random linear combination of a few atoms from a pre-specified dictionary. In this work we analyze two Bayesian denoising algorithms -the Maximum-Aposteriori Probability (MAP) and the Minimum-Mean-Squared-Error (MMSE) estimators, under the assumption that the dictionary is unitary. It is well known that both these estimators lead to a scalar shrinkage on the transformed coefficients, albeit with a different response curve. In this work we start by deriving closed-form expressions for these shrinkage curves and then analyze their performance. Upper bounds on the MAP and the MMSE estimation errors are derived. We tie these to the error obtained by a so-called oracle estimator, where the support is given, establishing a worst-case gain-factor between the MAP/MMSE estimation errors and the oracle's performance. These denoising algorithms are demonstrated on synthetic signals and on true data (images).

The scale of functional magnetic resonance image data is rapidly increasing as large multi-subject datasets are becoming widely available and high-resolution scanners are adopted. The inherent low-dimensionality of the information in this data has led neuroscientists to consider factor analysis methods to extract and analyze the underlying brain activity. In this work, we consider two recent multi-subject factor analysis methods: the Shared Response Model and Hierarchical Topographic Factor Analysis. We perform analytical, algorithmic, and code optimization to enable multi-node parallel implementations to scale. Single-node improvements result in 99× and 1812× speedups on these two methods, and enables the processing of larger datasets. Our distributed implementations show strong scaling of 3.3× and 5.5× respectively with 20 nodes on real datasets. We also demonstrate weak scaling on a synthetic dataset with 1024 subjects, on up to 1024 nodes and 32,768 cores.

The sparse synthesis model for signals has become very popular in the last decade, leading to improved performance in many signal processing applications. This model assumes that a signal may be described as a linear combination of few columns (atoms) of a given synthesis matrix (dictionary). The Co-Sparse Analysis model is a recently introduced counterpart, whereby signals are assumed to be orthogonal to many rows of a given analysis dictionary. These rows are called the co-support. The Analysis model has already led to a series of contributions that address the pursuit problem: identifying the co-support of a corrupted signal in order to restore it. While all the existing work adopts a deterministic point of view towards the design of such pursuit algorithms, this paper introduces a Bayesian estimation point of view, starting with a random generative model for the co-sparse analysis signals. This is followed by a derivation of Oracle, Minimum-Mean-Squared-Error (MMSE), and Maximum-A-posteriori-Probability (MAP) based estimators. We present a comparison between the deterministic formulations and these estimators, drawing some connections between the two. We develop practical approximations to the MAP and MMSE estimators, and demonstrate the proposed reconstruction algorithms in several synthetic and real image experiments, showing their potential and applicability.

In ultrasound, second harmonic imaging is usually preferred due to the higher clutter artifacts and speckle noise common in the fundamental frequency image. Typical approaches use either one or the other image, applying corresponding filters for each case. In this work we propose a method based on a joint sparsity model that fuses the first and second harmonic images while performing clutter mitigation and noise reduction. Our approach, Fused-MCA, uses two adaptive dictionaries for characterizing the clutter components in each image, and a common dictionary for the tissue representation. Our results indicate that the resulting images contain less clutter artifacts, less speckle noise and present some of the benefits of both harmonic input images.

We consider the problem of estimating the inverse of a covariance matrix of a normal distribution, assuming that it is sparse. To this end, an $l_1$ regularized log-determinant optimization problem is solved. We present a multilevel framework for accelerating existing solvers of this problem. Taking advantage of the sparseness of the matrix, we create a multilevel hierarchy of similar problems, which are traversed in order to accelerate the optimization process. Our numerical experiments demonstrate the efficiency of the multilevel framework for solving both medium and large scale instances of this problem.

The sparse inverse covariance estimation problem arises in many statistical applications in machine learning and signal processing. In this problem, the inverse of a covariance matrix of a multivariate normal distribution is estimated, assuming that it is sparse. An $\ell_1$ regularized log-determinant optimization problem is typically solved to approximate such matrices. Because of memory limitations, most existing algorithms are unable to handle large scale instances of this problem. In this paper we present a new block-coordinate descent approach for solving the problem for large-scale data sets. Our method treats the sought matrix block-by-block using quadratic approximations, and we show that this approach has advantages over existing methods in several aspects. Numerical experiments on both synthetic and real gene expression data demonstrate that our approach outperforms the existing state of the art methods, especially for large-scale problems.

Clutter is an artifact in cardiac ultrasound that obscures parts of the heart. A cluttered signal is seen as a superposition of tissue, clutter and noise components. In this work, we introduce two novel methods for reducing clutter by separating these components using Morphological Component Analysis, where each component has a sparse representation under some dictionary. The clutter dictionary is trained using data acquired from right side of the chest, overcoming any assumption about the clutter behavior. The tissue dictionary is trained from off-line tissue data in one method, and adaptively from the patient data in the other. These methods are shown to be robust to the input data characteristics and yield state-of-the-art performance.

In ultrasound imaging, clutter artifacts degrade images and may cause inaccurate diagnosis. In this paper, we apply a method called Morphological Component Analysis (MCA) for sparse signal separation with the objective of reducing such clutter artifacts. The MCA approach assumes that the two signals in the additive mix have each a sparse representation under some dictionary of atoms (a matrix), and separation is achieved by finding these sparse representations. In our work, an adaptive approach is used for learning the dictionary from the echo data. MCA is compared to Singular Value Filtering (SVF), a Principal Component Analysis- (PCA-) based filtering technique, and to a high-pass Finite Impulse Response (FIR) filter. Each filter is applied to a simulated hypoechoic lesion sequence, as well as experimental cardiac ultrasound data. MCA is demonstrated in both cases to outperform the FIR filter and obtain results comparable to the SVF method in terms of contrast-to-noise ratio (CNR)...

The field of machine programming (MP), the automation of the development of software, is making notable research advances. This is, in part, due to the emergence of a wide range of novel techniques in machine learning. In this paper, we apply MP to the automation of software performance regression testing. A performance regression is a software performance degradation caused by a code change. We present AutoPerf - a novel approach to automate regression testing that utilizes three core techniques: (i) zero-positive learning, (ii) autoencoders, and (iii) hardware telemetry. We demonstrate AutoPerf's generality and efficacy against 3 types of performance regressions across 10 real performance bugs in 7 benchmark and open-source programs. On average, AutoPerf exhibits 4% profiling overhead and accurately diagnoses more performance bugs than prior state-of-the-art approaches. Thus far, AutoPerf has produced no false negatives.

Recent work has shown that topological enhancements to recurrent neural networks (RNNs) can increase their expressiveness and representational capacity. Two popular enhancements are stacked RNNs, which increases the capacity for learning non-linear functions, and bidirectional processing, which exploits acausal information in a sequence. In this work, we explore the delayed-RNN, which is a single-layer RNN that has a delay between the input and output. We prove that a weight-constrained version of the delayed-RNN is equivalent to a stacked-RNN. We also show that the delay gives rise to partial acausality, much like bidirectional networks. Synthetic experiments confirm that the delayed-RNN can mimic bidirectional networks, solving some acausal tasks similarly, and outperforming them in others. Moreover, we show similar performance to bidirectional networks in a real-world natural language processing task. These results suggest that delayed-RNNs can approximate topologies including st...

The problem of automatic software generation is known as Machine Programming. In this work, we propose a framework based on genetic algorithms to solve this problem. Although genetic algorithms have been used successfully for many problems, one criticism is that hand-crafting its fitness function, the test that aims to effectively guide its evolution, can be notably challenging. Our framework presents a novel approach to learn the fitness function using neural networks to predict values of ideal fitness functions. We also augment the evolutionary process with a minimally intrusive search heuristic. This heuristic improves the framework's ability to discover correct programs from ones that are approximately correct and does so with negligible computational overhead. We compare our approach with several state-of-the-art program synthesis methods and demonstrate that it finds more correct programs with fewer candidate program generations.

There is a growing interest in joint multi-subject fMRI analysis. The challenge of such analysis comes from inherent anatomical and functional variability across subjects. One approach to resolving this is a shared response factor model. This assumes a shared and time synchronized stimulus across subjects. Such a model can often identify shared information, but it may not be able to pinpoint with high resolution the spatial location of this information. In this work, we examine a searchlight based shared response model to identify shared information in small contiguous regions (searchlights) across the whole brain. Validation using classification tasks demonstrates that we can pinpoint informative local regions.

Sequential information contains short- to long-range dependencies; however, learning long-timescale information has been a challenge for recurrent neural networks. Despite improvements in long short-term memory networks (LSTMs), the forgetting mechanism results in the exponential decay of information, limiting their capacity to capture long-timescale information. Here, we propose a power law forget gate, which instead learns to forget information along a slower power law decay function. Specifically, the new gate learns to control the power law decay factor, p, allowing the network to adjust the information decay rate according to task demands. Our experiments show that an LSTM with power law forget gates (pLSTM) can effectively capture long-range dependencies beyond hundreds of elements on image classification, language modeling, and categorization tasks, improving performance over the vanilla LSTM. We also inspected the revised forget gate by varying the initialization of p, setti...