Sparse representation

Deep convolutional neural networks (CNNs) have become one of the most successful methods for image processing tasks in past few years. Recent studies on modern residual architectures, enabling CNNs to be much deeper, have achieved much better results thanks to their high expressive ability by numerous parameters. In general, CNNs are known to have the robustness to the small parallel shift of objects in images by their local receptive fields, weight parameters shared by each unit, and pooling layers sandwiching them. However, CNNs have a limited robustness to the other geometric transformations such as scaling and rotation, and this lack becomes an obstacle to performance improvement even now. This paper proposes a novel network architecture, the weight-shared multi-stage network (WSMS-Net), and focuses on acquiring the scale invariance by constructing of multiple stages of CNNs. The WSMS-Net is easily combined with existing deep CNNs, enables existing deep CNNs to acquire a robustness to the scaling, and therefore, achieves higher classification accuracy on CIFAR-10, CIFAR-100 and ImageNet datasets.

Discrete return (DR) Laser Detection and Ranging (Ladar) systems provide a series of echoes that reflect from objects in a scene. These can be first, last or multi-echo returns. In contrast, Full-Waveform (FW)-Ladar systems measure the intensity of light reflected from objects continuously over a period of time. In a camouflaged scenario, e.g., objects hidden behind dense foliage, a FW-Ladar penetrates such foliage and returns a sequence of echoes including buried faint echoes. The aim of this paper is to learn local-patterns of co-occurring echoes characterised by their measured spectra. A deviation from such patterns defines an abnormal event in a forest/tree depth profile. As far as the authors know, neither DR or FW-Ladar, along with several spectral measurements, has not been applied to anomaly detection. This work presents an algorithm that allows detection of spectral and temporal anomalies in FW-Multi Spectral Ladar (FW-MSL) data samples. An anomaly is defined as a full waveform temporal and spectral signature that does not conform to a prior expectation, represented using a learnt subspace (dictionary) and set of coefficients that capture co-occurring local-patterns using an overlapping temporal window. A modified optimization scheme is proposed for subspace learning based on stochastic approximations. The objective function is augmented with a discriminative term that represents the subspace's separability properties and supports anomaly characterisation. The algorithm detects several man-made objects and anomalous spectra hidden in a dense clutter of vegetation and also allows tree species classification.

We are given an image I and a library of templates L, such that L is an overcomplete basis for I. The templates can represent objects, faces, features, analytical functions, or be single pixel templates (canonical templates). There are infinitely many ways to decompose I as a linear combination of the library templates. Each decomposition defines a representation for the image I, given L. What is an optimal representation for I given L and how to select it? We are motivated to select a sparse/compact representation for I, and to account for occlusions and noise in the image. We present a concave cost function criterion on the linear decomposition coefficients that satisfies our requirements. More specifically, we study a "weighted L p norm" with 0 < p < 1. We prove a result that allows us to generate all local minima for the L p norm, and the global minimum is obtained by searching through the local ones. Due to the computational complexity, i.e., the large number of local minima, we also study a greedy and iterative "weighted L p Matching Pursuit" strategy.

The ability to adapt language models to specific domains from large generic text corpora is of considerable interest to the language modeling community. One of the key challenges is to identify the text material relevant to a domain in the generic text collection. The text selection problem can be cast in a semi-supervised learning framework where the initial hypothesis from a speech recognition system is used to identify relevant training material. We present a novel sparse representation formulation which selects a sparse set of relevant sentences from the training data which match the test set distribution. In this formulation, the training sentences are treated as the columns of the sparse representation matrix and the n-gram counts as the rows. The target vector is the n-gram probability distribution for the test data. A sparse solution to this problem formulation identifies a few columns which can best represent the target test vector, thus identifying the relevant set of sentences from the training data. Rescoring results with the language model built from the data selected using the proposed method yields modest gains on the English broadcast news RT-04 task, reducing the word error rate from 14.6% to 14.4%.

Perceptual image quality assessment (IQA) and sparse signal representation have recently emerged as high-impact research topics in the field of image processing. Here we make one of the first attempts to incorporate the structural similarity (SSIM) index, a promising IQA measure, into the framework of optimal sparse signal representation and approximation. In particular, we introduce a novel image denoising scheme where a modified orthogonal matching pursuit algorithm is proposed for finding the best sparse coefficient vector in maximum-SSIM sense for a given set of linearly independent atoms. Furthermore, a gradient descent algorithm is developed to achieve SSIM-optimal compromise in combining the input and sparse dictionary reconstructed images. Our experimental results show that the proposed method achieves better SSIM performance and provide better visual quality than least square optimal denoising methods.

The efficiency of graph-based semi-supervised algorithms depends on the graph of instances on which they are applied. The instances are often in a vectorial form before a graph linking them is built. The construction of the graph relies on a metric over the vectorial space that help define the weight of the connection between entities. The classic choice for this metric is usually a distance measure or a similarity measure based on the euclidean norm. We claim that in some cases the euclidean norm on the initial vectorial space might not be the more appropriate to solve the task efficiently. We propose an algorithm that aims at learning the most appropriate vectorial representation for building a graph on which the task at hand is solved efficiently.

At the heart of non-parametric spectral estimation, lies the dilemma known as the bias-variance trade-off : low-biased estimators tend to have high variance and low variance estimators tend to have high bias. In 1982, Thomson introduced a multitaper method where this trade-off is made explicit by choosing a target bias resolution and obtaining a corresponding variance reduction. The method became the standard in many applications. Its favorable bias-variance trade-off is due to an empirical fact, conjectured by Thomson based on numerical evidence: assuming bandwidth W and N time domain observations, the average of the square of the first K = 2N W Slepian functions approaches, as K grows, an ideal band-pass kernel for the interval [−W, W ]. We provide an analytic proof of this fact and quantify the approximation error in the L 1 norm; the approximation error is then used to control the bias of the multitaper estimator resulting from spectral leakage. This leads to new performance bounds for the method, explicit in terms of the bandwidth W and the number N of time domain observations. Our method is flexible and can be extended to higher dimensions and different geometries.

Video streaming in large-scale multihop wireless networks of embedded devices is still open and largely unexplored. In fact, traditional video streaming systems based on transmitting predictively-encoded video through a layered communication protocol stack suffer from high complexity at the encoder and low resiliency to channel errors. The WMSN designs a cross-layer system that jointly controls the video encoding rate, the transmission rate, and the channel coding rate to maximize the received video quality. First, compressed sensing based video encoding for transmission over WI-FI multimedia sensor networks (WMSNs) is studied. It is shown that compressed sensing can overcome many of the current problems of video over WMSNs, primarily encoder complexity and low resiliency to channel errors. It is shown that the rate of compressed sensed video can be predictably controlled by varying only the compressive sensing sampling rate. It is then shown that the developed rate controller can be interpreted as the iterative solution to a convex optimization problem representing the optimization of the rate allocation across the network. The rate controller is shown to outperform existing TCP-friendly rate control schemes in terms of both fairness and received video quality.

Fast and accurate algorithms are essential for the efficient search and retrieval of the huge amount of video data that is generated for different purposes and applications every day. The interesting properties of sparse representation and the new sampling theory named Compressive Sensing (CS) constitute the core of the new approach to video representation and retrieval we are presenting in this paper to deal with the search of noisy video clips with also possibly missing frames. Once the representation (where sparsity is ...

This paper introduces an elemental building block which combines Dictionary Learning and Dimension Reduction (DRDL). We show how this foundational element can be used to iteratively construct a Hierarchical Sparse Representation (HSR) of a sensory stream. We compare our approach to existing models showing the generality of our simple prescription. We then perform preliminary experiments using this framework, illustrating with the example of an object recognition task using standard datasets. This work introduces the very first steps towards an integrated framework for designing and analyzing various computational tasks from learning to attention to action. The ultimate goal is building a mathematically rigorous, integrated theory of intelligence.

Recently, great attention was intended toward overcomplete dictionaries and the sparse representations they can provide. In a wide variety of signal processing problems, sparsity serves a crucial property leading to high performance. Decomposition of a given signal over two or more dictionaries with sparse coefficients is investigated in this paper. This kind of decomposition is useful in many applications such as inpainting, denoising, demosaicing, speech source separation, high-quality zooming and so on. This paper addresses a novel technique of such a decomposition and investigates this idea through inpainting of images which is the process of reconstructing lost or deteriorated parts of images or videos.When samples are missed in an image, the the original sparsity level in representing coefficients is changed, so with an iterative method we can estimate the original level. Simulations are presented to demonstrate the validation of our approach.

Sparse representations have emerged as a powerful tool in signal and information processing, culminated by the success of new acquisition and processing techniques such as Compressed Sensing (CS). Fusion frames are very rich new signal representation methods that use collections of subspaces instead of vectors to represent signals. This work combines these exciting fields to introduce a new sparsity model

Anisotropic representation systems such as curvelets and shearlets have had a significant impact on applied mathematics in the last decade. The main reason for their success is their superior ability to optimally resolve anisotropic structures such as singularities concentrated on lower dimensional embedded manifolds, for instance, edges in images or shock fronts in solutions of transport dominated equations. By now, a large variety of such anisotropic systems has been introduced, for instance, second generation curvelets, bandlimited shearlets, and compactly supported shearlets, all based on a parabolic dilation operation. These systems share similar approximation properties, which is usually proven on a case-by-case basis for each different construction. The novel concept of parabolic molecules, which was recently introduced by two of the authors, allows for a unified framework encompassing all known anisotropic frame constructions based on parabolic scaling. The main result essentially states that all such systems share similar approximation properties. One main consequence is that at once all the desirable approximation properties of one system within this framework can be deduced for virtually any other system based on parabolic scaling. The present paper motivates and surveys recent results in this direction.

We study the exact recovery of signals from quantized frame coefficients. Here, the basis of the quantization is hard thresholding, and we present a simple algorithm for the recovery of reconstructable signals. The set of non-reconstructable signals is shown to be star-shaped and symmetric with respect to the origin. Moreover, we provide a criterion on the frame for the boundedness of this set. In this case, we also give a priori bounds.

In this paper, we propose a time-frequency analysis method to obtain instantaneous frequencies and the corresponding decomposition by solving an optimization problem. In this optimization problem, the basis to decompose the signal is not known a priori. Instead, it is adapted to the signal and is determined as part of the optimization problem. In this sense, this optimization problem can be seen as a dictionary learning problem. This dictionary learning problem is solved by using the Augmented Lagrangian Multiplier method (ALM) iteratively. We further accelerate the convergence of the ALM method in each iteration by using the fast wavelet transform. We apply our method to decompose several signals, including signals with poor scale separation, signals with outliers and polluted by noise and a real signal. The results show that this method can give accurate recovery of both the instantaneous frequencies and the intrinsic mode functions.

In a recent paper [12], Hou and Shi introduced a new adaptive data analysis method to analyze nonlinear and non-stationary data. The main idea is to look for the sparsest representation of multiscale data within the largest possible dictionary consisting of intrinsic mode functions of the form {a(t) cos(θ(t))}, where a ∈ V (θ), V (θ) consists of the functions smoother than cos(θ(t)) and θ ′ ≥ 0. This problem was formulated as a nonlinear L 0 optimization problem and an iterative nonlinear matching pursuit method was proposed to solve this nonlinear optimization problem. In this paper, we prove the convergence of this nonlinear matching pursuit method under some sparsity assumption on the signal. We consider both well-resolved and sparse sampled signals. In the case without noise, we prove that our method gives exact recovery of the original signal.

This paper addresses the possibility of mathematically partition and process urine 1 H-NMR spectra to enhance the efficiency of the subsequent multivariate data analysis in the context of metabolic profiling of a toxicity study. We show that by processing the NMR data with the peak alignment using reduced set mapping (PARS) algorithm and the use of sparse representation of the data results in the information contained in the original NMR data being preserved with retained resolution but free of the problem of peak shifts. We can now describe a method for differential expression analysis of NMR spectra by using prior knowledge, i.e., the onset of dosing, a partitioning not possible to achieve using raw or bucketed data. In addition we also outline a scheme for soft removal of ''biological noise'' from the aligned data: exhaustive bio-noise subtraction (EBS). The result is a straightforward protocol for detection of peaks that appear as a consequence of the drug response. In other words, it is possible to elucidate peak origin, either from endogenous substances or from the administered drug/biomarkers. The partition of data originating from the normally regulating metabolome can, furthermore, be analyzed free of the superimposed biological noise. The proposed protocol results in enhanced interpretability of the processed data, i.e., a more refined metabolic trace, simplification of detection of consistent biomarkers, and a simplified search for metabolic end products of the administered drug.

Sparse Representation Classifier (SRC) and its variants were considered as powerful classifiers in the domains of computer vision and pattern recognition. However, classifying test samples is computationally expensive due to the 1 norm minimization problem that should be solved in order to get the sparse code. Therefore, these classifiers could not be the right choice for scenarios requiring fast classification. In order to overcome the expensive computational cost of SRC, a two-phase coding classifier based on classic Regularized Least Square was proposed. This classifier is more efficient than SRC. A significant limitation of this classifier is the fact that the number of the samples that should be handed over to the next coding phase should be specified a priori. This paper overcomes this main limitation and proposes five data-driven schemes allowing an automatic estimation of the optimal size of the local samples. These schemes handle the three cases that are encountered in any learning system: supervised, unsupervised, and semi-supervised. Experiments are conducted on five image datasets. These experiments show that the introduced learning schemes can improve the performance of the two-phase linear coding classifier adopting ad-hoc choices for the number of local samples.

A new concept is introduced for the adaptive finite element discretization of partial differential equations that have a sparsely representable solution. Motivated by recent work on compressed sensing, a recursive mesh refinement procedure is presented that uses linear programming to find a good approximation to the sparse solution on a given refinement level. Then only those parts of the mesh are refined that belong to large expansion coefficients. Error estimates for this procedure are refined and the behavior of the procedure is demonstrated via some simple elliptic model problems.

Three properties of matrices: the spark, the mutual incoherence and the restricted isometry property have recently been introduced in the context of compressed sensing. We study these properties for matrices that are Kronecker products and show how these properties relate to those of the factors. For the mutual incoherence we also discuss results for sums of Kronecker products.

This paper addresses the problem of Through-the-Wall Radar Imaging (TWRI) using the Multiple-Measurement Vector (MMV) compressive sensing model. TWR image formation is reformulated as a compressed sensing (CS) problem, seeking a sparse representation in the spatial domain. In traditional CS-based through-the-wall radar imaging (TWRI) methods, the measurement matrix is vectorized so that a single measurement vector (SMV) model is applied to generate a sparse solution, which represents a scene comprising pointlike targets. For multiple measurement TWRI problems, the SMV model may produce a sub-optimum sparse solution. On the other hand, the proposed MMV model for TWRI generates a more sparse scene by processing all the measurements simultaneously. To evaluate the effectiveness of the proposed method, it is applied to fuse multiple polarization data to form the radar image. Based on simulated data with different number of measurements and noise levels, the proposed MMV-based TWRI method produces better TWR images in terms of image quality and detection accuracy.

Large scale, streaming datasets are ubiquitous in modern machine learning. Streaming algorithms must be scalable, amenable to incremental training and robust to the presence of non-stationarity. In this work consider the problem of learning 1 regularized linear models in the context of streaming data. In particular, the focus of this work revolves around how to select the regularization parameter when data arrives sequentially and the underlying distribution is non-stationary (implying the choice of optimal regularization parameter is itself time-varying). We propose a framework through which to infer an adaptive regularization parameter. Our approach employs an 1 penalty constraint where the corresponding sparsity parameter is iteratively updated via stochastic gradient descent. This serves to reformulate the choice of regularization parameter in a principled framework for online learning. The proposed method is derived for linear regression and subsequently extended to generalized linear models. We validate our approach using simulated and real datasets and present an application to a neuroimaging dataset.

Large scale, streaming datasets are ubiquitous in modern machine learning. Streaming algorithms must be scalable, amenable to incremental training and robust to the presence of non-stationarity. In this work consider the problem of learning $\ell_1$ regularized linear models in the context of streaming data. In particular, the focus of this work revolves around how to select the regularization parameter when data arrives sequentially and the underlying distribution is non-stationary (implying the choice of optimal regularization parameter is itself time-varying). We propose a novel framework through which to infer an adaptive regularization parameter. Our approach employs an $\ell_1$ penalty constraint where the corresponding sparsity parameter is iteratively updated via stochastic gradient descent. This serves to reformulate the choice of regularization parameter in a principled framework for online learning and allows for the derivation of convergence guarantees in a non-stochasti...

We propose two graph matrix completion algorithms called GMCM-DL and GMCR-DL, by employing a new definition of Graph Total Variation (GTV) for matrices based on the directed Laplacian matrix. We show that these algorithms outperform their peers in terms of RMSEs for both cases of uniform and row observations.

We discuss approaches for an efficient handling of the correction equation in the Jacobi-Davidson method. The correction equation is effective in a subspace orthogonal to the current eigenvector approximation. The operator in the correction equation is a dense matrix, but it is composed from three factors that allow for a sparse representation. If the given matrix eigenproblem is sparse then one often aims for the construction of a preconditioner for that matrix. We discuss how to restrict this preconditioner effectively to the subspace orthogonal to the current eigenvector. The correction equation itself is formulated in terms of approximations for an eigenpair. In order to avoid misconvergence one has to make the right selection for the approximations, and this aspect will be discussed as well.

Sparse representations account for most or all of the information of a signal by a linear combination of a few elementary signals called atoms, and have increasingly become recognized as providing high performance for applications as diverse as noise reduction, compression, inpainting, compressive sensing, pattern classification, and blind source separation. In this dissertation, we learn the sparse representations of high-dimensional signals for various learning and vision tasks, including image classification, single image super-resolution, compressive sensing, and graph learning. Based on the bag-of-features (BoF) image representation in a spatial pyramid, we first transform each local image descriptor into a sparse representation, and then these sparse representations are summarized into a fixed-length feature vector over different spatial locations across different spatial scales by max pooling. The proposed generic image feature representation properly handles the large in-class variance problem in image classification, and experiments on object recognition, scene classification, face recognition, gender recognition, and handwritten digit recognition all lead to state-of-the-art performances on the benchmark datasets. We cast the image super-resolution problem as one of recovering a highresolution image patch for each low-resolution image patch based on recent sparse signal recovery theories, which state that, under mild conditions, a high-resolution signal can be recovered from its low-resolution version if the signal has a sparse representation in terms of some dictionary. We jointly learn the dictionaries for high-and low-resolution image patches and enforce them to have common sparse representations for better recovery. Furthermore, we employ image features and enforce patch overlapping constraints to improve prediction accuracy. Experiments show that the algorithm leads to surprisingly good results. Graph construction is critical for those graph-orientated algorithms designed for the purposes of data clustering, subspace learning, and semi-supervised learning. We model the graph construction problem, including neighbor selection and chance to work with Dr. Kai Yu as a summer intern at NEC Laboratories America. From him, I learned not only techniques and skills but also his precious research insights. I am also very thankful to Professor Narendra Ahuja and Professor Zhi-Pei Liang for their advice and comments. I am fortunate to have them as my thesis committee members. During my Ph.D. study over the last five years, it has been a great honor for me to collaborate with Hasegawa-Johnson at UIUC,

In this paper, a new and simple palmprint recognition solution based on sparse representation is suggested. It is shown that when the aim is to recover a palmprint from a limited number of observations as a linear combination of measurements of the same palmprint class, the ensuing representation in intrinsically very sparse. It can be efficiently computed by solving an l 1 norm convex minimisation problem. When combined with well known subspace feature selection techniques such as PCA and LDA as well as with downsampled images, our tests, which have been carried out on 250 classes of the widely used PolyU database, have yielded an EER as low as 0.11% depending on the palmprints selected during the enrolment phase. Coupled with an execution time as short as 8.4 ms, the obtained results outperform similar work in the literature including EigenPalms, FisherPalms and Gabor based palmprint matching algorithms, which shows the effectiveness of the new solution.

We consider network sparsification as an L0-norm regularized binary optimization problem, where each unit of a neural network (e.g., weight, neuron, or channel, etc.) is attached with a stochastic binary gate, whose parameters are jointly optimized with original network parameters. The Augment-Reinforce-Merge (ARM) [28], a recently proposed unbiased gradient estimator, is investigated for this binary optimization problem. Compared to the hard concrete gradient estimator from Louizos et al. [20], ARM demonstrates superior performance of pruning network architectures while retaining almost the same accuracies of baseline methods. Similar to the hard concrete estimator, ARM also enables conditional computation during model training but with improved effectiveness due to the exact binary stochasticity. Thanks to the flexibility of ARM, many smooth or non-smooth parametric functions, such as scaled sigmoid or hard sigmoid, can be used to parameterize this binary optimization problem and the unbiasness of the ARM estimator is retained, while the hard concrete estimator has to rely on the hard sigmoid function to achieve conditional computation and thus accelerated training. Extensive experiments on multiple public datasets demonstrate state-ofthe-art pruning rates with almost the same accuracies of baseline methods. The resulting algorithm L0-ARM sparsifies the Wide-ResNet models on CIFAR-10 and CIFAR-100 while the hard concrete estimator cannot. The code is public available at https://github.com/leo-yangli/l0-arm.

We consider tomographic reconstruction using priors in the form of a dictionary learned from training images. The reconstruction has two stages: first we construct a tensor dictionary prior from our training data, and then we pose the reconstruction problem in terms of recovering the expansion coefficients in that dictionary. Our approach differs from past approaches in that a) we use a third-order tensor representation for our images and b) we recast the reconstruction problem using the tensor formulation. The dictionary learning problem is presented as a non-negative tensor factorization problem with sparsity constraints. The reconstruction problem is formulated in a convex optimization framework by looking for a solution with a sparse representation in the tensor dictionary. Numerical results show that our tensor formulation leads to very sparse representations of both the training images and the reconstructions due to the ability of representing repeated features compactly in the dictionary.

There are several models for sparse approximation: one where a signal is a sparse linear combination of vectors over a redundant dictionary and a second model in which a collection of signals is a simultaneous sparse linear combination over a single dictionary. In this work, interpolate between these two models to synthesize a single signal of interest from K highly incoherent dictionaries while enforcing simultaneous sparsity on the K resulting coefficient vectors. We define this as the parallel approximation problem, which arises quite naturally in many applications such as MRI parallel excitation using multiple transmission coils. We present an efficient algorithm to solve the parallel approximation problem called Parallel Orthogonal Matching Pursuit (POMP). We prove its correctness in a general setting and then discuss adaptations needed to make it suitable for use in an MRI parallel excitation setting. We then discuss parallel excitation in more detail and demonstrate how POMP solves the problem as accurately, but much faster, than previously proposed convex optimization methods.

The multicategory SVM (MSVM) of Lee et al. (2004) is a natural generalization of the classical, binary support vector machines (SVM). However, its use has been limited by computational difficulties. The simplex-cone SVM (SCSVM) of Mroueh et al. (2012) is a computationally efficient multicategory classifier, but its use has been limited by a seemingly opaque interpretation. We show that MSVM and SCSVM are in fact exactly equivalent, and provide a bijection between their tuning parameters. MSVM may then be entertained as both a natural and computationally efficient multicategory extension of SVM. We further provide a Donsker theorem for finite-dimensional kernel MSVM and partially answer the open question pertaining to the very competitive performance of One-vs-Rest methods against MSVM. Furthermore, we use the derived asymptotic covariance formula to develop an inverse-variance weighted classification rule which improves on the One-vs-Rest approach.

In recent years, the enormous demand for computing resources resulting from massive data and complex network models has become the limitation of deep learning. In the large-scale problems with massive samples and ultrahigh feature dimensions, sparsity has gradually drawn much attention from academia and the industrial field. In this article, the new generation of braininspired sparse learning is reviewed comprehensively. First, sparse cognition learning is introduced from the visual biology mechanism to modeling for the natural image. Second, the sparse representation algorithms are summarized to sort out the research progress of sparse learning. Third, the relevant research on sparse feature selection learning is reviewed. Then, the sparse deep networks and applications are summed up. Last but not least, ten public issues and challenges of sparse learning are discussed. By investigating the development process of sparse learning, this article summarizes the advantages, disadvantages, limitations, and future research directions of the algorithm, which can help readers conduct further study. Impact Statement-Sparse representation is an essential branch of machine learning. The sparsity has been considered to solve the limitation of computation resources in practical deep learning applications. Unlike previous reviews, our survey comprehensively...

Feature representation has been widely used and developed recently. Multiscale features have led to remarkable breakthroughs for representation learning process in many computer vision tasks. This paper aims to provide a comprehensive survey of the recent multiscale representation learning achievements in classification tasks. Multiscale representation learning methods can be divided into two broad categories (multiscale geometric analysis and multiscale networks). Eleven kinds of multiscale geometric tools and seven kinds of multiscale networks are introduced. Some corresponding fundamental subproblems of these two broad categories are also described, including some concepts in representation process, specific representation methods with multiscale geometric analysis, and multiscale representation design strategies for networks. Then, the correlation between these two broad categories is illustrated, including respective characteristics, combination strategies, and characteristics of optimal representation. Some datasets and evaluation results are included to verify the effectiveness of the multiscale representation learning. Eventually, conclusion and future work are given, covering four directions [a) Choice and fusion; b) Self-adaption; c) Structure; and d) Generalization and proof]. Impact Statement-Multiscale representation learning techniques are fully utilized in existing artificial intelligence tasks. Compared to the existing papers, which pay attention to a particular task or algorithm, this paper is the first systematical review of multiscale

This paper addresses the problem of efficient SIFTbased image description and searches in large databases within the framework of local querying. A descriptor called the bagof-features has been introduced in [1] which first vector quantizes SIFT descriptors and then aggregates the set of resulting codeword indices (so-called visual words) into a histogram of occurence of the different visual words in the image. The aim is to make the image search complexity tractable by transforming the set of local image descriptor vectors into a single sparse vector as sparsity particularly permits efficient inner product calculations. However, aggregating local descriptors into a single histogram decreases the discerning power of the system when performing local queries. In this paper, we propose a new approach that aims to enjoy the complexity benefits of sparsity while at the same time retaining the local quality of the input descriptor vectors. This is accomplished by searching for a sparse approximation of the input SIFT descriptors. The sparse approximation yields a sparse vector per local SIFT descriptor, and helps preserving local description properties by using each sparse-transformed descriptor independently in a voting system to retrieve indexed images. Our system is shown experimentally to perform better than histogram based systems under query locality, albeit at an increased complexity.

We present a new, block-based image codec based on sparse representations using a learned, structured dictionary called the Iteration-Tuned and Aligned Dictionary (ITAD). The question of selecting the number of atoms used in the representation of each image block is addressed with a new, global (image-wide), rate-distortion-based sparsity selection criterion. We show experimentally that our codec outperforms JPEG2000 in both quantitative evaluations (by 0.9 dB to 4 dB) and qualitative evaluations.

We introduce a new dictionary structure for sparse representations better adapted to pursuit algorithms used in practical scenarios. The new structure, which we call an Iteration-Tuned Dictionary (ITD), consists of a set of dictionaries each associated to a single iteration index of a pursuit algorithm. In this work we first adapt pursuit decompositions to the case of ITD structures and then introduce a training algorithm used to construct ITDs. The training algorithm consists of applying a K-means to the (i − 1)-th residuals of the training set to thus produce the i-th dictionary of the ITD structure. In the results section we compare our algorithm against the state-of-the-art dictionary training scheme and show that our method produces sparse representations yielding better signal approximations for the same sparsity level.