Tagucghi Loss Function

We consider the problem of training discriminative structured output predictors, such as conditional random fields (CRFs) and structured support vector machines (SSVMs). A generalized loss function is introduced, which jointly maximizes the entropy and the margin of the solution. The CRF and SSVM emerge as special cases of our framework. The probabilistic interpretation of large margin methods reveals insights about margin and slack rescaling. Furthermore, we derive the corresponding extensions for latent variable models ...

This paper studies the proposition that an inflation bias can arise in a setup where a central banker with asymmetric preferences targets the natural unemployment rate. Preferences are asymmetric in the sense that positive unemployment deviations from the natural rate are weighted more (or less) severely than negative deviations in the central banker's loss function. The bias is proportional to

The Baysian estimation of the mean vector θ of a p-variate normal distribution under linear exponential (LINEX) loss function is studied when as a special restricted model, it is suspected that for a p × r known matrix Z the hypothesis θ = Zβ, β ∈ r may hold. In this area we show that the Bayes and empirical Bayes estimators dominate the unrestricted estimator (when nothing is known about the mean vector θ ).

A general framework is proposed for gradient boosting in supervised learning problems where the loss function is defined using a kernel over the output space. It extends boosting in a principled way to complex output spaces (images, text, graphs etc.) and can be applied to a general class of base learners working in kernelized output spaces. Empirical results are provided on three problems: a regression problem, an image completion task and a graph prediction problem. In these experiments, the framework is combined with tree-based base learners, which have interesting algorithmic properties. The results show that gradient boosting significantly improves these base learners and provides competitive results with other tree-based ensemble methods based on randomization.

We consider the problem of numerical stability and model density growth when training a sparse linear model from massive data. We focus on scalable algorithms that optimize certain loss function using gradient descent, with either ℓ0 or ℓ1 regularization. We observed numerical stability problems in several existing methods, leading to divergence and low accuracy. In addition, these methods typically have weak controls over sparsity, such that model density grows faster than necessary. We propose a framework to address the above problems. First, the update rule is numerically stable with convergence guarantee and results in more reasonable models. Second, besides ℓ1 regularization, it exploits the sparsity of data distribution and achieves a higher degree of sparsity with a PAC generalization error bound. Lastly, it is parallelizable and suitable for training large margin classifiers on huge datasets. Experiments show that the proposed method converges consistently and outperforms other baselines using 10% of features by as much as 6% reduction in error rate on average. Datasets and software are available from the authors.

This paper presents Bayesian estimation of the survival function of the Pareto distribution of the second kind using the methods of and . A numerical example is given to illustrate the results derived. Based on a Monte Carlo simulation study, comparisons are made between these two methods, as well as, their competitor, the maximum likelihood method, by considering di erent censored samples and several values of the true shape and scale parameters.

Metric and kernel learning are important in several machine learning applications. However, most existing metric learning algorithms are limited to learning metrics over low-dimensional data, while existing kernel learning algorithms are often limited to the transductive setting and do not generalize to new data points. In this paper, we study metric learning as a problem of learning a linear transformation of the input data. We show that for high-dimensional data, a particular framework for learning a linear transformation of the data based on the LogDet divergence can be efficiently kernelized to learn a metric (or equivalently, a kernel function) over an arbitrarily high dimensional space. We further demonstrate that a wide class of convex loss functions for learning linear transformations can similarly be kernelized, thereby considerably expanding the potential applications of metric learning. We demonstrate our learning approach by applying it to large-scale real world problems in computer vision and text mining.

This paper aims to conduct a study on the listwise approach to learning to rank. The listwise approach learns a ranking function by taking individual lists as instances and minimizing a loss function defined on the predicted list and the ground-truth list. Existing work on the approach mainly focused on the development of new algorithms; methods such as RankCosine and ListNet have been proposed and good performances by them have been observed. Unfortunately, the underlying theory was not sufficiently studied so far. To amend the problem, this paper proposes conducting theoretical analysis of learning to rank algorithms through investigations on the properties of the loss functions, including consistency, soundness, continuity, differentiability, convexity, and efficiency. A sufficient condition on consistency for ranking is given, which seems to be the first such result obtained in related research. The paper then conducts analysis on three loss functions: likelihood loss, cosine loss, and cross entropy loss. The latter two were used in RankCosine and ListNet. The use of the likelihood loss leads to the development of

Bandwidth selection in multivariate kernel density estimation has received considerable attention. In addition to classical methods of bandwidth selection, such as plug-in and cross-validation methods, Bayesian approaches have also been previously investigated. Bayesian estimation of adaptive bandwidth matrices in multivariate kernel density estimation are investigated, when the quadratic and entropy loss functions are used. Under the quadratic loss function, the proposed method is evaluated through a simulation study and two real datasets, which were already discussed in the literature. For these real-data applications, very interesting advantages of the proposed method are pointed out.

The decision-theoretic approach to statistics and econometrics explicitly specifies a set of models under consideration, a set of actions that can be taken, and a loss function that quantifies the value to the decision-maker of applying a particular action when a particular model holds. Decision rules, or procedures, map data into actions, and can be ordered according to their Bayes, minmax, or minmax regret risks. In complicated decision problems, large sample approximations can be used to approximate the decision problem by a simpler one which is easier to solve. Some examples of applications of decision theory in econometrics are discussed.

In this paper, we consider the minimization of the conditional value-at-risk (CVaR), a most preferable risk measure in financial risk management, in the context of the well-known single-period newsvendor problem, which is originally formulated as the maximization of the expected profit or the minimization of the expected cost. We show that downside risk measures including the CVaR are tractable in the problem due to their convexity, and consequently, under mild assumptions on the probability distribution of products' demand, we provide analytical solutions or linear programming (LP) formulation of the minimization of the CVaR measures defined with two different loss functions. Numerical examples are also exhibited, clarifying the difference among the models analyzed in this paper, and demonstrating the efficiency of the LP solutions.

Taguchi loss function is a diagram of the loss for the company that actual results differ from a target value. Taguchi loss function is intended to capture not only the loss to the customer, but to society and society at large which can be measured by cost. The loss function is not to grasp what cannot be quantified. For that reason the loss of function includes increased pollution as a cost to society. In general, the loss function is shown as a parabola at the point of minimum loss directly above the target value. As she is drawn as a parabola symmetrical in most instances it is not necessarily as the loss function in a real case would be that the actual loss and could be discontinuous and asymmetric. An extremely important concept is the function that displays the weakness of the use of specifications that deal with any outcome between the values that are not rejected as equal. More often while the customer can accept an outcome in the specification they become less satisfied and more likely to find another supplier, as results leave their desired target. This concept has even more value when looking at the loss if it can be measured. While it would be nice to be able to measure all the costs it is often not possible and is worth including known but immeasurable losses in decision-making. And while paying attention to the value of the loss to society is it is also logical to calculate the loss function with respect to all customer and supplier. Here is a detailed analysis and examination costs Taguchi, philosophy and the Taguchi loss function.

This article presents a practical tool for optimally scheduling wine grape harvesting operations taking into account both operational costs and grape quality. We solve a mixed-integer linear programming model to support harvest scheduling, labor allocation, and routing decisions. A quality loss function is used to represent wine quality reduction at each vineyard block due to premature or deferred harvest with respect to an optimal date. We present computational results which show that the proposed tool could be used to support grape harvest planning in a large vineyard, at both a tactical and operational level. r

We analyze the predictive performance of various volatility models for stock returns. To compare their performance, we choose loss functions for which volatility estimation is of paramount importance. We deal with two economic loss functions (an option pricing function and an utility function) and two statistical loss functions (a goodness-of-fit measure for a Value-at-Risk (VaR) calculation and a predictive likelihood function). We implement the tests for superior predictive ability of White and Hansen . We find that, for option pricing, simple models like the Riskmetrics exponentially weighted moving average (EWMA) or a simple moving average, which do not require estimation, perform as well as other more sophisticated specifications. For a utility based loss function, an asymmetric quadratic GARCH seems to dominate, and this result is robust to different degrees of risk aversion. For a VaR based loss function, a stochastic volatility model is preferred. Interestingly, the Riskmetrics EWMA model, proposed to calculate VaR, seems to be the worst performer. For the predictive likelihood based loss function, modeling the conditional standard deviation instead of the variance seems to be a dominant modeling strategy.

The definition and modeling of customer loyalty have been central issues in customer relationship management since many years. Recent papers propose solutions to detect customers that are becoming less loyal, also called churners. The churner status is then defined as a function of the volume of commercial transactions. In the context of a Belgian retail financial service company, our first contribution is to redefine the notion of customer loyalty by considering it from a customer-centric viewpoint instead of a productcentric one. We hereby use the customer lifetime value (CLV) defined as the discounted value of future marginal earnings, based on the customer's activity. Hence, a churner is defined as someone whose CLV, thus the related marginal profit, is decreasing. As a second contribution, the loss incurred by the CLV decrease is used to appraise the cost to misclassify a customer by introducing a new loss function. In the empirical study, we compare the accuracy of various classification techniques commonly used in the domain of churn prediction, including two cost-sensitive classifiers. Our final conclusion is that since profit is what really matters in a commercial environment, standard statistical accuracy measures for prediction need to be revised and a more profit oriented focus may be desirable.

For good reason, quality by design (QbD) has become a topic of significant interest within the pharmaceutical industry. Whereas regulatory agencies and standardsetting organizations are moving swiftly to establish QbD guidance relevant to the needs of pharmaceutical manufacturing, much of the core science of QbD was established long ago by Dr. Genichi Taguchi and other leaders in the field of quality management. In order for the pharmaceutical industry to fully capitalize on QbD, new methods for quantitatively assessing product quality need to be established. This perspective article presents a risk-based strategy for quality assessment which uses model-based simulation to link variation in drug product parameters and clinical performance. This work is intended to provide background and introduction for a series of manuscripts dealing with QbD based on probabilistic risk assessment.

The Erlang B formula is one of the most important formulas in the field of telecommunication. In this paper a simple approximation for the Erlang B formula has been presented. The accuracy of the produced approximate formula has been investigated. In various problems of communications it is often required to calculate the inverse Erlang B formula. Using the offered approximation the explicit expressions for estimation of the inverse Erlang losses function have been offered.

This paper extends the existing literature on empirical research in the field of credit risk default for Small Medium Enterprizes (SMEs). We propose a non-parametric approach based on Random Survival Forests (RSF) and we compare its performance with a standard logit model. To the authors’ knowledge, no studies in the area of credit risk default for SMEs have used a variety of statistical methodologies to test the reliability of their predictions and to compare their performance against one another. As for the in-sample results, we find that our non-parametric model performs much better that the classical logit model. As for the out-of-sample performances, the evidence is just the opposite, and the logit performs better than the RSF model. We explain this evidence by showing how error in the estimates of default probabilities can affect classification error when the estimates are used in a classification rule.

The definition and modeling of customer loyalty have been central issues in customer relationship management since many years. Recent papers propose solutions to detect customers that are becoming less loyal, also called churners. The churner status is then defined as a function of the volume of commercial transactions. In the context of a Belgian retail financial service company, our first contribution will be to redefine the notion of customer's loyalty by considering it from a customer-centric point-of-view instead of a product-centric point-of-view. We will hereby use the customer lifetime value (CLV) defined as the discounted value of future marginal earnings, based on the customer's activity. Hence, a churner will be defined as someone whose CLV, thus the related marginal profit, is decreasing. As a second contribution, the loss incurred by the CLV decrease will be used to appraise the cost to misclassify a customer by introducing a new loss function. In the empirical study, we will compare the accuracy of various classification techniques commonly used in the domain of churn prediction, including two cost-sensitive classifiers. Our final conclusion is that since profit is what really matters in a commercial environment, standard statistical accuracy measures for prediction need to be revised and a more profit oriented focus may be desirable.

Cryogenic air separation is an efficient technology for supplying large quantities of nitrogen, argon, and oxygen to chemical, petroleum and manufacturing customers. However, numerous uncertainties make effective operation of these complex processes difficult. This work addresses the problem of determining an optimal operating strategy to maximize the total profit of a cryogenic air separation process while considering demand uncertainty and contractual obligations. A rigorous process model is included as constraints in a nonlinear programming formulation. Uncertain demands are assumed to be normally distributed with known mean and standard deviation, and expected profit in the objective function is evaluated using the standard loss function. A probabilistic fill-rate expression, also based on the loss function, is used to model the contractual obligations by providing a lower bound on the expected product sales. In the single period case with one customer satisfaction constraint, the nonlinear programming formulation can be solved efficiently using the general purpose nonlinear optimization package, IPOPT. This formulation is then extended to include multiple time periods, the potential for product storage, and customer satisfaction constraints on multiple products. To solve the large-scale nonlinear programming formulation that considers a seven-day operating horizon, a tailored parallel nonlinear programming algorithm is used. This approach makes use of a Schur complement decomposition strategy to exploit the block structure of the problem and allow efficient solution in parallel. Using these tools, we solve for a set of optimal operating strategies over the complete space of different fill rates. This produces planning figures that identify key trade-offs between profitability and contractual obligations.

The present paper obtains a Bayes estimator of the disturbance variance in a linear regression model under two asymmetric loss functions viz., LINEX loss function and SQUAREX loss function, assuming a natural conjugate prior distribution for the parameters. The expression for the Bayes risk of the estimator is also derived. Ó 2000 Elsevier Science Inc. All rights reserved.

A major challenge to the successful full-scale development of modern aerospace systems is to address competing objectives such as improved performance, reduced costs, and enhanced safety. Accurate, high-fidelity models are typically time consuming and computationally expensive. Furthermore, informed decisions should be made with an understanding of the impact (global sensitivity) of the design variables on the different objectives. In this context, the so-called surrogate-based approach for analysis and optimization can play a very valuable role. The surrogates are constructed using data drawn from high-fidelity models, and provide fast approximations of the objectives and constraints at new design points, thereby making sensitivity and optimization studies feasible. This paper provides a comprehensive discussion of the fundamental issues that arise in surrogate-based analysis and optimization (SBAO), highlighting concepts, methods, techniques, as well as practical implications. The issues addressed include the selection of the loss function and regularization criteria for constructing the surrogates, design of experiments, surrogate selection and construction, sensitivity analysis, convergence, and optimization. The multi-objective optimal design of a liquid rocket injector is presented to highlight the state of the art and to help guide future efforts. r

Se define pérdida de calidad como la pérdida financiera de la compañía después de que un producto es rechazado. Dentro de estas pérdidas se incluye: La mala calidad lleva a la insatisfacción del cliente Costes de reparaciones y servicios de los productos defectuosos, algunos en el campo Disminuye la credibilidad del fabricante en el mercado El fabricante pierde la fracción del mercado, al menos eventualmente. En la contabilidad tradicional, una pieza es defectuosa y por lo tanto incurre en una pérdida económica si está fuera de tolerancia de diseño, si no, no hay pérdida para la compañía. Sin embargo, para Taguchi, la desviación respecto al objetivo de diseño ya constituye un pérdida de calidad por insatisfacción del cliente. Si Y es el valor medio de la fabricación, T es el valor objetivo de diseño, s es la desviación típica, la función de pérdida se puede representar como una parábola con vértice en la función objetivo y 0 pérdida de calidad, y en sus extremos (en el Límite de Control Inferior (LCI) y en el Límite de Control Superior) LCS)) de tolerancia de diseño adquiere el valor de sustitución de una pieza:

Multivariate GARCH models are in principle able to accommodate the features of the dynamic conditional correlations processes, although with the drawback, when the number of financial returns series considered increases, that the parameterizations entail too many parameters.In general, the interaction between model parametrization of the second conditional moment and the conditional density of asset returns adopted in the estimation determines the fitting of such models to the observed dynamics of the data. This paper aims to evaluate the interactions between conditional second moment specifications and probability distributions adopted in the likelihood computation, in forecasting volatilities and covolatilities. We measure the relative performances of alternative conditional second moment and probability distributions specifications by means of Monte Carlo simulations, using both statistical and financial forecasting loss functions.

In many tasks in pattern recognition, such as automatic speech recognition (ASR), optical character recognition (OCR), part-of-speech (POS) tagging, and other string recognition tasks, we are faced with a well-known inconsistency: The Bayes decision rule is usually used to minimize string (symbol sequence) error, whereas, in practice, we want to minimize symbol (word, character, tag, etc.) error. When comparing different recognition systems, we do indeed use symbol error rate as an evaluation measure. The topic of this work is to analyze the relation between string (i.e., 0-1) and symbol error (i.e., metric, integer valued) cost functions in the Bayes decision rule, for which fundamental analytic results are derived. Simple conditions are derived for which the Bayes decision rule with integer-valued metric cost function and with 0-1 cost gives the same decisions or leads to classes with limited cost. The corresponding conditions can be tested with complexity linear in the number of classes. The results obtained do not make any assumption w.r.t. the structure of the underlying distributions or the classification problem. Nevertheless, the general analytic results are analyzed via simulations of string recognition problems with Levenshtein (edit) distance cost function. The results support earlier findings that considerable improvements are to be expected when initial error rates are high.

This Working Paper is brought to you for free and open access by the School of Economics at Institutional Knowledge at Singapore Management University. It has been accepted for inclusion in Research Collection School of Economics by an authorized administrator of ...

In this paper, a perceptron-based algorithm for fusion of multiple fingerprint matchers is presented. The person to be identified submits to the personal authentication system her/his fingerprint and claimed identity. Multiple fingerprint matchers provide a set of verification scores, that are then fused by a single-layer perceptron with class-separation loss function. Weights of such perceptron are explicitly optimised to increase the separation between genuine users and impostors (i.e., unauthorized users). Reported experiments show that such modified perceptron allows improving the performances and the robustness of the best individual fingerprint matcher, and outperforming some commonly used fusion rules.

Covariance matrix forecasts of financial asset returns are an important component of current practice in financial risk management. A wide variety of models, ranging from matrices of simple summary measures to covariance matrices implied from option prices, are available for generating such forecasts. In this paper, we evaluate the relative accuracy of different covariance matrix forecasts using standard statistical loss functions and a value-at-risk (VaR) framework. This framework consists of hypothesis tests examining various properties of VaR models based on these forecasts as well as an evaluation using a regulatory loss function.

This paper applies a dynamic space-vector model to loss-minimizing control in induction motor drives. The induction motor model, which takes hysteresis losses and eddy-current losses as well as the magnetic saturation into account, improves the flux estimation and rotor-flux-oriented control. Based on the corresponding steady-state loss function, a method is proposed for solving the loss-minimizing flux reference at each sampling period. A flux controller augmented with a voltage feedback algorithm is applied for improving the dynamic operation and field weakening. Both the steady-state and dynamic performance of the proposed method is investigated using laboratory experiments with a 2.2-kW induction motor drive. The method improves the accuracy of the loss minimization and torque production, it does not require excessive computational resources, and it shows fast convergence to the optimum flux level. he has been working toward the Ph.D. degree at Aalto University, Espoo, Finland.

As customer's wait longer in line they become more dissatisfied. Because a wait time of zero is not economical, a balance must be obtained between the cost of waiting and the cost of service. Classic queuing theory does not generally provide cost figures. Therefore, in order to quantify the cost of waiting, this article utilizes the Taguchi loss function, which was developed for physical products, to determine the cost of customer dissatisfaction. Specifically, this article combines the M/M/1 queuing model with the Taguchi loss function to establish the cost of customer dissatisfaction.

A D-optimal minimax design criterion is proposed to construct two-level fractional factorial designs, which can be used to estimate a linear model with main effects and some specified interactions. D-optimal minimax designs are robust against model misspecification and have small biases if the linear model contains more interaction terms. When the D-optimal minimax criterion is compared with the D-optimal design criterion, we find that the D-optimal design criterion is quite robust against model misspecification. Lower and upper bounds derived for the loss functions of optimal designs can be used to estimate the efficiencies of any design and evaluate the effectiveness of a search algorithm. Four algorithms to search for optimal designs for any run size are discussed and compared through several examples. An annealing algorithm and a sequential algorithm are particularly effective to search for optimal designs.

Metric and kernel learning are important in several machine learning applications. However, most existing metric learning algorithms are limited to learning metrics over low-dimensional data, while existing kernel learning algorithms are often limited to the transductive setting and do not generalize to new data points. In this paper, we study metric learning as a problem of learning a linear transformation of the input data. We show that for high-dimensional data, a particular framework for learning a linear transformation of the data based on the LogDet divergence can be efficiently kernelized to learn a metric (or equivalently, a kernel function) over an arbitrarily high dimensional space. We further demonstrate that a wide class of convex loss functions for learning linear transformations can similarly be kernelized, thereby considerably expanding the potential applications of metric learning. We demonstrate our learning approach by applying it to large-scale real world problems in computer vision and text mining.

The conditions behind the sudden approximation are critically examined. The fuzzy band expression is derived in detail from first principles. We go beyond the sudden approximation to account for both extrinsic losses and interference effects. In an extension of earlier work we discuss both core and valence satellites including both intrinsic and extrinsic amplitudes, and high energy excitations as well as low energy electron-hole pairs. We show how the extrinsic losses in photoemission can be connected with the electron energy loss function. This is achieved by three approximations, to connect the dynamically screened potential in the bulk solid to the loss function, to account for the surface, and lastly to extrapolate zero momentum loss data to include dispersion. The extrinsic losses are found to be weak for small loss energies. For larger loss energies the extrinsic losses can be strong and have strong interference with the intrinsic losses depending on the nature of the solid. The transition from the adiabatic to the sudden regime is discussed for atoms and localized systems and compared with the situation for solids. We argue that in solids for photoelectron energies above some 10 eV the external losses are mainly connected with spacially extended excitations. We discuss strongly correlated quasi-two-dimensional solids with Bi2212 as example, and find an asymmetric broadening of the main peak from shake up of acoustic three-dimensional plasmons. In the superconducting state the loss function is assumed to have a gap, which then leads to a peak-dip-hump structure in the photoemission spectrum. This structure is compared with a corresponding structure in tunneling.

Many industrial experiments based on Taguchi's parameter design (PD) methodology deal with the optimization of a single performance quality characteristic. Studies have shown that the optimal factor settings for one performance characteristic are not necessarily compatible with those of other performance characteristics. Multi-response problems have received very little attention among industrial engineers and Taguchi practitioners. Many Taguchi practitioners have employed engineering judgement for determining the final optimal condition when several responses are to be optimized. However, this approach always brings some level of uncertainty to the decisionmaking process and is very subjective in nature. In order to rectify this problem, the author proposes an alternative approach using a powerful multivariate statistical method called principal component analysis (PCA). The paper also presents a case study in order to demonstrate the potential of this approach.

Support vector machines (SVMs) are special kernel based methods and belong to the most successful learning methods since more than a decade. SVMs can informally be described as a kind of regularized M-estimators for functions and have demonstrated their usefulness in many complicated real-life problems. During the last years a great part of the statistical research on SVMs has concentrated on the question how to design SVMs such that they are universally consistent and statistically robust for nonparametric classification or nonparametric regression purposes. In many applications, some qualitative prior knowledge of the distribution P or of the unknown function f to be estimated is present or the prediction function with a good interpretability is desired, such that a semiparametric model or an additive model is of interest. In this paper we mainly address the question how to design SVMs by choosing the reproducing kernel Hilbert space (RKHS) or its corresponding kernel to obtain co...

As customer’s wait longer in line they become more dissatisfied. Because a wait time of zero is not economical, a balance must be obtained between the cost of waiting and the cost of service. Classic queuing theory does not generally provide cost figures. Therefore, in order to quantify the cost of waiting, this article utilizes the Taguchi loss function, which was developed for physical products, to determine the cost of customer dissatisfaction. Specifically, this article combines the M/M/1 queuing model with the Taguchi loss function to establish the cost of customer dissatisfaction.

This paper briefly describes the mathematics of a discontinuous Bayesian command decision model. The model uses the principle of minimisation of expected loss and is based on two basic elements: the uncertainty in the decision-makers’ belief in outcome; and, his perception of loss (or penalty) associated with all possible actual outcomes (measured with respect to the desired or planned outcome). The belief function is combined with the perceived loss function to give an expected loss function. Minimisation of the expected loss function results in a cusp catastrophe. The two control parameters of the cusp cubic equation are defined by the width of the belief in outcome function (i.e. uncertainty) and the shape of the perceived loss function (i.e. criticality of decided action). In summary, the paper proposes that two main factors drive military decision-making and offers qualitative appreciation of their impact, which is useful, in particular, for Information Operations and C2W. The ...

Statistical modelling and inference problems with sample sizes substantially smaller than the number of available covariates are challenging. This is known as large p small n problem. Furthermore, the problem is more complicated when we have multiple correlated responses. We develop multivariate nonlinear regression models in this setup for accurate prediction. In this paper, we introduce a full Bayesian support vector regression model with Vapnik's ǫ-insensitive loss function, based on reproducing kernel Hilbert spaces (RKHS) under the multivariate correlated response setup. This provides a full probabilistic description of support vector machine (SVM) rather than an algorithm for fitting purposes. We have also introduced a multivariate version of the relevance vector machine (RVM). Instead of the original treatment of the RVM relying on the use of type II maximum likelihood estimates of the hyper-parameters, we put a prior on the hyper-parameters and use Markov chain Monte Carlo technique for computation.

Kernel logistic regression (KLR) is a powerful discriminative algorithm. It has similar loss function and algorithmic structure to the kernel support vector machine (SVM). Recently, Zhu and Hastie proposed the import vector machine (IVM) in which a subset of the input vectors of KLR are selected by minimizing the regularized negative loglikelihood to improve the generalization performance and to reduce computation cost. In this paper, two modifications of the original IVM are proposed. The cross-validation based criterion is used to select import vectors instead of the likelihood based criterion. Also binary particle swarm optimization is used to select good subset instead of the greedy stepwise algorithm of the original IVM. Through the comparison experiment, the improvement of the generalization performance of the proposed algorithm was confirmed.