All Questions
Tagged with kernel-trick classification
47 questions
7
votes
5
answers
4k
views
Would a machine learning classifier algorithm be able to determine whether a number is odd or even?
I was testing out some classifier algorithms in scikit but wasn't able to find a classifier (linear or non-linear) that managed to provide good prediction on whether an input number is odd or even. ...
- 189
1
vote
0
answers
92
views
Implementing kernel alignment for SVM algorithms
I am trying to understand and re-implement the results from Table 2 in the first Kernel-Target Alignment paper. The task that is being done is a simple classification task using an SVM with RBF ...
1
vote
0
answers
153
views
Why are odd-degreed polynomial kernels slower than those with even degrees for SVM?
I have been using one-class support vector classifiers to extract features for multinomial classification. I noticed that fitting time is much longer when the degree of the polynomial kernel is odd. ...
- 111
6
votes
1
answer
3k
views
What does representer theorem in machine learning tell us?
In reference to the Representer Theorem in machine learning, Why this is so important? Somehow, this theorem justifies the importance of Kernels in machine learning, i.e. the Kernel trick - a more ...
5
votes
2
answers
421
views
Why use RBF kernel if less is needed?
I have seen online theorem's such as Cover's theorem Wikipedia which prove how given $p$ points in $\mathbb{R}^N$ the linear separability is almost certain as the fraction $\dfrac{p}{N}$ is kept close ...
1
vote
0
answers
69
views
How to extend kernel-based classifier to non-euclidean space like SO3
What is the proper way to extend kernel-based classifier to non-euclidean space like SO3? This kind of situation happens a lot in robotics, where the data points all live in a specific manifold. (Note:...
- 211
3
votes
1
answer
86
views
In this example, which of these vectors are support vectors?
The hyperplane of hard margin SVM with $\phi$ kernel is calculated as following that input space using $\phi$ to map to higher dimension space.
$$f(\phi(x))=4\phi_1(x)+9\phi_2(x)+4\phi_3(x)$$
$$ \phi(...
- 31
2
votes
1
answer
1k
views
What is the intuition behind changing the dot product for another inner product in SVM?
I understand that, when classifying with a SVM using a non-linear kernel, we are basically changing the dot product for a "custom" inner product. Is there some reason for working with a different ...
- 728
1
vote
1
answer
2k
views
What is the difference between explicit and implicit mapping in SVM?
1) What is the difference between explicit and implicit mapping
2) What is the difference between mapping and kernel trick?
dinesh12
2
votes
2
answers
728
views
Is alpha*RBF a valid kernel, where alpha >= 0 is a parameter?
I wonder if K = alpha*RBF can be a valid kernel satisfying Mercer's condition, where ...
- 173
3
votes
1
answer
689
views
A bunch of questions about Kernels in Machine Learning
i've read many topics on this platform about this topic but i still have some questions, mainly theoretical.
We are dealing with ML, so if we are here means that we have to classify with linear ...
- 131
1
vote
1
answer
685
views
Does SVM get biased towards majority class in case of imbalanced class proportion?
After reading many posts, I thought of asking: Why should a SVM be biased towards majority class like other classifiers, since an SVM never used the whole data of the training data set—it only uses ...
- 111
1
vote
1
answer
775
views
How to understand the predicted **negative** values by Kernel Regularized Least Squares (KRLS)?
I am learning the prediction algorithm, Kernel Regularized Least Squares (KRLS). The predicted values are listed in the follows:
$$\hat{y} = K((K + 1 \times I)^{-1}y)$$
For example, I have 100 ...
- 221
1
vote
1
answer
573
views
Does SVM prediction accuracy depend on a positive scaling of the kernel function?
Support vector machine (SVM) is a supervised learning algorithm. It draws hyperplanes to separate data points of different classes. The objective function involves inner products of pairs of feature ...
- 133
8
votes
1
answer
1k
views
Should we account for the intercept term when kernelizing algorithms?
When a learning algorithm (e.g. classification, regression, clustering or dimension reduction) uses only the dot product between data points $\mathbf {x x^T}$ we can implicitly use a higher ...
- 19.5k
9
votes
3
answers
7k
views
Projecting to lower/higher-dimensional space for classification: dimensionality reduction vs kernel trick
Whilst learning about classification, I have seen two different arguments. One is that projecting the data to a lower-dimensional space, such as with PCA, makes the data more easily separable. The ...
- 7,129
6
votes
2
answers
1k
views
Understanding Kernel Functions for SVMs
I am learning about Support Vector Machines, and in particular, those with kernels for non-linear decision boundaries. I understand the concept of projecting the original data to a higher-dimensional ...
- 7,129
3
votes
1
answer
459
views
How are Hyperplane Heatmaps created and how should they be interpreted?
For nonlinear data, when we are using Support Vector Machines, we can use kernels such as Gaussian RBF, Polynomial, etc to achieve linearity in a different (potentially unknown to us) feature space ...
- 254
4
votes
1
answer
1k
views
SVM Kernel confusion
Suppose that we have an array of 10x2 elements (features). Each of these features are two-dimensional. Something like this:
...
- 53
1
vote
1
answer
184
views
How to know which Kernel is better?
I am working on an Image recognition software -
My first question is since I already explicitly turm my training images to features vector (and also my test images) what is the point of using ...
- 123
0
votes
1
answer
134
views
How to detect classifier curve in non-separable SVM problem
Suppose we want to classify two class of data that are non-separable with hyper-plane. So we use kernels to map data to high-dimensional space.
See my codes:
...
- 121
4
votes
1
answer
287
views
Using kernels with Fisher's linear discriminant analysis
I am a bit stuck implementing the Kernel Fisher Discriminant.
$$
J(\mathbf{w}) = \frac{\mathbf{w}^{\text{T}}\mathbf{S}_B^{\phi}\mathbf{w}}{\mathbf{w}^{\text{T}}\mathbf{S}_W^{\phi}\mathbf{w}}
$$
$$
...
- 93
1
vote
0
answers
328
views
Probabilistic degree of confidence for the kernel SVM with RBF
Let $f\colon\Bbb{R}^n\to\Bbb{R}$ be the decision function of an SVM using the radial basis function (RBF),
$$
k(\mathbf{x},\mathbf{x}')=\exp\Big(-\gamma\|\mathbf{x}-\mathbf{x}'\|^2\Big).
$$
That is,
$...
- 353
1
vote
2
answers
2k
views
Kernel PCA and classification
I need to perform kernel PCA on the colon-‐cancer dataset and then I need to plot number of principal components vs classification accuracy with PCA data.
For the first part I am ...
- 135
2
votes
1
answer
177
views
Integrating length for input-space feature PC projections in kernel PCA
I read a paper detailing the algebraic process of kernel PCA. I have question though: the paper details the projection of new points onto the new eigenvectors in the feature space, but what if I want ...
- 2,121
1
vote
2
answers
3k
views
Kernel SVM on sparse data
I have a sparse dataset where a lot of the columns (features) contain mostly zero values. Class labels are multiple discrete categories (10 classes to be precise). I'm wondering if this should trouble ...
- 403
3
votes
1
answer
1k
views
Which PCA (or kernel PCA) basis better describes a single test sample?
I have two PCA bases obtained by decomposition of two groups of training data. I also have some samples of test data.
How can I decide which PCA basis fits better each test sample?
I tried to ...
- 1,526
3
votes
3
answers
1k
views
Applying an RBF kernel first and then train using a Linear Classifier
I will start off by saying that I don't have a concrete understanding of what's under the hood of a SVM classifier.
I am interested in using an SVM with the RBF kernel to train a two class ...
- 31
0
votes
1
answer
86
views
Evolution strategies in libsvm
I'm working on a protein multi-classification problem, using libsvm and the edit distance kernel. This kernel depends on a parameter $\gamma$. I'm able to get the best parameters ($\gamma$ and $C$) ...
- 11
2
votes
1
answer
444
views
Improving SVM classification
I have a classification problem (bioinformatics domain) where I have around 333 features. Currently, I am first selecting features (using importance feature of random forest) and then pushing the same ...
- 325
1
vote
0
answers
51
views
construct/load dataset that performs better with diffusion kernel than other kernel
I'm looking for a dataset on which a diffusion kernel (also called heat kernel), used via SVM, would get better accuracy than other kernels for the classification task. I want to use such a dataset to ...
- 11
3
votes
1
answer
1k
views
Binary classification using radial basis kernel SVM with a single feature
Is there any interpretation (graphical or otherwise) of a radial basis kernel SVM being trained with a single feature? I can visualize the effect in 2 dimensions (the result being a separation ...
- 459
0
votes
1
answer
561
views
SVM cost, kernel and dimension
Why is it SVM computation cost does not depend on kernel value, dimensions (when separating hyperplane )? Is it because all it does is just classifying and
not much calculation involved?
- 69
0
votes
1
answer
485
views
Difference between Kernel classifier and linear classifier [duplicate]
I would just like to know what are the differences between kernel classifier and linear classifier?
In what kind of problems the first is used and in what kind the second?
What could be the ...
- 644
59
votes
2
answers
89k
views
Linear kernel and non-linear kernel for support vector machine?
When using support vector machine, are there any guidelines on choosing linear kernel vs. nonlinear kernel, like RBF? I once heard that non-linear kernel tends not to perform well once the number of ...
- 5,282
5
votes
1
answer
916
views
SVM classification step on embedded system with RBF kernel
I am about to implement the classification step of a trained SVM model. I would like to ask, how the actual classification step is carried out (assuming I would like to port that step to some low-...
10
votes
2
answers
20k
views
Which SVM kernel to use for a binary classification problem?
I'm a beginner when it comes to support vector machines. Are there some guidelines that say which kernel (e.g. linear, polynomial) is best suited for a specific problem? In my case, I have to classify ...
- 445
6
votes
1
answer
2k
views
Regarding redundant training data in building SVM-based classifier
To build a SVM-based classifier, I have a training data set consisting of N data points. Some of them are redundant. For instance, there have 50 data points which are exactly the same, and there have ...
- 2,827
4
votes
1
answer
10k
views
About SVM cost and gamma parameters tuning
I am using R and e1071 package to tune a C-classification SVM.
My question is: regardless of the kernel type (linear, ...
- 637
42
votes
2
answers
64k
views
Which search range for determining SVM optimal C and gamma parameters?
I am using SVM for classification and I am trying to determine the optimal parameters for linear and RBF kernels. For the linear kernel I use cross-validated parameter selection to determine C and for ...
- 421
9
votes
1
answer
18k
views
Non-linear SVM classification with RBF kernel
I'm implementing a non-linear SVM classifier with RBF kernel. I was told that the only difference from a normal SVM was that I had to simply replace the dot product with a kernel function:
$$
K(x_i,...
- 115
2
votes
1
answer
798
views
Possible reason for failing to build a support vector machine
I was trying to build a classifier for a set of documents using a support vector machine. I choose to build the feature space using term occurrence. While experimenting, I found the following scenario:...
- 1,307
12
votes
1
answer
3k
views
The relationship between the number of support vectors and the number of features
I ran an SVM against a given data set, and made the following observation: If I change the number of features for building the classifier, the number of resulting support vectors will also be changed. ...
- 5,282
7
votes
4
answers
8k
views
Train a SVM-based classifier while taking into account the weight information
Currently I have a data set which are known to belong to two classes, and would like to build a classifier using SVM. However, there exist different confidence levels for this data set. For example, ...
- 3,089
5
votes
1
answer
249
views
Constrain decision boundary to fall on grid lines in multiple class logistic regression
I would like to use multiple class logistic regression to learn the decision boundaries separating the different classes (denoted by color) in the image below. Kernel logistic regression with a RBF ...
- 1,200
3
votes
1
answer
918
views
Linear discriminant analysis and the "kernel trick"?
This is problem 12.10 in "The Elements of Statistical Learning":
Suppose you wish to carry
out a linear discriminant analysis
(two classes) using a vector of
transformations of the input ...
- 1,393
7
votes
3
answers
7k
views
VC dimension of SVM with polynomial kernel in $\mathbb{R^{2}}$
What is the VC dimension of SVM with the polynomial kernel $k(x,x')=(1+<x,x'>_{\mathbb{R^{2}}})^{2}$ for binary classification in $\mathbb{R^{2}}$?
It would be equal or more than v iff ...
- 1,105