All Questions
Tagged with precision-recall confusion-matrix
36 questions
5
votes
3
answers
566
views
Judging a model through the TP, TN, FP, and FN values
I am evaluating a model that predicts the existence or not existence of a "characteristic" (for example, "there is a dog in this image") using several datasets. The system outputs ...
- 257
2
votes
3
answers
234
views
Is relying on just the confusion matrix for highly imbalanced test sets to evaluate model performance a bad idea?
I have a binary classification model with a test set that is highly skewed, the majority class 0 is 22 times greater than the minority class 1.
This causes my Precision to be low and Recall to be high,...
- 21
11
votes
8
answers
9k
views
My machine learning model has precision of 30%. Can this model be useful?
I've encountered an interesting discussion at work on interpretation of precision (confusion matrix) within a machine learning model. The interpretation of precision is where there is a difference of ...
- 121
2
votes
2
answers
1k
views
Sensitivity vs. specificity vs. recall
Given a binary confusion matrix with true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN), what are the formulas for sensitivity, specificity, and recall?
I'm ...
- 219
2
votes
1
answer
62
views
A metric for a big/medium/small ML classification
I am working on an ML classification task which is similar to the following:
Apples have to be classified to three classes: Big, Medium and Small.
I need a metric which I can use to assess the system. ...
- 123
2
votes
2
answers
1k
views
Logistic Regression: What is the value for precision when recall (true positive rate) is 0?
A quick overview of definitions before I get into the question:
True Positive (TP): An actual positive that the model classified as positive
False Positive (FP): An actual negative that the model ...
- 719
1
vote
1
answer
40
views
Estimate sample size from a variable population
Context:
I want to measure accuracy, precision, and recall for individual raters. Each rater completes a variable amount of labels, for ex. rater A may complete 500 in a given time period while rater ...
3
votes
1
answer
178
views
When comparing classifiers on different datasets with different prevalences, is it valid to calculate prevalence-adjusted PPV?
Scenario: comparison of 2 different binary classifiers
Both classifiers report sensitivity and specificity and number actually positive (P), but classifier 1 is tested on a dataset with prevalence 20%,...
- 31
1
vote
0
answers
138
views
Calculating overall Precision and Recall, one versus all
I've got several confusion matrices, all of binary classification (negative, positive). I would like to get general scores of all the matrices combine. Problem is, that the data is not balanced at all....
- 111
0
votes
1
answer
474
views
How to practically calculate the accuracy of each class in muliclass classification problem?
I have the following confusion matrix:
...
- 212
2
votes
2
answers
4k
views
How can accuracy be greater than my precision, recall and F-Score metrics?
I have trained two models to detect gestures using ambient light and solar panels. I am now testing the two models in different light scenarios. I have a Convolutional Neural Network model that ...
- 21
1
vote
1
answer
918
views
Interpreting NaN values for precision in Confusion Matrix
Please refer to the confusion matrix here: https://i.sstatic.net/Yxh5V.jpg
Would I get precision values of NaN because of 0/0 in the right most columns? Is that even possible? How should I interpret ...
- 49
1
vote
1
answer
142
views
How to obtain conditional use accuracy equality with communities with different real positive rates?
The short version is that I would like to know what the confusion matrices (numbers of true positives, false positives, true negatives, and false negatives) should be to achieve conditional use ...
- 163
0
votes
0
answers
42
views
A question about precision
We have 2 Classification models (Random forest on balanced Data Set), the first one classify a Bank's client as a Churner (closing his account) or an active client, the second one classify a Bank's ...
1
vote
1
answer
364
views
Why is it called Sensitivity/Recall and Specificity?
Where do the terms: Sensitivity, Recall and Specificity come from historically? I've been looking for an answer for quite some time but to no avail.
I understand the formulae and what they mean but I ...
- 279
2
votes
0
answers
487
views
When should one look at sensitivity vs. specificity instead of precision vs. recall?
The precision vs. recall tradeoff is the most common tradeoff evaluated while developing models, but sensitivity vs. specificity addresses a similar issue. When should one of these pairs of metrics be ...
- 145
0
votes
3
answers
1k
views
Is it ok a threshold of 0?
I am dealing with a classification problem with a dataset containing 60k rows: 69k are negative class, and 1k is positive.
I trained my models and I obtained the confusion matrices with a threshold of ...
- 123
2
votes
2
answers
2k
views
Weighting common performance metrics by classification outcomes?
Cost-sensitive classification metrics are somewhat common (whereby correctly predicted items are weighted to 0 and misclassified outcomes are weighted according to their specific cost). Some examples ...
- 649
2
votes
0
answers
753
views
Precision-Recall Curve Intuition for Multi-Class Classification
I am running a CNN image multi-class classification model with Keras/Tensorflow and have established about a 90% overall accuracy with my best model trial. I have 10 unique classes I am trying to ...
- 181
0
votes
0
answers
1k
views
Imbalanced classification what is good F1 score?
For imbalanced classification (say 85:15), what is good value of F1 score? An answer https://stats.stackexchange.com/a/217343/285091 says "Experiments indicate that the sweet spot here is around ...
- 101
1
vote
0
answers
75
views
ROC Curve for data sets with large negative bias
For context, I've read this forum here regarding a similar issue, and it seems the conclusion on there was that precision-recall curves are better-suited for data sets where there is a large negative ...
- 11
1
vote
0
answers
19
views
How to verify that one ML-classifier is better then the other using the same train and test data set without cross-validation?
I have compared 5 methods (ML classifiers) on the same data set. These methods are 5 different types of neural networks. Each is trained on the training set and evaluated with precision, recall and f1-...
- 174
1
vote
1
answer
161
views
Real vs True Positives
Wikipedia defines TPR (True Positive Rate) as $\frac{\text{TP}}{P}$
where:
$\text{TP}$ = # of true positives
$\text{P}$ = # of real positives
This confuses me.
I thought:
$\text{TP}$ is supposed ...
- 4,598
3
votes
1
answer
656
views
F1 score macro-average
In this question I'll differentiate by using lower-case for class-wise scores, e.g. prec, rec, f1, which would be vectors, and the aggregate macro-average Prec, Rec, F1. My formulae below are written ...
- 230
4
votes
2
answers
4k
views
Is there a name for this metric: TN / (TN + FN)?
Given a confusion matrix, there's all kind of metrics: Accuracy, Precision, Recall/Sensitivity, Specificity.
But I haven't seen any name for the ratio between the TN (True Negative) and the sum of ...
- 3,680
1
vote
0
answers
348
views
Precision, Recall, F1 Score and multiclass confusion matrix
My goal is to evaluate some clustering algorithms. Lets say that I’ve got one set o elements: { 1 ; 2; 3; 4; 5; 6; 7; 8 }. This set was grouped by human into two groups: (1 ; 2; 3; 4; 5; 6; 8) and (7)....
- 31
0
votes
0
answers
51
views
Understanding ratio between precision and recall
I have a naive question regarding the ratio of precision and recall. When I build the model, I am able to get precision and recall. Later, I could use this model to make perditions, which is ...
- 113
2
votes
2
answers
274
views
Precision-Recall curve interpretation
When given an example confusion matrix:
TP = 5000 FP = 1000
FN = 0 ...
- 121
19
votes
1
answer
14k
views
Confidence interval of precision / recall and F1 score
To summarise the predictive power of a classifier for end users, I'm using some metrics. However, as the users input data themselves, the amount of data and class distribution varies a lot. So to ...
- 547
15
votes
2
answers
22k
views
Is there any difference between Sensitivity and Recall?
In most of the places, I have found that sensitivity=recall. In terms of the Confusion Matrix, the formula for both of these is the same: $TP/(TP+FN)$.
Is there any difference between these two ...
0
votes
1
answer
4k
views
How to correctly read a classification report?
Firstly, is there a difference between model performance and it's accuracy? If yes, what exactly?
Secondly, what can I interpret from this classification_report of ...
4
votes
1
answer
1k
views
How can Precision-Recall (PR) curves be used to judge overall classifier performance when Precision and Recall are class based metrics?
How can Precision-Recall (PR) curves be used to judge overall classifier performance when Precision and Recall are class based metrics?
Since in a binary classifier, there are two classes, often ...
- 305
5
votes
1
answer
2k
views
Confusion matrix, metrics, & joint vs. conditional probabilities
In the binary classification/prediction problem we have unknown labels $y\in\{0,1\}$, which we try to predict using an estimator $\hat{y}$. Commonly the performance of an estimator is summarized using ...
- 13.1k
3
votes
1
answer
2k
views
Classification using ctree in R [closed]
I've been trying to perform classification on Indian Liver Patients Disease dataset available at https://archive.ics.uci.edu/ml/datasets/ILPD+(Indian+Liver+Patient+Dataset)
I have used Conditional ...
- 53
1
vote
1
answer
1k
views
Are the total false positives and false negatives of a large confusion matrix equal?
I have a confusion matrix which is 20x20 that is the product of a random forest classification of ~20k instances. Each of these instances was put into a specific class where rows are actual class and ...
1
vote
2
answers
4k
views
How to get confusion matrix with 100% precision or 100% recall in Weka
Here is the original confusion matrix:
...
- 111