Questions tagged [ranking]
Ranking is the task (for respondents) or the result of ordering the given stimuli or performers from "highest" to "lowest" (or *vice versa*) in some respect. It is usually contrasted with rating of stimuli. (For ranking as a way of data transformation - use tag [ranks]).
658 questions
1
vote
0
answers
15
views
Most natural class of machine learning models for race data/ group data
I have a dataset of performance of students in exams which looks like:
...
- 195
5
votes
1
answer
123
views
XGBoost/ XGBRanker to produce probabilities instead of ranking scores
I have a dataset of the performance of students in exams which looks like:
...
- 195
0
votes
1
answer
20
views
Survey with forced ranking
I have created a survey with a ranking scale of 1 = Least effective, 2= Somewhat effective 3= Effective 4 = Most effective.
People answering the questions were forced to select only one response per ...
4
votes
2
answers
206
views
Relationship between AUC and Cross-entropy
I understand that AUC measures the model's ability to rank the subjects (see Why is ROC AUC equivalent to the probability that two randomly-selected samples are correctly ranked?).
In contrast, binary ...
- 41
0
votes
0
answers
18
views
Ranking the best products based on availability
Let's say I have a typical sales dataset where I have a row per order number and the individual products in each order. With this dataset, I am asked to recommend a new product.
My initial approach ...
11
votes
3
answers
226
views
How can I order or sort many distributions, then summarize the "quantiles" of this ordered list?
What are the best practices for sorting a large number of distributions? What are some of the pros and cons of different ways to do this?
Let's say I am working with Likert responses to a survey ...
- 4,603
6
votes
1
answer
281
views
Percentage changes versus absolute changes when comparing rankings
We are analyzing the ranking changes of countries based on an economic index to determine whether these variations are significant. For example, Country A moves from rank 5 to rank 3, representing a ...
- 81
1
vote
2
answers
45
views
How to create a rank from ranks?
I have calculated ranks for each row based on a condition on each column. Now I have,
...
- 37
1
vote
1
answer
46
views
Computing heuristic from winrate and number of games
I need to sort big sets of players (of a MOBA) based on their performance represented by their - number of - games and their winrate on these games.
I reduced the initial set of players based on their ...
- 11
0
votes
0
answers
12
views
fuzzy decission making
I'm working on decision-making using Double Hierarchy Fuzzy Linguistic terms. I want to ask if can we compare two Double Hierarchy Fuzzy Linguistic terms.
I am trying to implement the TODIM algorithm ...
10
votes
3
answers
2k
views
What is the best way of combining multiple rankings into one?
Suppose I have $N$ different items to be ranked, and $M$ different people who provide a ranking of the items according to their personal preference. What is the best way of combining all rankings into ...
- 223
0
votes
0
answers
41
views
Effective pairwise comparisons for group ranking?
Suppose I have a list of items(~3500 in number) and I want to do a group based ranking into 4-5 groups. I can get pairwise comparisons between the elements but I want to choose the items to compare in ...
1
vote
0
answers
24
views
Finding the most-similar color palette [closed]
I have a dataset of several thousand color palettes: for each row in a dataset, I have the ten most common RGB values in an image, and the fraction of that color in the overall image:
...
2
votes
1
answer
74
views
Predicting the order/ranking of continuous observations
I have a problem that is kind of similar to ordinal regression, but not the same. If I understand it correctly, in ordinal regression we try to predict ordered labels, e.g. the number of stars a book ...
1
vote
1
answer
107
views
Estimating maximum effect size and Skillings-Mack test
I'm working with data that fit the Skillings-Mack test: unbalanced incomplete block design. Or Friedman-test-like with missing data.
In case of negative test result I'd like to estimate the maximum ...
- 190
3
votes
1
answer
503
views
How to rank data based on multiple variables
I need help in ranking data, says car models in this case, based on multiple variables. For some variables (eg. mpg), the higher the better. For some variables (eg. car age), the lower the better. For ...
- 79
1
vote
1
answer
52
views
A Rank comparison test? [closed]
Is there a test that one can use to perform rank comparison tests.
In other words if I have 2 variables (say income in all 10 regions of country X; and, income in all 10 regions of country Y) and I ...
- 11
1
vote
1
answer
53
views
Correlations among rankings
In my study, participants ranked their preferences for several choices. Because moving one choice up necessarily means moving one or more choices down, nearly all correlations (Pearson's r) are ...
6
votes
1
answer
275
views
XGBoost Learning to Rank with XGBClassifier
I am trying to build a model trained on binary labels that has a high precision for the top k predicted instances, and don’t care too much about recall or precision more generally. I was then ...
- 228
1
vote
0
answers
27
views
Compare multiclass probabilities for two classifiers
I have two algorithms that produce, for every observation, a vector of probabilities for 3 classes
...
4
votes
1
answer
153
views
Is the relation "not FOSD" transitive?
$X:\Omega\to [0,1]$ is a random variable. It is known that first order stochastic dominance FOSD is a partial order that is transitive:
$X$ FOSD $Y$, $Y$ FOSD $Z$ implies $X$ FOSD $Z$.
Now consider ...
- 885
1
vote
0
answers
42
views
Model test sensitivity to ability
I have a set of tests and a population of agents whose ability I want to assess. Each agent has taken some of the tests. The agents have no memory of the tests they have taken, so each time an agent ...
- 193
0
votes
0
answers
56
views
Determine "best" winrate between data with different sample sizes
Setup
I've a collection of winrates where each winrate represents a particular combination of "power ups" a player can select for a character they play in a game. The numerator represents ...
- 28
0
votes
0
answers
36
views
How to evaluate the validity of my teaching methods through analyzing the data of the students' ranks in a grade through time?
I also posted the question at https://math.stackexchange.com/questions/4793565/how-to-evaluate-the-validity-of-my-teaching-methods-through-analyzing-the-data-o
Problems I want to solve
I want to be a ...
- 1
0
votes
0
answers
29
views
Comparing proxy metrics to human evaluations
I have two proxy metrics, and I'd like to see which of them correlates more strongly with human ratings. I have ~30 questions, and for each question 3 humans independently give a score on a 1-10 scale....
- 101
6
votes
5
answers
781
views
Hypothesis testing for a sequence (ranking)
Let's say I must taste and rank 4 bottles of wine by price.
The true ranking is [X,Y,Z,W]. The ranking I provide after tasting is [X,Y,W,Z], where X,Y,Z,W are different bottles of wine.
Which ...
- 403
1
vote
0
answers
58
views
Modeling player interactions in multi-dimensional rating systems
In traditional rating systems (such as Elo), a player's strength is represented by a single scalar value, which is assumed to be consistent across different opponents. However, in some games, the ...
- 11
2
votes
0
answers
51
views
rank aggregation with partial ranking lists
My question is about rank aggregation, which is in relation with statistics and decision sciences. (if this question is off the topic of this site, plz let me know.) The question is that I have N ...
- 31
1
vote
0
answers
121
views
Example probability model predicting vectors of ranks?
Background
I'll confess, I like Richard McElreath's lectures so much that I've been watching his backlog of lectures (even though I've already seen all of the recent lectures of the same course).
In ...
- 9,680
1
vote
0
answers
25
views
calculating a statistically significant difference in means between groups
A large number of subjects repeatedly pass or fail a test. The test frequency is irregular and randomized per user.
Organize this data set in groups by the average test frequency being once per week (...
- 11
2
votes
0
answers
35
views
Best way of determining how unique an inspector's ratings are compared to the average inspector
Noob question.
I have a full population of every inspector rating for all items being rated. Each inspector has inspected a subset of the items and given them a rating, but different inspectors have ...
2
votes
1
answer
108
views
What are the undefined constants and functions in Stern's 2011 paper?
I'm reading the 2011 paper on ranking called Moderated Paired Comparisons by Steven E. Stern and there are no definitions given for some of the constants and functions in equation 1.
As you can see, ...
- 33
1
vote
1
answer
50
views
Rank clusters based on mean feature value and variance feature value of the cluster
I would like to rank clusters according to their importance.
From 113 demand time series data I extract the following 6 features from each time series.
"Average Demand", "n95 demand&...
- 294
1
vote
0
answers
24
views
Are paired matches in ranked data independent, if the original, unranked, continuous pairs are independent?
Let $(X, Y)$ be a randomly drawn sample of $n$ paired observations from a bivariate continuous population. It is clear that each pair is independent of the others, both pairwise and mutually. Let $R(X,...
- 642
0
votes
1
answer
97
views
How do I test the significance of a distribution of tau coefficients?
I have (I think) a simple question. I estimated rank-orderings on a variable from a sample of 225 people, and I'm interested to know whether they generally correlated with a "correct" rank-...
- 129
0
votes
1
answer
327
views
Weighted Spearman Rank correlation proof
I have been searching for alternate measures to compute spearman rank correlation that incorporates some sort of weighting in the calculation. I came across this
paper by David C. Blest on weighted ...
- 403
0
votes
0
answers
79
views
Bayesian estimation for a ranking outcome variable
I'm interested in modeling how a ranking depends on a continuous feature. I have many related groups of these rankings, so I want to use partial pooling with the usual Bayesian machinery, but I'm ...
- 101
3
votes
0
answers
207
views
How to determine sample size for number of pairwise comparisons for item ranking
Suppose I have 100 items and I would like to come up with their ranking based on user preferences. i.e. each item should receive a rank from between 1-100, where 1 is the most preferred item and 100 ...
- 351
0
votes
2
answers
111
views
Rank Neurons Importance of the latent space of an Autoencoder using PCA
I am trying to extract only the important neurons from the latent space of an Autoencoder to be converted later to a pattern for a model pattern recognizer.
PCA Loadings helps in finding the highest ...
1
vote
0
answers
92
views
How to Compute Weighted Kendall's Tau for Health Data?
I am analyzing a dataset that includes sequences of patients and their associated health values, both provided by different doctors. Each sequence represents the treatment order, and the health values ...
- 111
1
vote
1
answer
110
views
Best way to rank list of values having overlapping uncertainties
I have a list of individuals and their success rates on a task. The count of tasks for each varies widely, so the uncertainty in the estimates of their proficiency varies. Furthermore, typical ...
- 111
1
vote
2
answers
101
views
Statistical model for likelihood that an athlete will "place" in track and field
I'm trying to come up with some sort of statistical model to predict how likely an athlete would be to place in a track and field event (e.g. 100m run) based on their time. I was thinking it would be ...
- 111
1
vote
0
answers
58
views
Comparing two ranked list with scores
I want to compare two ranked lists with items and their confidence scores assigned by the model. The lists are non-conjoint, i.e. can have different items. The number of items is the same for both ...
- 11
4
votes
1
answer
378
views
Comparing frequencies or proportions between two groups to find differences
Suppose I have two groups of users, A and B. In case it is relevant, B is much smaller than A.
I have a feature of products purchased across two groups. I want to find items whose popularity is ...
5
votes
1
answer
227
views
Ranking of multiple raters with partially overlapping assessments
Ranking of Raters and Cases
I am looking for a method for ranking a set of raters with partially overlapping binary assessments. I also want to rank the cases according to their difficulty.
Context
I ...
- 59
1
vote
0
answers
24
views
Statistical methods for ranking units
The problem: Suppose I have an ordered sample of $n$ observations (e.g. a playlist of songs) that are ranked according to some latent (unobserved) feature, for each of which $p$ covariates are ...
- 66
0
votes
0
answers
58
views
Does the rank transform preserve signum of Spearman correlation between parameter estimates across samples?
Suppose we have real-valued random variables $X$, $Y$, with noise $\epsilon$ that is independent of $X$ and $Y$ and $\mathbb{E}[\epsilon] = 0$, and measurable function $f$.
I am thinking about ...
- 9,680
1
vote
0
answers
44
views
Ranked data and hypothesis testing
I have a few questions regarding ranked data and hypothesis testing.
Hypothesis: Car drivers are more likely to consider the car as the most beneficial transportation method for mental health than ...
2
votes
1
answer
131
views
Optimizing XGBClassifier for discrimination power
I am developing a credit risk decisioning model, i.e. a model that assesses the risk of default of an incoming transactions and decides whether to accept it or not. Of course my dataset is imbalanced :...
- 21
1
vote
1
answer
346
views
Distribution of the rank of a sample from a normal distribution among a set of samples from a different normal distribution
I draw a single sample from a given standard normal distribution: $Y \sim N(0, 1)$. I also draw $n$ samples from a different normal distribution: $X_i \sim N(\mu, \sigma^2)$ for $i=1...n$. What is the ...
- 18k