What are the best rules/guidelines for determining statistical ranking methods?

In summary, people seeking statistical ranking methods/rules/guidelines should completely describe the real world problem they are trying to solve. Only then can they hope to receive helpful advice.
  • #1
StatsForLife
7
0
Greetings All!

I have what appears to be the most difficult question in the world based on the absence of even a single answer found on Google.

I am looking for statistical ranking methods/rules/guidelines, etc.

I work with data that is the result of taking continuous data (distance) and dividing it by discrete data (specific # of events). The results are often a group of 32 dense variables ranging from say, 1.1 to 3.4. However, sometimes the group of 32 resulting variables can have a broad range, say from 3.6 to 17.3. Overall, the ranges of these 32 variables can have even narrower or broader ranges.

What I am currently using but not feeling great about the results are, the dense ranking method, (1,2,2,3); the standard ranking method (1,2,2,4) and the fractional ranking method (1, 2.5, 2.5, 4). I know there are others as well.

My question is as follows:

Are there specific rules or guidelines which I can follow that will help me determine the best ranking method to use, as well as when and how to use it?

Many Thanks!
 
Physics news on Phys.org
  • #2
Is it possible I have stumbled upon an idea no one has ever considered before? I am beginning to think so as even Sal Kahn has no answer and has never considered the concept (and clearly the importance) of contriving a statistical ranking method which would actually provide performance relative meaning between the rankings.

I have played with the idea of a weighted ranking method which certainly presents the information more closely describing the actual differences between the ranks, although, I'm not sure if I am violating any arithmetic rules, thus the reason for this (and other) posts.

Over 300 views and not one single idea.
 
  • #3
Thank You to the Mod whom moved my thread - Sorry for the mis-post...Thanks again.
 
  • #4
StatsForLife said:
Greetings All!

I have what appears to be the most difficult question in the world based on the absence of even a single answer found on Google.

(In my opinion) people who pose real life problems involving statistics in the math sections of the forum often make the courteous gesture of attempting to reduce these problems to their mathematical essence. However, given that they are seeking mathematical advice in the first place, this is not a good idea since they usually fail to extract the relevant details.

Someone could cross -examine you and find out the crucial details ("What makes one ranking method better than another?" "What decisions will be made on the basis of the ranking?" etc.) However, I think the simplest way for you to get good advice to completely describe the real world problem as such.
 
  • #5
Stephen Tashi said:
(In my opinion) people who pose real life problems involving statistics in the math sections of the forum often make the courteous gesture of attempting to reduce these problems to their mathematical essence. However, given that they are seeking mathematical advice in the first place, this is not a good idea since they usually fail to extract the relevant details.

Someone could cross -examine you and find out the crucial details ("What makes one ranking method better than another?" "What decisions will be made on the basis of the ranking?" etc.) However, I think the simplest way for you to get good advice is to completely describe the real world problem as such.
Thank You very much for the response.

The real life problem I'm trying to solve is how to accurately compare and contrast human performance where several measurements are taken, weighted, averaged then ranked to provide meaningful performance information between the ranks.

E.g., if I were to measure the performance of 32 tennis players I would measure all critical aspects of their individual games which would include between 6-10 different measurements for each player.

Once the measurements are taken, weighted, averaged and ranked then each of the ranks are simply averaged into one overall rank. With a solid ranking method one would be able to see that, for example, the player ranked 16th is actually a lot closer to the player ranked #1 then he/she is to the player ranked 32.

Below is a random sample of data and the three different ranking methods:...Scores...Nominal...Dense...Weighted

1) 3.6...1......1......1.00

2) 3.6...1......1......1.00

3) 4.3...3......2......2.58

4) 4.3...3......2......2.58

5) 4.5...5......3......3.04

6) 4.8...6......4......3.72

7) 4.8...6......4......3.72

8) 4.9...8......5......3.94

9) 5.0...9......6......4.17

10) 5.0...9......6......4.17

11) 5.2...11....7......4.62

12) 5.2...11....7......4.62

13) 5.4...13....8......5.07

14) 5.5...14....9......5.30

15) 5.6...15....10....5.53

16) 5.6...15....10....5.53

17) 5.8...17....11....5.98

18) 6.3...18....12....7.11

19) 6.3...18....12....7.11

20) 6.5...20....13....7.56

21) 6.5...20....13....7.56

22) 6.7...22....14....8.01

23) 6.7...22....14....8.01

24) 6.9...24....15....8.47

25) 6.9...24....15....8.47

26) 7.1...26....16....8.92

27) 7.2...27....17....9.15

28) 8.8...28....18....12.77

29) 9.5...29....19....14.35

30) 12.8...30....20....21.82

31) 15.0...31....21....26.80

32) 17.3...32....22....32.00

I have also used a modified ranking which uses both the nominal and dense methods depending upon the distance between variables.

As you can see using the nominal ranking method (1,2,2,4) provides little if any information about the performance differences between the ranks.

Using the dense ranking method (1,2,2,3) is an improvement but still doesn't paint a very clear performance picture and it provides only 22 ranks for 32 scores, which may or may not be bad I don't know.

Finally, using the weighted method does, in my opinion, provide a much more accurate portrait of the performance differences between the ranks but I just made up this formula and I am not sure if it is even a viable solution.

Assuming the scores to be ranked are in cells B2-B33 of an Excel spreadsheet, the formula for the weighted method begins in cell C3 and is as follows:

1-(((B$2-B3)/(B$33-B$2))*31

To prevent the need for a boolean formula I simply enter the rank of 1 in cell C2

Again, I am simply looking for some guidance here. Ideally, I would like to know if I am on the right track or if there are some additional ranking methods you can share with me to help me achieve my goal of creating a ranking system that is as qualitative as it is quantitative.

Thank You for any help you can provide.
 
  • #6
You haven't explained what criteria make one method of ranking better than another.

Contrast your lack of a goal with the "Elo" system of ranking chess players. One goal of that system is to predict the probability that a player wins a chess game. That's a clear mission.

If the goal of your ranking is something other that predicting that kind of bottom line performance, you have to explain (or discover yourself) what the goal is. Are you trying to predict the emergence of future champions from current underdogs?
 
  • #7
Stephen Tashi said:
You haven't explained what criteria make one method of ranking better than another.

Contrast your lack of a goal with the "Elo" system of ranking chess players. One goal of that system is to predict the probability that a player wins a chess game. That's a clear mission.

If the goal of your ranking is something other that predicting that kind of bottom line performance, you have to explain (or discover yourself) what the goal is. Are you trying to predict the emergence of future champions from current underdogs?
Thanks again for responding. Actually, Stephen, I have explained it but perhaps not well enough; or, perhaps your responses illuminate what I believe is a tragic problem within the statistics community which is that ranking is not seen as an end goal in and of itself, which it absolutely is.

However, to address your inquiry, yes, one end goal of creating a more statistically viable ranking method combined with a trend analysis could be used in a predictive model. However, before we put that cart before the horse I need a more accurate ranking method.

The criteria that makes one ranking method better than another is simply how the different ranking methods benchmark the actual performance differences between players (ranks).

E.g., look at the random sample I provided and look at the nominal ranking vs. the weighted ranking. Go down to line 17 and see the score of 5.8. Although 5.8 is only 2.2 performance ranks from the top ranked player it is ranked at 17 on a nominal scale.

By comparison, on the weighted scale that same player is ranked 5.98 which is a much more accurate portrayal of how close his actual score is to both the top and bottom of the entire 32 player ranking.

The Elo ranking method has merit in perhaps a future iteration of what I am trying to do today.

Again, thank you for your time and discourse.
 
  • #8
StatsForLife said:
what I believe is a tragic problem within the statistics community which is that ranking is not seen as an end goal in and of itself, which it absolutely is.

This isn't a tragic problem with the statistics community, it's simple logic. If you don't know what information a ranking is supposed to convey and what decisions are made on the basis for the ranking, there is no reason to claim that one method of ranking is better than another. It's just an in-the-eye-of-the-beholder type of judgement.


The criteria that makes one ranking method better than another is simply how the different ranking methods benchmark the actual performance differences between players (ranks).

That's a reasonably clear goal, but you have to define "performance differences".

E.g., look at the random sample I provided and look at the nominal ranking vs. the weighted ranking. Go down to line 17 and see the score of 5.8. Although 5.8 is only 2.2 performance ranks from the top ranked player it is ranked at 17 on a nominal scale.

I don't know anything about tennis and I have no idea what those numbers mean. I also don't know which columns represent "actual peformance" or what that "actual performance" is.

A usual function of a statistical consultant is to do mind reading. I try that. I'll guess that you have defined certain measurements that can be taken about a tennis player, presumably from observing the actual games. They might be something like the average number of seconds it takes for them to move from service position of center court (if that's where they go) or the number of first serves that score points, etc. For each individual statistic, players can be ranked from best to worst. What you want is a summary ranking. You want players that are high on the summary ranking to be high in all the rankings based on the individual statistics.

By the way, we should distinguish between a "ranking" (1,2,3, etc.) and a "rating scale" (4.17, 3.72, etc.) . Some of your data appears to be a ranking and some of it appears to be a rating.
 
  • #9

This isn't a tragic problem with the statistics community, it's simple logic. If you don't know what information a ranking is supposed to convey and what decisions are made on the basis for the ranking, there is no reason to claim that one method of ranking is better than another. It's just an in-the-eye-of-the-beholder type of judgment.

The ranking is the information.

What the ranking is derived from is meaningless to the discussion. It could be 40-yard dash times or yards per carry for a running back, or minutes per foul in basketball - none of that matters.

All that matters is that when a collection of measurements from the same observation over a population of subjects is taken and ranked that the ranking provide meaningful and statistically relative information about the performance differences of the subjects.

It is conceptually quite simple to understand. I'm just not sure how to represent it mathematically so here I am. Go back and look at the list of random numbers I assembled - keep in your mind these numbers do not measure anything specific - my use of tennis players is confusing you so ignore that.

Now, it is very easy to see that most of the group of 32 performance measurements are relatively close with some outliers toward the end. Clearly a nominal ranking method would be inadequate to use; and while using a dense ranking method is better, I believe the weighted method is closer still. But, as I previously mentioned I just made that up so I am unsure of its viability.
For each individual statistic, players can be ranked from best to worst. What you want is a summary ranking. You want players that are high on the summary ranking to be high in all the rankings based on the individual statistics.
Not at all. I want the summary ranking to fairly reflect the aggregate of their accurate individual performance rankings.

By the way, we should distinguish between a "ranking" (1,2,3, etc.) and a "rating scale" (4.17, 3.72, etc.) . Some of your data appears to be a ranking and some of it appears to be a rating.

I understand what you are saying here, but for the sake of this discussion the two are synonymous.

Thanks again for sticking with this conversation.
 
Last edited:
  • #10
StatsForLife said:
The ranking is the information.

But if what you mean by "the information" is not defined, then the ranking had no meaning.

All that matters is that when a collection of measurements from the same observation over a population of subjects is taken and ranked that the ranking provide meaningful and statistically relative information about the performance differences of the subjects.

You aren't making it clear whether your measurements in this project will include several different statistics, each of which will capture a different aspect of performance or whether you are talking about taking repeated measurements of the same statistic, which measures only one aspect of performance.

I'll try another mind reading exercise. Suppose you are talking about one statistic, which measures only one aspect of a players performance. Suppose your data consists of several measurements of these statistics for each player. Suppose what you want is a ranking system so that players with (say) lower rank numbers (such as 1st) tend to have better values of this statistic.
 
  • #11

But if what you mean by "the information" is not defined, then the ranking had no meaning.

All information is defined. (Both the measurements and the ranking of those measurements).
You aren't making it clear whether your measurements in this project will include several different statistics, each of which will capture a different aspect of performance or whether you are talking about taking repeated measurements of the same statistic, which measures only one aspect of performance.

Actually, I feel I have made that perfectly clear - the answer is both.

I appreciate you trying to help but you're responses illuminate quite clearly the need to re-think statistical rankings.

Again, Thank you for all your help.
 

What is statistical ranking and why is it important?

Statistical ranking is a method used to organize and compare data by assigning ranks or scores to each data point. It is important because it allows for easy interpretation and understanding of large data sets, and can help identify trends and patterns within the data.

How is statistical ranking different from other statistical methods?

Statistical ranking is different from other methods, such as mean or median calculations, because it does not rely on numerical values. Instead, it uses the order or position of data points to compare and analyze the data.

What are the different types of statistical ranking methods?

There are several types of statistical ranking methods, including standard ranking, modified ranking, fractional ranking, and dense ranking. These methods differ in how they handle ties and assign ranks to data points.

How do I choose the appropriate statistical ranking method for my data?

The appropriate statistical ranking method will depend on the type of data you have and the purpose of your analysis. For example, if your data contains many ties, fractional ranking may be more appropriate than standard ranking. It is important to understand the strengths and limitations of each method before selecting one for your data.

Are there any potential drawbacks to using statistical ranking?

One potential drawback of statistical ranking is that it may not accurately represent the true values of the data. This is especially true when there are many ties or outliers in the data. Additionally, the choice of ranking method can also affect the results and interpretation of the data. It is important to carefully consider these factors when using statistical ranking.

Similar threads

  • Set Theory, Logic, Probability, Statistics
Replies
4
Views
996
  • Set Theory, Logic, Probability, Statistics
Replies
3
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
2
Views
943
  • General Math
Replies
1
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
4
Views
12K
  • Introductory Physics Homework Help
Replies
4
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
3K
  • STEM Academic Advising
Replies
13
Views
2K
  • Introductory Physics Homework Help
Replies
14
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
1K
Back
Top