Help With Statistical Ranking

1. Apr 14, 2012

StatsForLife

Greetings All!

I have what appears to be the most difficult question in the world based on the absence of even a single answer found on Google.

I am looking for statistical ranking methods/rules/guidelines, etc.

I work with data that is the result of taking continuous data (distance) and dividing it by discrete data (specific # of events). The results are often a group of 32 dense variables ranging from say, 1.1 to 3.4. However, sometimes the group of 32 resulting variables can have a broad range, say from 3.6 to 17.3. Overall, the ranges of these 32 variables can have even narrower or broader ranges.

What I am currently using but not feeling great about the results are, the dense ranking method, (1,2,2,3); the standard ranking method (1,2,2,4) and the fractional ranking method (1, 2.5, 2.5, 4). I know there are others as well.

My question is as follows:

Are there specific rules or guidelines which I can follow that will help me determine the best ranking method to use, as well as when and how to use it?

Many Thanks!!

2. Apr 16, 2012

StatsForLife

Is it possible I have stumbled upon an idea no one has ever considered before? I am beginning to think so as even Sal Kahn has no answer and has never considered the concept (and clearly the importance) of contriving a statistical ranking method which would actually provide performance relative meaning between the rankings.

I have played with the idea of a weighted ranking method which certainly presents the information more closely describing the actual differences between the ranks, although, I'm not sure if I am violating any arithmetic rules, thus the reason for this (and other) posts.

Over 300 views and not one single idea.

3. Apr 16, 2012

StatsForLife

Thank You to the Mod whom moved my thread - Sorry for the mis-post...Thanks again.

4. Apr 17, 2012

Stephen Tashi

(In my opinion) people who pose real life problems involving statistics in the math sections of the forum often make the courteous gesture of attempting to reduce these problems to their mathematical essence. However, given that they are seeking mathematical advice in the first place, this is not a good idea since they usually fail to extract the relevant details.

Someone could cross -examine you and find out the crucial details ("What makes one ranking method better than another?" "What decisions will be made on the basis of the ranking?" etc.) However, I think the simplest way for you to get good advice to completely describe the real world problem as such.

5. Apr 17, 2012

StatsForLife

Thank You very much for the response.

The real life problem I'm trying to solve is how to accurately compare and contrast human performance where several measurements are taken, weighted, averaged then ranked to provide meaningful performance information between the ranks.

E.g., if I were to measure the performance of 32 tennis players I would measure all critical aspects of their individual games which would include between 6-10 different measurements for each player.

Once the measurements are taken, weighted, averaged and ranked then each of the ranks are simply averaged into one overall rank. With a solid ranking method one would be able to see that, for example, the player ranked 16th is actually a lot closer to the player ranked #1 then he/she is to the player ranked 32.

Below is a random sample of data and the three different ranking methods:

...Scores....Nominal..........Dense...........Weighted

1) 3.6...........1...................1...................1.00

2) 3.6...........1...................1...................1.00

3) 4.3...........3...................2...................2.58

4) 4.3...........3...................2...................2.58

5) 4.5...........5...................3...................3.04

6) 4.8...........6...................4...................3.72

7) 4.8...........6...................4...................3.72

8) 4.9...........8...................5...................3.94

9) 5.0...........9...................6...................4.17

10) 5.0...........9...................6...................4.17

11) 5.2...........11.................7...................4.62

12) 5.2...........11.................7...................4.62

13) 5.4...........13.................8...................5.07

14) 5.5...........14.................9...................5.30

15) 5.6...........15.................10.................5.53

16) 5.6...........15.................10.................5.53

17) 5.8...........17.................11.................5.98

18) 6.3...........18.................12.................7.11

19) 6.3...........18.................12.................7.11

20) 6.5...........20.................13.................7.56

21) 6.5...........20.................13.................7.56

22) 6.7...........22.................14.................8.01

23) 6.7...........22.................14.................8.01

24) 6.9...........24.................15.................8.47

25) 6.9...........24.................15.................8.47

26) 7.1...........26.................16.................8.92

27) 7.2...........27.................17.................9.15

28) 8.8...........28.................18.................12.77

29) 9.5...........29.................19.................14.35

30) 12.8.........30.................20.................21.82

31) 15.0.........31.................21.................26.80

32) 17.3.........32.................22.................32.00

I have also used a modified ranking which uses both the nominal and dense methods depending upon the distance between variables.

As you can see using the nominal ranking method (1,2,2,4) provides little if any information about the performance differences between the ranks.

Using the dense ranking method (1,2,2,3) is an improvement but still doesn't paint a very clear performance picture and it provides only 22 ranks for 32 scores, which may or may not be bad I don't know.

Finally, using the weighted method does, in my opinion, provide a much more accurate portrait of the performance differences between the ranks but I just made up this formula and I am not sure if it is even a viable solution.

Assuming the scores to be ranked are in cells B2-B33 of an Excel spreadsheet, the formula for the weighted method begins in cell C3 and is as follows:

1-(((B$2-B3)/(B$33-B\$2))*31

To prevent the need for a boolean formula I simply enter the rank of 1 in cell C2

Again, I am simply looking for some guidance here. Ideally, I would like to know if I am on the right track or if there are some additional ranking methods you can share with me to help me achieve my goal of creating a ranking system that is as qualitative as it is quantitative.

6. Apr 17, 2012

Stephen Tashi

You haven't explained what criteria make one method of ranking better than another.

Contrast your lack of a goal with the "Elo" system of ranking chess players. One goal of that system is to predict the probability that a player wins a chess game. That's a clear mission.

If the goal of your ranking is something other that predicting that kind of bottom line performance, you have to explain (or discover yourself) what the goal is. Are you trying to predict the emergence of future champions from current underdogs?

7. Apr 17, 2012

StatsForLife

Thanks again for responding. Actually, Stephen, I have explained it but perhaps not well enough; or, perhaps your responses illuminate what I believe is a tragic problem within the statistics community which is that ranking is not seen as an end goal in and of itself, which it absolutely is.

However, to address your inquiry, yes, one end goal of creating a more statistically viable ranking method combined with a trend analysis could be used in a predictive model. However, before we put that cart before the horse I need a more accurate ranking method.

The criteria that makes one ranking method better than another is simply how the different ranking methods benchmark the actual performance differences between players (ranks).

E.g., look at the random sample I provided and look at the nominal ranking vs. the weighted ranking. Go down to line 17 and see the score of 5.8. Although 5.8 is only 2.2 performance ranks from the top ranked player it is ranked at 17 on a nominal scale.

By comparison, on the weighted scale that same player is ranked 5.98 which is a much more accurate portrayal of how close his actual score is to both the top and bottom of the entire 32 player ranking.

The Elo ranking method has merit in perhaps a future iteration of what I am trying to do today.

Again, thank you for your time and discourse.

8. Apr 17, 2012

Stephen Tashi

This isn't a tragic problem with the statistics community, it's simple logic. If you don't know what information a ranking is supposed to convey and what decisions are made on the basis for the ranking, there is no reason to claim that one method of ranking is better than another. It's just an in-the-eye-of-the-beholder type of judgement.

That's a reasonably clear goal, but you have to define "performance differences".

I don't know anything about tennis and I have no idea what those numbers mean. I also don't know which columns represent "actual peformance" or what that "actual performance" is.

A usual function of a statistical consultant is to do mind reading. I try that. I'll guess that you have defined certain measurements that can be taken about a tennis player, presumably from observing the actual games. They might be something like the average number of seconds it takes for them to move from service position of center court (if that's where they go) or the number of first serves that score points, etc. For each individual statistic, players can be ranked from best to worst. What you want is a summary ranking. You want players that are high on the summary ranking to be high in all the rankings based on the individual statistics.

By the way, we should distinguish between a "ranking" (1,2,3, etc.) and a "rating scale" (4.17, 3.72, etc.) . Some of your data appears to be a ranking and some of it appears to be a rating.

9. Apr 17, 2012

StatsForLife

The ranking is the information.

What the ranking is derived from is meaningless to the discussion. It could be 40-yard dash times or yards per carry for a running back, or minutes per foul in basketball - none of that matters.

All that matters is that when a collection of measurements from the same observation over a population of subjects is taken and ranked that the ranking provide meaningful and statistically relative information about the performance differences of the subjects.

It is conceptually quite simple to understand. I'm just not sure how to represent it mathematically so here I am. Go back and look at the list of random numbers I assembled - keep in your mind these numbers do not measure anything specific - my use of tennis players is confusing you so ignore that.

Now, it is very easy to see that most of the group of 32 performance measurements are relatively close with some outliers toward the end. Clearly a nominal ranking method would be inadequate to use; and while using a dense ranking method is better, I believe the weighted method is closer still. But, as I previously mentioned I just made that up so I am unsure of its viability.

Not at all. I want the summary ranking to fairly reflect the aggregate of their accurate individual performance rankings.

I understand what you are saying here, but for the sake of this discussion the two are synonymous.

Thanks again for sticking with this conversation.

Last edited: Apr 17, 2012
10. Apr 17, 2012

Stephen Tashi

But if what you mean by "the information" is not defined, then the ranking had no meaning.

You aren't making it clear whether your measurements in this project will include several different statistics, each of which will capture a different aspect of performance or whether you are talking about taking repeated measurements of the same statistic, which measures only one aspect of performance.

I'll try another mind reading exercise. Suppose you are talking about one statistic, which measures only one aspect of a players performance. Suppose your data consists of several measurements of these statistics for each player. Suppose what you want is a ranking system so that players with (say) lower rank numbers (such as 1st) tend to have better values of this statistic.

11. Apr 17, 2012

StatsForLife

All information is defined. (Both the measurements and the ranking of those measurements).

Actually, I feel I have made that perfectly clear - the answer is both.

I appreciate you trying to help but you're responses illuminate quite clearly the need to re-think statistical rankings.

Again, Thank you for all your help.