USA Weightlifting Ranking System, does it tell us anything? Part 1

Motivation for post:

So, a few weeks ago, a long time weightlifting friend and former coach of mine (not mentioning any names, Mark Cohen), asked me to do an analysis on the USA Weightlifting (USAW ) ranking system.  Specifically, what would the National Team look like if the data points used to create the system shifted slightly. Would there be big swings in rankings 1-7?

Secondly, just for fun, I looked at the rankings, and as expected I was Nationally Ranked. However, I also looked at the Sinclair rankings-- ie the method of how our "Best Lifter" at the National Championships is chosen.  To my utter shock, I ranked significantly higher on this system. 

Why do I care?  Well, the rankings are, in the weightlifting communities eyes, how good a lifter you are, hence how "qualified" you are to represent the USA at international meets.   So, if you are going to label lifters as qualified or unqualified, then it best be based on some cogent data.

Finally, almost every year, the USA Weightlifting ranking system changes.  In other words, you could lift exactly the same weights, or even less weights, and rank several places higher. ie your goodness improves by doing nothing.  Yeah, that's not exactly a good way to motivate athletes to new heights of performances.

How the USAW totals currently work

The goal of the USAW ranking system is to select athletes who have the best chance of medaling at international meets.

In competition, there are 7 women's weight classes; the system sets  a qualifying total in each weight class.  This total is calculated by taking the AVERAGE of the 3rd place result at the last 5 world championships or Olympics (in Olympic Years, there is no World Championship). 

Technical Flaws in the system:

Allow me to start with a story. When I was a Grad Student at MIT, I took a class in Statistics.  One of the first lessons we learned, is that an average, alone, tells you nothing.  As an example our professor offered a personal example, paraphrasing (from 2009):  

      During her review, the department head reviewed survey results on her teaching.  He said her score was below the average, hence she was a below average teacher.  She countered by asking what was the standard deviation, or the spread of the data points around the average.  Looking at the plot of all the faculty's scores and the average,  her score fell close to the average (less than one standard deviation). Hence, she countered with " There is no significant difference between my score and the average.  Therefore, I am as good a teacher as someone with an average score.".

In effect, a system that only looks at averages, will not tell you how good the athlete actually is, as we do not know what the "spread" or depth is in each weight class.   My favorite example is that the 69kg class is way more deep than the 63kg class, yet the "taking an average" method only puts the 63kg and 69kg weight class qualifying totals a mere 9kg's apart.   Whereas, in reality, a lifter who lifts 80% of the total in 63kg class will probably place higher than a lifter who lifts 80.1% of the qualifying total in the 69kg class.  

Secondly, the system is in discord with our current method of ranking lifters for "best lifter" awards on Sinclair formula.

Sinclair formula aims to rate lifters across weight classes.  In other words, if all lifters were superheavyweights, what would they lift.  You might think of this as, "pound for pound", who is the best.  The formula employs a logarithmic relationship between bodyweight and weight lifted (http://www.qwa.org/default.asp).  This formula is even used at the World Championships to determine best lifters. Finally, many foreign countries, who win more medals at the Olympics in weightlifting than the USA, look at the Sinclair formula as a leading indicator of athletic performance. They use Sinclair to aid them in choosing teams.

Now, let's see what happens to the USA Weightlifting National Women's rankings if we sort the National Championships results by weight lifted, and add back in lifters who bombed out at Nationals and are still ranked (yes, the age of the lifter's qualifying total is a whole other subject).

The placement change column is the ranking by Sinclair - USAW ranking system. It shows how a lifter increases or drops in ranking, simply by switching to the "3rd place average method".

The most dramatic changes are a drop in ranking by 5 places.  As our National teams have a maximum of 7 spots per team, in non-technical terms, 5 places is a pretty big deal.

Note, there is one lifter who shows-up in our top 20 rankings (#13) and is not even top 23 in the Sinclair rankings. Likewise, one of the top 20 lifters on Sinclair does not show-up in the Top 20 official ranking.

I am not advocating that only the Sinclair formula is used to pick teams. I am asserting that it IS used by other countries, who are winning more medals than us, to help select teams, hence our ranking system should have some agreement with it.

Next Post... how sensitive our average of 3rd place at World Championships is year to year and its ability to predict placement....