I saw a really interesting discussion on EJ about those Top 50 DPS class leaderboards. The argument is that, because the leaderboards take the top 50 parses, the class with high variance--perhaps due to a high crit rate--in their DPS end up higher on the boards than classes with low variance.
For example, if you took two classes that did 5000 DPS, one with a 50% crit rating, and the other with a 20% crit rating. If each class did 100 fights, and you picked the Top 20 fights, the class with the 50% crit rating is going to be on the top. This is because the Top 20 fights are fights with a greater than average critical strike rate, inflating the damage done.
So if the Top 50 boards aren't a reliable way to measure Class DPS, what would be?
One idea is to look at the average DPS across *all* parses. The problem here is that as you drop lower and lower, skill and gear becomes dominant factors. At least the Top 50 parses are probably going to be equally skilled and geared, so you're only looking at the differences between classes. But maybe you can make the assumption that each class is equally likely to have under-skilled and under-geared players. Is that a good assumption? I'm not sure.