@Ixidane Most of the players doing LFR just didn't raid at all before. They were never really eligible for recruitment.
— Greg Street (@Ghostcrawler) June 6, 2013
I've seen this tweet before, and it annoys me. Even though it seems like GC is rebutting Ixidane, both statements can be true at the same time.
To see this, imagine a population of 100 players. Before LFR, 20 of them participate in regular raids. We'll call these 20 players Raiders, and the other 80 players Casuals.
Then LFR comes out. 10 Raiders switch to LFR, along with 30 Casuals. So the following statements are both true:
- LFR has significantly damaged normal raiding. Normal raiding lost 50% (10 of 20) of its players.
- The majority of people in LFR didn't raid before. 75% of raiders in LFR (30 of 40) didn't raid before.
I'm not saying that Ixidane's contention, or statement 1 above, is true. It might very well be false. I don't have the numbers to verify, though the various progress sites are showing a significant reduction in normal raiding guilds and raiders.
But GC's rebuttal is deliberately misleading, spinning numbers in a way that appears to refute the argument, but really does not.
But GC's rebuttal is deliberately misleading, spinning numbers in a way that appears to refute the argument, but really does not.
I think this bothers me because I see it more and more on the internet. Getting people to give straightforward numbers in favor of their arguments is like pulling teeth. Numbers are always phrased to be slightly misleading, shown as a percentage when the denominator is not exactly what you would expect, or is correct to use. Or the number is normalized in some misleading fashion, and the normalization is waved away.
Lately, I get highly suspicious anytime I see "average, "percent", "most" or "majority" in support of arguments. These statistics never seem to be the expected or obvious statistic. It's always a statistic deliberately chosen to prop up one side of the argument. Raw numbers showing the totals and breakdown into each category are always to be preferred, but are rarely shown.
It wouldn't shock me if Blizz' devs were explicitly told to not provide detailed numbers on anything, because investors might get wind of it.
ReplyDeleteGC's argument might also be valid because of design decisions in Mists, as they eliminated development of 5-mans in favor of Scenarios and LFR. Casuals who would have gone into 5-mans instead are being funneled into LFR.
If my experience working at a newspaper was anything to go by, choosing statistics to evade the argument is just part of the job. As a consequence, I'm always suspicious of statements that use the key words you mention. Great post :)
ReplyDeleteFrankly neither of those tweets has much value. Someone stating that point z is a fact, without offering supporting evidence, is no more valid than hazy statistics and weasel words. Emotional appeal is just that and does not provide a basis for rational discussion.
ReplyDeleteGC's counter to @ixidane is no more deliberately misleading than the argument that if you remove LFR, all the players currently running LFR will flock to guilds and run normals modes, which is the point that GC was pushing back against with his comment. Way to ignore the context that doesn't fit your argument.
ReplyDeleteI agree with JThelan. Focusing in on this pair of tweets is also misleading the argument, since if you expand it out some more you see that it is, in fact, talking about something specific. The rebuttal was for an earlier point being made.
ReplyDeleteThe specific argument was basically "LFR is terrible because it doesn't give incentive to progress" -> "Yes, but now thousands of players who were never raiding before are now raiders and potentially recruitable" -> "People who do LFR just don't want to do anything else" -> "Most of those people weren't doing anything else before anyway."
Are both statements still true? Yes. And GC didn't say anything to dispute that "normal" raiders have been lost, nor did he really bother to go into the fact that LFR could turn never-raiders into normal raiders just by being an easier on-ramp; he simply acknowledged that while there's normal raiders that went "down" to LFR, the majority of that pool never raided in the first place, and thus wouldn't have been recruitable if LFR never existed.
To put my own opinion on the overall matter, LFR itself is not the reason raiding guilds are losing people, or are finding it hard to recruit. It is because what raiding guilds offer, in the sense of someone who isn't already in one, simply isn't attractive.
The process of needing to, for all intents and purposes, apply for a job, and then have to deal with people who may or may not be shitheads, and everything that is associated with that is stressful, and not something most people look forward to when they play a GAME. So it makes sense to me that people who are raiders for the story and challenge would be far more interested in match-made versions of that to get out of all that shitty drama.
Are all raiding guilds like this? No, but the ones who are complaining about losing people tend to be. Their problem isn't that LFR is killing raiding. Their problem is that they are no longer providing a compelling argument as to why someone should join them.
@RJ, JThelen, you guys are reading too much into my post.
ReplyDeleteI'm not saying anything about whether LFR is good or bad. This post is about a very narrow topic, the use of the wrong statistics in an argument.
I agree with your facts but disagree with the sentiment.
ReplyDeleteThese are competitive businesses, so WoW will never give out much statistical info. Not so much for investors as competitors. How valuable would it be for a TESO or Wildstar to get a printout of all the userdata from WoW.
Without knowing what % of the customers is covered by WoWProgress, we cant really say much definitive. If 90% of the customers are in Asia (# for example only) then a 10% decline in Asia would offset a 90% increase in the west so saying the WoWProgress # went up does not tell all we would like to know.
Since it is nearly impossible to see comprehensive numbers, I at least prefer to see some actual (not made up) numbers. The alternative is the usual posts of "Everyone knows that X is garbage and Bliz should focus on Y and Z is OP.
@Rohan: No, I get that. My second segment was intentionally a tangent. My first part, though, was focusing on how the conversation was more then just those two tweets.
ReplyDeleteRJ, "Are all raiding guilds like this" . Every 25 man raiding guild that was a 25 man is now 10 man on my server. Their are some newer 25's but Every older 25 is now 10 or gone.
ReplyDeleteBut in my opinion the loss of casual 10's caused more harm then LFR.
Raiding right now is LFR, hard and really hard. Hopefully flex will resolve that.
Rohan your argument was obvious to me as well straight away. Their was actually a puff piece about lfr in 5.0 with information from Blizzard that indicated the number of players in lfr was about 10x the number of raiders.
GC might also be spinning it by saying people who raided in Cataclysm before LFR weren't the primary audience of LFR. But maybe many of them raided in Wrath.
ReplyDeleteThis is why they say there are lies, damned lies and statistics :)
ReplyDeleteTo address the issue, however, the reduction in Normal Mode raiders at the same time as LFR was introduced does not imply causation. There could be any number of related facts that influence this.
What Blizzard have is a team of experts who have access to all of this data. They can follow the journey of individual players through the game in complex ways. They know if your 10 players also reduced their game time, suggesting they have less play time. They know if your 10 players left the game and were replaced with 10 new players who all went to LFR. They know why those players left the game (based on feedback). They know how many players came back to the game when LFR was released and then played LFR. They know if the reduction in normal mode raiding began before LFR was introduced.
Blizzard has all the numbers. What they can't do is share them on Twitter. Partly because of the character limit and partly because gaming companies don't share all their data if they want to survive.
Once upon a time, I managed to prove, to a statistically significant degree, that my classmates were psychic. I didn't fake the result - I just had a very small population and a bit of luck. If Blizzard wanted to pull the wool over our eyes they could release lots of numbers to 'prove' their point and still hide whatever they wanted to hide.
So there are three options:
1. Blizzard is being honest and we trust them.
2. Blizzard's analysts are incompetent and can't see what is really going on.
3. Blizzard are lying as part of some evil master plan.
The choice is yours.