Monday, June 21, 2010

Measuring Performance with Plus-Minus

Measuring performance in a raid is somewhat hard. You can do a bit with damage and healing meters, but those meters are often skewed by the specific fight or class or spec. Sometimes it's hard to track the small things, like people who hit cooldowns at the perfect time, or kite mobs perfectly, or step beyond their specific role at the absolute correct time.

There is a statistic in ice hockey called Plus-Minus. If you're on the ice when your team scores an even-strength goal, you get +1. If you're on the ice when the opposing teams scores, you get a -1. So at the end, if you're positive, it implies that you are helping your team. If you are very negative, your presence on the ice actually hurts the team.

Of course, it's not a perfect stat. Since it relies on the team, a person on a bad team usually has a lower plus-minus than a person on a good team.

But it might be an interesting stat to use to measure a player's contribution to her raid team. After all, the ultimate goal is killing bosses. If you being in the raid means that bosses are more likely to be killed, then you are doing well. If the raid is more likely to wipe if you are in raid, then that's a problem. Rather than trying to measure and quantify the individual aspects of your role, we could just try and measure your effectiveness.

Here's how I'd set up a Plus-Minus system for a Raid Guild:
  1. +1 for each boss kill.
  2. -1 for each boss wipe.
  3. Only count a maximum of 4 wipes per boss. This is so that raids with an obviously bad composition don't just give up to keep from wrecking their rating.
  4. Don't count wipes on new bosses that haven't been defeated.

Going by this system, I was +4 this week. Not bad, but not really good either. But Plus-Minus is a relative system. So it would depend on what scores my other guildies would have.

Of course there are issues with this system. People who are in the raid all the time will have the same rating. It's not a problem in hockey because people are constantly taking shifts out on the ice. Certain bosses are more likely to cause wipes than other bosses. The method only works if you actually expect there to be some wipes.

Ultimately, killing bosses is the goal. Measuring effectives by how much a raider contributes to that goal might be a better method than trying to rank damage meters. But on the other hand, going through meters gives feedback on how to improve. A Plus-Minus system only identifies outlying players in either direction. It does not say how that player makes the raid better or worse.

9 comments:

  1. I see two more potential issues with such a system:

    - It could easily encourage raiders to show up for farm nights, while discouraging them from showing up to progression nights and hard bosses to keep their score up.
    - Because your personal rating is reflective of team performance, it could create a lot of animosity and finger pointing towards the person who is perceived to have caused the wipe.

    ReplyDelete
  2. You'd need to normalise to the number of attempts, or something.

    Otherwise, the magnitude of positive/negative would be dependant on attendance, and magnitude comparisons wouldn't be useful.

    ReplyDelete
  3. I like the idea, you may wish to but ecclesiasticaldiscipline, is right... that does promote showing up to farm nights.

    You could change it to +2 for a guild first boss kill, and - 0.5 for a whip to an unkilled boss. To help 'promote' progression.

    ReplyDelete
  4. Well, in my system, unkilled bosses don't actually lower your score. So real progression nights can't adjust your score down.

    I think trying to use one system to handle both attendance and performance is hard. Better to have system to measure attendance, and another for performance.

    ReplyDelete
  5. Interesting system, but I would try to use it in conjunction with maybe some other tools as well.

    In hockey, they don't just look at +/-... they have goals, assists, etc. You can be the best scorer on your team with a minus rating in theory, which would suggest a good player on a crappy line.

    So, if you're going to evaluate the player, you should look at more than just one stat.

    ReplyDelete
  6. +5 points just cause I'm Canadian? We know our ice hockey:)

    ReplyDelete
  7. I dunno about he whole +/- system. It might be alright for comparing different players, but measuring against yourself could be done a lot better. For instance, the number of times your assignment died, the amount of time s/he spent above 50%/75%/90% vs how much overhealing. Just like in sports, the introduction of statistics will take a while. Maybe we should be looking at "Adjusted +/-" from basketball instead :)

    ReplyDelete
  8. I like that you are trying to quantify raid participation/effectiveness based on team success rather than individual success but this system has one very fatal flaw. I think ecclesiasticdiscipline and Inquisitor already touched on it...

    On my server only one guild has beaten HLK. It took them weeks to do it. They are a very accomplished guild that blew through every other fight in ICC. We all know that fights become ten times easier once the boss has been killed the first time, especially if the same people are showing up consistently to raid.

    In an instance like ICC, there are some easy fights and then some difficult fights and then one really tough fight. Your system doesn't factor in a learning curve. In professional hockey every knows how to play. In WoW each boss requires a certain degree of time to learn your role in that fight. Since each boss fight is relatively unique this incoporates a dynamic factor that hockey just can't support.

    The main idea I'm trying to support here is that all boss fights aren't created equal. Applying a +/- to every fight assumes that every fight is created equal. I think HLK is THE hardest boss in the expansion by far. LK on normal took us forever to beat as well. How would you account for this?

    ReplyDelete
  9. Not only is it true that 'all boss fights aren't created equal', but all kills and wipes aren't created equal, either. A system like this doesn't really reflect on an individual's effort/contribution to the group. It's a nice idea, and I believe it's always good to try to find a fresh way to evaluate players, but I think it falls short. Meters, as flawed as they are, can tell you much more if you look at them right. Unfortunately, that's a skill that not enough people have.

    ReplyDelete