You are what you measure
avatar

Copyright: <a href='http://www.123rf.com/profile_scorpp'>scorpp / 123RF Stock Photo</a>That’s an oversimplification, really. Maybe it’s more that what you become is influenced by what you measure. I’ve been thinking about measurements in EVE ever since CCP kicked off the EVE 11 Celebration with incentives for unlocking more new Gecko drones.

It’s great that they based the reward on several factors — ore mined, NPC bounties collected, manufacturing jobs run, and ISK destroyed. This gives everyone something to work toward. And it turned out that of the four groups, miners and industrialists hit both their targets.

While including different categories of activity is definitely a step in the right direction, I see this as more of the same somewhat simplistic measurement that has shaped EVE culture in a way that may not be the best path to grow the game and make things more fun.

Let’s look at the manufacturing incentive. Simply tallying the number of jobs favors lots of smaller runs and doesn’t do anything for the person who’s making, say, Lokis or Titans or POSes. Those take time and multiple steps. To hit the goal for Geckos, we’d be better off running a bunch of quick ammo jobs.

In fact, I did just that when I logged in the other night — cranked out a ton of Hybrid Antimatter M. To me, this is akin to minmaxing in D&D. Not that minmaxing is *bad* — at BadWrongFun, we try to stay away from UR DOIN’ IT WRONG — but it seems like measuring a wider array of factors would be to more people’s benefit. It doesn’t cut anything OFF, and it opens us all up to more ways to progress in the game.

You have to be careful what you choose to measure. If you’ve ever worked for a tech support organization, you know what happens if the biggest (or even the only) measure of your performance is how many tickets you close. People cherry-pick tickets so they can close more, and they often rush through to a solution just to get the damn thing finished to keep their numbers clean.

Fortunately, most support organizations have wised up and started measuring performance on a more complex set of factors, and they weight them according to whatever they consider important or are trying to improve.

As humans, we want to feel like we’re getting better at stuff. Some also want to get better than other people. And that’s all okay. In fact, it’s a good thing, up to a point.

It’s also human nature to do things the easiest way possible. That’s not what we do all the time, but the tendency is there in everyone. It has to be. You’re hungry for a peanut butter and jelly sandwich, so you throw it together and proceed to chow down. You don’t paint a Monet with the jelly unless there’s an actual reason for it.

Combine these two things with simplistic measurement and you end up with outcomes you may not have wanted. Instead of really getting better at fighting, for example, a person may be inclined to just kill as many easy targets as possible. Their killboard wins and ISK destroyed tallies go up. That’s tangible evidence they’re improving, right?

I’ve spent a while reading interview forums for PVP corps, and I sometimes see recruiters asking pointed questions about win-loss ratios. One pattern I’ve seen is a recruiter speculating that a candidate’s unusually high win-loss ratio is the result of taking too few chances. He may suggest the candidate is repeatedly pursuing easy kills and not challenging himself enough.

But chasing volume is exactly the behavior that things like killboards and the EVE 11 contest reward. What you measure has a strong effect on what you become, and we see that today in EVE.

spartans_8002500 years later, we still talk about the 300-ish Spartans facing down 500 times as many Persians. In EVE, we don’t often have a convenient way of measuring something like that. A killboard for Thermopylae would show a bunch of ISK and pods lost, quite a lot even on the winning side, but not necessarily the fact that the guys on the losing side were so vastly outnumbered. An amazing story can get lost if volumes are all you look at.

I think it’d improve EVE’s fun factor if we made an effort to measure things less simplistically. Let’s say you get more points for a battle where you and your pals in T1 frigates and cruisers managed to hold your own against a larger gang in Tengus, than if there were six of you in PVP-fit ships popping some guy in an Iteron.

The data is out there for doing more complex analyses of battles and, for that matter, industrial activities. The SQL is easier to write for ISK destroyed or manufacturing jobs run, but I think the benefits for painting a more accurate picture of what went down are worth the extra effort.

It’s in all of our best interests to attract more new players to EVE and to bring old players back. To do that, we need to be supporting a broader spectrum of play styles and giving people ways to see improvement in whatever path they’ve chosen. We shouldn’t abolish the somewhat simplistic measurements we already have, but we definitely need to add new ones that show more angles of what’s going on.

Posted in Chatter, Gameplay Tagged , , , permalink

About Suzariel Kel-Paten

Software developer by day, writer and EVE player by night. My three mains are Suzariel Kel-Paten, Ystvan Chang, and Weyland Zaslow. I live in the Southeastern US with my husband and our cats and dogs. I have a Twitter account (@suzariel), but I mostly just read and don't often post there.

Comments

You are what you measure — 1 Comment

  1. Oddly enough, a “vigorous discussion” about killboards broke out on Twitter sometime yesterday and has been continued on EVEOGANDA in a post today by Rixx Javix.

    This post didn’t start the Twitter discussion — the timing isn’t right, and besides, I don’t think any of the participants read BWF — and the discussion didn’t inspire this post. I started it several days ago and didn’t post it until I’d had some time to dial back the snark.

    It’s just funny to me how these two things were going on in parallel.

Leave a Reply

Your email address will not be published. Required fields are marked *