Stack Overflow: Classic Metric Abuse
It had to happen sooner or later, but when a Stack Overflow user posted tips to gain reputation fast, the proverbial hit the fan. Why? Go ahead and read the 6 Simple Tips, which describe how to change your contribution behaviour to optimise for reputation. Notice how none of these tips are concerned with improving the quality of your contributions. In fact, some of them actively encourage sacrificing quality in favour of gaining points:
1. Be the First to Answer. Even at the cost of quality.
In Stackoverflow answers are arranged first by vote number and submission/edit time. Being the first answer that people see is a huge advantage.
This is a perfect illustration of metric abuse in action. By using the reputation “magic number” as an incentive, Stack Overflow encourages users to optimise for reputation rather than quality. Amusingly, Joel himself posted on this very topic some years ago:
“Thank you for calling Amazon.com, may I help you?” Then — Click! You’re cut off. That’s annoying. You just waited 10 minutes to get through to a human and you mysteriously got disconnected right away.
Or is it mysterious? According to Mike Daisey, Amazon rated their customer service representatives based on the number of calls taken per hour. The best way to get your performance rating up was to hang up on customers, thus increasing the number of calls you can take every hour.
Perhaps Joel should have a word to Jeff. I suspect that the response will be to tune the reputation system to plug these holes. If that is indeed the response, I’d like to ask Jeff: Have You Met Your Dog, Patches?
This entry was posted on Thursday, September 3rd, 2009 at 5:37 am and is filed under Opinion, Technology. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.