a little madness

A man needs a little madness, or else he never dares cut the rope and be free -Nikos Kazantzakis


Archive for September, 2009

Zutubi @ CITCON Paris 2009

Any excuse is good enough to get me to Paris, especially while it is only a train ride away. Daniel has actually been tempted all the way from Sydney!1 So you’ll find us both at CITCON Europe 2009 tomorrow night and Saturday. We’re both looking forward to a great weekend, after nothing but positive experiences at previous events. Hopefully we’ll even get a few questions about the new Pulse 2.1 Beta while we’re there!

1 Although combining it with a well-deserved holiday may have been a factor…

Software As Engineering

Bertrand Meyer (of Eiffel fame) posted recently on The one sure way to advance software engineering. Meyer’s thesis is that by thorough examination of past software failures, we could learn from those mistakes. This is true enough, but then Meyer goes on to suggest that we:

… pass a law that requires extensive professional analysis of any large software failure. … The law would have to define what constitutes a “large” failure; for example it could be any failure that may be software-related and has resulted in loss either of human life or of property beyond a certain threshold, say $50 million.

Leaving the issue of making this law aside, the fact that this would only apply to such “large” failures necessarily marginalises the utility of the analysis. The truth is that such projects are a small minority of overall software development. Lessons learned from projects where such a large failure is a possibility – particularly where safety is involved – may not apply to the vast majority of software development. This doesn’t make the suggestion useless, but it makes it unlikely to have a significant impact on software engineering as a whole.

Meyer’s post also triggered a resurgence of the age-old debate of whether software development is worthy of the “engineering” label. With bugs so prevalent in mainstream software, people (mostly developers themselves) question the professionalism of our industry. This reminds of the EclipseCon 2007 keynote by Robert Lefkowitz, which draws attention to the EULA used by Microsoft. In particular, Lefkowitz asks why Microsoft1 should be allowed to disclaim all warranty and liability for damages incurred by the use of their software. Why doesn’t the law protect the market from buggy software?

The answer is simple: we aren’t all Larry Ellison, and cost does matter to the rest of us. The market might wish for higher-quality software, but to make this law would lead directly to a huge increase in the cost of software development. Consider the canonical example of software that is made to the highest standard: the code that controls space shuttles. The linked article quotes a figure of $35 million per year to develop and maintain 420,000 lines of code2. Consider that Microsoft Windows is about two orders of magnitude larger than this, and that the complexity of a code base grows faster than linearly with respect to the number of lines of code. The cost of bringing Windows up to shuttle-quality is unbearable, even for an 800-pound gorilla.

Does the fact that we need to trade off quality versus cost mean that development should not be considered as engineering? On the contrary, I believe that making sensible trade-offs is one of the very things that defines engineering. Engineering operates in the real world of scarce resources and budget constraints, and great engineers how to strike the right balance. They can also do more with less, by caring about their work and always seeking efficiency. I’ve met and worked with many developers that exhibit these traits, so I believe that there is hope for us yet.

1 I presume as a proxy for the whole industry.
2 Lines of code may be a weak metric, but I don’t think it detracts from the main point.

Stack Overflow: Classic Metric Abuse

It had to happen sooner or later, but when a Stack Overflow user posted tips to gain reputation fast, the proverbial hit the fan. Why? Go ahead and read the 6 Simple Tips, which describe how to change your contribution behaviour to optimise for reputation. Notice how none of these tips are concerned with improving the quality of your contributions. In fact, some of them actively encourage sacrificing quality in favour of gaining points:

1. Be the First to Answer. Even at the cost of quality.

In Stackoverflow answers are arranged first by vote number and submission/edit time. Being the first answer that people see is a huge advantage.

This is a perfect illustration of metric abuse in action. By using the reputation “magic number” as an incentive, Stack Overflow encourages users to optimise for reputation rather than quality. Amusingly, Joel himself posted on this very topic some years ago:

“Thank you for calling Amazon.com, may I help you?” Then — Click! You’re cut off. That’s annoying. You just waited 10 minutes to get through to a human and you mysteriously got disconnected right away.

Or is it mysterious? According to Mike Daisey, Amazon rated their customer service representatives based on the number of calls taken per hour. The best way to get your performance rating up was to hang up on customers, thus increasing the number of calls you can take every hour.

Perhaps Joel should have a word to Jeff. I suspect that the response will be to tune the reputation system to plug these holes. If that is indeed the response, I’d like to ask Jeff: Have You Met Your Dog, Patches?