a little madness

People regularly write about good and bad tech interview questions. Certain companies are famous for their interviews, there are books and websites full of such questions, and there are countless blogs on the topic. Still, many of the questions I have heard asked and seen quoted in blogs are just plain ordinary. Heck, even questions I have used myself look worse to me as time goes on.

Certain common question types are useful, such as:

• Simple coding tasks: as a quick screen. If the candidate can’t reverse a list, it’s an immediate no.
• Problem-solving puzzles: made famous by Microsoft. A good puzzle or two can be a fun way to find out how the candidate approaches a problem.

However, the quick screen doesn’t do the hard job of separating decent coders and great coders. And it can be hard to draw conclusions from a puzzle question: is the candidate just a fan of puzzles (i.e. clued in to the tricks)? I think part of the problem is we need to look at the issue a slightly different way. These questions are trying to emulate the challenges of the job, but they fall short in two key ways:

1. To fit the task into a 10 minute question, it is scaled down to a toy size. Nice for an exercise, but not very realistic.
2. The interviewers themselves know the answer, giving them a position of power over the candidate. They may be biased towards their known answer (despite a unique alternative from the candidate). They could either lead or intimidate the candidate from this position.

So what kind of question can be used to overcome these shortfalls? Simple: a question with no answer. Or, at least, a question that you don’t yet have an answer to. Every day on the job we run across these problems, and have to solve them. It’s the essence of the job. And we solve them together, not as individuals, and certainly not with an all-knowing interviewer smugly sitting on the perfect solution.

Sure, this sort of question is not nice and clean to ask. As the interviewer, you have to be willing to be as out of depth as the candidate. The benefit is you are now working on a level playing field together with the candidate, just as you would if they were hired. This both adds realism and allows you to assess their ability to fit in with the team. You can brainstorm solutions, debate alternatives, hit dead ends and make breakthroughs together. You may have no answer, you may come across many, it’s not really important. Tomorrow always brings a new problem.

So how do you come up with these questions? One possibility is harvesting them from real life examples. Is there a problem you are struggling with today? If it makes sense in isolation, it could be a perfect candidate. Is there a design issue you run across every now and again, and never seem to have a good answer? Note it down. If you’re really stuck, try what we did a couple of times at my last company. Pick a random question out of a programming competition archive: one that you have never seen before. Take it into the interview blind, don’t have a head start on the candidate. Some realism is lost, but many of the benefits remain.

Happy interviewing!

This entry was posted on Tuesday, October 10th, 2006 at 1:03 pm and is filed under Technology, Business of Software. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.