How I Hire Programmers (Aaron Swartz's Raw Thought)
How I Hire Programmers
There are three questions you have when you’re hiring a programmer (or anyone, for that matter): Are they smart? Can they get stuff done? Can you work with them? Someone who’s smart but doesn’t get stuff done should be your friend, not your employee. You can talk your problems over with them while they procrastinate on their actual job. Someone who gets stuff done but isn’t smart is inefficient: non-smart people get stuff done by doing it the hard way and working with them is slow and frustrating. Someone you can’t work with, you can’t work with.
The traditional programmer hiring process consists of: a) reading a resume, b) asking some hard questions on the phone, and c) giving them a programming problem in person. I think this is a terrible system for hiring people. You learn very little from a resume and people get real nervous when you ask them tough questions in an interview. Programming isn’t typically a job done under pressure, so seeing how people perform when nervous is pretty useless. And the interview questions usually asked seem chosen just to be cruel. I think I’m a pretty good programmer, but I’ve never passed one of these interviews and I doubt I ever could.
So when I hire people, I just try to answer the three questions. To find out if they can get stuff done, I just ask what they’ve done. If someone can actually get stuff done they should have done so by now. It’s hard to be a good programmer without some previous experience and these days anyone can get some experience by starting or contributing to a free software project. So I just request a code sample and a demo and see whether it looks good. You learn an enormous amount really quickly, because you’re not watching them answer a contrived interview question, you’re seeing their actual production code. Is it concise? clear? elegant? usable? Is it something you’d want in your product?
To find out whether someone’s smart, I just have a casual conversation with them. I do everything I can to take off any pressure off: I meet at a cafe, I make it clear it’s not an interview, I do my best to be casual and friendly. Under no circumstances do I ask them any standard “interview questions” — I just chat with them like I would with someone I met at a party. (If you ask people at parties to name their greatest strengths and weaknesses or to estimate the number of piano tuners in Chicago, you’ve got bigger problems.) I think it’s pretty easy to tell whether someone’s smart in casual conversation. I constantly make judgments about whether people I meet are smart, just like I constantly make judgments about whether people I see are attractive.
But if I had to write down what it is that makes someone seem smart, I’d emphasize three things. First, do they know stuff? Ask them what they’ve been thinking about and probe them about it. Do they seem to understand it in detail? Can they explain it clearly? (Clear explanations are a sign of genuine understanding.) Do they know stuff about the subject that you don’t?
Second, are they curious? Do they reciprocate by asking questions about you? Are they genuinely interested or just being polite? Do they ask follow-up questions about what you’re saying? Do their questions that make you think?
Third, do they learn? At some point in the conversation, you’ll probably be explaining something to them. Do they actually understand it or do they just nod and smile? There are people who know stuff about some small area but aren’t curious about others. And there are people who are curious but don’t learn, they ask lots of questions but don’t really listen. You want someone who does all three.
Finally, I figure out whether I can work with someone just by hanging out with them for a bit. Many brilliant people can seem delightful in a one-hour conversation, but their eccentricities become grating after a couple hours. So after you’re done chatting, invite them along for a meal with the rest of the team or a game at the office. Again, keep things as casual as possible. The point is just to see whether they get on your nerves.
If all that looks good and I’m ready to hire someone, there’s a final sanity check to make sure I haven’t been fooled somehow: I ask them to do part of the job. Usually this means picking some fairly separable piece we need and asking them to write it. (If you really insist on seeing someone working under pressure, give them a deadline.) If necessary, you can offer to pay them for the work, but I find most programmers don’t mind being given a small task like this as long as they can open source the work when they’re done. This test doesn’t work on its own, but if someone’s passed the first three parts, it should be enough to prove they didn’t trick you, they can actually do the work.
(I’ve known some people who say “OK, well why don’t we try hiring you for a month and see how it goes.” This doesn’t seem to work. If you can’t make up your mind after a small project you also can’t make it up after a month and you end up hiring people who aren’t good enough. Better to just say no and err on the side of getting better people.)
I’m fairly happy with this method. When I’ve skipped parts, I’ve ended up with bad hires who eventually had to be let go. But when I’ve followed it, I’ve ended up with people I like so much so that I actually feel bad I don’t get to work with them anymore. I’m amazed that so many companies use such silly hiring methods instead.
November 29, 2009
Read comments.Comments
Yup.
posted by Russell L. Carter on November 29, 2009 #
Thanks for writing this. I’ve always wondered why tech hiring is so screwed up. For an industry that thinks a lot about a lot of things, hiring is one thing that doesn’t seem to have changed much over the years. Scaling this method of hiring might be tricky though. Another idea worth trying is to hire everyone as an “intern” initially and then, make a decision after a month of actually working with them.
+1. Totally agree.
However, for a different point of view check out this StackOverflow question:
Exception programmers are also creative - nothing was mentioned of that. They can see beyond the smaller details of a task and problem. They can put themselves outside of themselves, into the user space.
posted by Mark Rushing on November 29, 2009 #
The problem with asking about what a person has done, is almost uniformly, people will answer with what ‘their team’ has done, couched as if they did it. They are familiar enough with what the team has done, to talk about it semi-intelligently.
Sorry, i want them to answer something in front of me. It doesn’t have to be real hard, but it has to be real, and they should ace it.
posted by MeBigFatGuy on November 29, 2009 #
“Someone who gets stuff done but isn’t smart is inefficient: non-smart people get stuff done by doing it the hard way and working with them is slow and frustrating. “
I couldn’t read anything passed that stubborn remark.
posted by on November 29, 2009 #
I really like your idea for figuring out if people are smart, especially the bit about asking what they’ve been thinking about, and continually probing (in a non threatening way) to see what level they understand something.
Very good ideas that I’ll be stealing from next time.
What I didn’t get though, was how you determine if someone actually gets something done. I’ve met a few smart people who have extreme understanding of a variety of complex subjects, but just don’t produce an awful lot.
Perhaps theres no way to tell for that particular case. There have also been times, though the exception, when I had talked to someone for an hour, been confident they were smart, and have them utterly fail the most simple programming test (fizbuzz, recursive method, etc.)
I wonder if your method would have ‘screened’ out those candidates that appeared smart in the hour I chatted with them…
posted by Vincent on November 29, 2009 #
The problem with a code sample is that it can be the product of other people. Even if the entire thing might not be the product of someone else, it could of been reviewed by many other people.
And if they can’t code under pressure how are they going to fix a bug that needs to be dealt with immediately. No one performs excellent under pressure, but a substantial amount of their programming abilities shouldn’t dissipate because of it.
The comments here and on Hacker News reminded me I left out a step. I’ve added it:
If all that looks good and I’m ready to hire someone, there’s a final sanity check to make sure I haven’t been fooled somehow: I ask them to do part of the job. Usually this means picking some fairly separable piece we need and asking them to write it. (If you really insist on seeing someone working under pressure, give them a deadline.) If necessary, you can offer to pay them for the work, but I find most programmers don’t mind being given a small task like this as long as they can open source the work when they’re done. This test doesn’t work on its own, but if someone’s passed the first three parts, it should be enough to prove they didn’t trick you, they can actually do the work.
(I’ve known some people who say “OK, well why don’t we try hiring you for a month and see how it goes.” This doesn’t seem to work. If you can’t make up your mind after a small project you also can’t make it up after a month and you end up hiring people who aren’t good enough. Better to just say no and err on the side of getting better people.)
posted by Aaron Swartz on November 29, 2009 #
great post. i loved it. it can be applied to so many other jobs besides programming.
posted by winzoAL on November 29, 2009 #
This seems like a good approach, but the one part I disagree with is having to provide a code sample. I don’t program outside of work, apart from the odd script or php debug. All my code samples would be from uni, and while some of them are pretty good, most are of the oh-god-five-hours-to-deadline-quick-finish-it! level of quality. This might be because I’ve only just finished uni, but only two of the 40+ people on my course have done any sort of extra-cirricular programming.
I’d like to think I’m a pretty decent programmer, but have never written (production-level) code in my free time (I prefer to do things not related to my day job). Sure you’ll be getting the cream of programmers if you request a code sample, but you’ll be missing out a lot of decent ones at the same time.
posted by Fin on November 29, 2009 #
That’s an interesting way to hire somebody, though I’m not sure that completely removing the “interview” part is a good thing. I mean, if I’m hiring a programmer for working over giant datasets at the very least I’d ask for how a binary search works and when it makes sense to use it or how he’d process a 4Tb file. Besides, I agree that this part should be reduced to a minimum, so we can at least know that the applicant knows the required theory. Then looking at source code he has written is much more interesting than asking him to write down a working quicksort.
posted by Ivan on November 29, 2009 #
You can also send comments by email.
than dead-centre, or foveal, vision. And in 2005, an American team suggested that 
Comments [0]