Articles like this remind me of everything that's wrong with the tech hiring process. Especially the linked "new-hire checklist", which says that you should demand "imbalance" and "a glass full of kool-aid" in your candidates, but mysteriously downplays the importance of "domain knowledge" (really...this is what the author considers to be good advice?)
Beyond that, I don't think I'm the only good developer who is tired of the "gauntlet" mentality in tech interviews. Coding questions are fine and puzzles are sometimes acceptable, but you have to remember that their only purpose is to answer one question: does the candidate know how to code? If you already know the answer to that question, it's time to sell the interviewee on your company -- not to alienate them by beating them senseless with ever-more-difficult problems and brain teasers.
Every interview question should satisfy one of two criteria:
1) if the candidate answers correctly, would it make me want to hire him/her?
2) if the candidate answers incorrectly, would it make me want to avoid him/her?
If your questions are "hard", but they don't satisfy one of these requirements, they're useless. And importantly, once you've got a sense of the answer to these questions, you're done with the interview. There's no need to drag it on, and subjecting a candidate to a grueling, capricious interview process can only serve to alienate her from your company.
Agree with the premise that interview questions should provide useful information.
However, recruitment is a delicate dance (much like any kind of relationship where there's both "buying" and "selling").
Anything that's overly formulaic is probably sub-optimal.
At the end of the day, you're trying to have a conversation that helps both parties determine if it's a good "fit". There should be an overarching "goals" for the questions, but not everything that's said or asked will neatly satisfy the criteria. Successful recruiting is more nuanced.
What's more formulaic: asking a small set of questions that are known to provide real information about a candidate's skill, or conducting an eight-hour "gauntlet" interview loop of brain-teasers and puzzles?
Questions assessing "fit" satisfy the criteria I outlined: a good answer "fit" question will make you want to hire the candidate. A bad answer will make you want to avoid her.
I agree with you on every point. I hate gauntlet interviews/interviewing and the questions should definitely be targeted towards hire/nonhire.
Only I don't think you meant this the way it sounded, so just to be clear: "does the candidate know how to code?" shouldn't be the only criteria you're testing for in a good hire.
There's much more to a good engineer than how well they code, of course. Technical skills are key, but hiring managers need to ask things that also help reveal behavioral things like communication, teamplay, unstoppable ego, etc.
These can all be major problems and, in my experience (hundreds of interviews now) by the time they're at an on-site interview, these should be a major (if not the majority of) focus. Ideally, if you handle your interview process in an efficient manner, you should be pretty comfortable with the technical skills before an on-site interview where the primary technical questions/tests should be just to verify they weren't cheating off-site.
In the world of ajaxy collaborative text editors (e.g. Google Docs) and screen sharing tools (e.g. dimdim/GoToMeeting/WebEx), etc, there's no reason you shouldn't include programming assignments as part of phone screens using their home machines, observing their coding live on the phone/screen. Bringing candidates on-site without doing something like this (especially for a 6-hour gauntlet interview circus) is risking a day of theirs and a day of yours to find out something you could gather without bringing them in.
The thing I hate about Gauntlet interviews is the emphasis on having everything right at the top of your head.
There are a million things I know how to do but have filed away. Like writing a typedef for a C function pointer -- I just check out a source file I wrote before or man qsort because I don't remember the exact syntax. Or tricks with data structures -- I WROTE my own linked lists and trees but it's not something I do every day because it's pointless. Or certain Python tricks -- if I haven't done them in a while I just look them up.
Gauntlets always turn out to be Gotchas!
The more junk you know, the less of it you are focused on at any given moment. Give any CS major the exact same tests he took, 5 years after the fact, on the spot, and it's probably a straight fail.
I remember the general sense of things and outsource the details to my prior work, open source code I can refer to, and Google.
I like this overall. I read Joel Spolsky's book on hiring techies and I've tried his interview process for the past ten interviews I've done. Some things work and some things don't.
I would add to this article and to Joel's approach that you need to ask some basic coding questions during the telephone interviews to at least pick up on things to review during the in-person interview. Example: for a programming job, I want to know if a person knows the size of an "int" or if they can tell me what the difference between a struct and a class is. If they can't, then I need to make the decision right now to drop this candidate or consider them for a more junior position.
Here's what happened to me recently: I have a job opening for a junior programmer (still do actually) and I had a phone interview scheduled. I prepped my notes and asked what I thought were good questions ("Tell me about that job. Tell me about a challenge you had. Where do you see yourself in 5 yrs?"). The candidate breezed through these questions and I was excited.
For my in-person interviews, I had decided to use a 10-question Basic Skills test. It is handwritten and very, very easy: "Write out the numbers 1-10 each on a separate line in a console app", "describe how a web app works in detail as though you are describing it to a brand new tech writer". It is timed - it's really, really easy - and I allot up to 20 mins.
I handed my candidate the test as the first thing and he failed miserable. Next!
My fault for not having a better phone interview process in place.
After a successful interview, both sides know whether the answers to all of "can, will, fit" are yes.
The other mistake that companies make is in the description that they use to find people. If a description says "{framework}" and nothing else, folks are going to assume that {framework} is your problem and that you don't have anything else. If you're developing {framework}, that's reasonable, but if it's just a tool being used correctly, it isn't your problem and you should talk more about what you want folks to do with {framework}.
Beyond that, I don't think I'm the only good developer who is tired of the "gauntlet" mentality in tech interviews. Coding questions are fine and puzzles are sometimes acceptable, but you have to remember that their only purpose is to answer one question: does the candidate know how to code? If you already know the answer to that question, it's time to sell the interviewee on your company -- not to alienate them by beating them senseless with ever-more-difficult problems and brain teasers.
Every interview question should satisfy one of two criteria:
1) if the candidate answers correctly, would it make me want to hire him/her?
2) if the candidate answers incorrectly, would it make me want to avoid him/her?
If your questions are "hard", but they don't satisfy one of these requirements, they're useless. And importantly, once you've got a sense of the answer to these questions, you're done with the interview. There's no need to drag it on, and subjecting a candidate to a grueling, capricious interview process can only serve to alienate her from your company.