I did an interview once at a company that is religious about pair programming. I found the exercise unsettling: would they evaluate technical ability or culture fit more? If they were focusing on the former, I should take the wheel and try to demonstrate my chops (perhaps making me appear arrogant); if the latter, I should try to demonstrate how I can collaborate with the other developer to make both of us better (perhaps at the expense of me not demonstrating as many skills within that brief window of time).
It turns out that they were really focusing on how I interacted with the fake "client" and whether I had the proper "consultant mindset". Didn't see that coming (perhaps that's my fault). Though I feel I ended up dodging a bullet in not getting an offer from them.
My suggestion is to try to give candidates specific times to demonstrate various competencies (technical, colleague interaction, customer-facing), and tell them which ones are the focus of each exercise. Yes, that's not how the real world works (you need to utilize them all at the same time, of course), but an interview setting is hardly the real world. And you'll still see glimpses of their overall capabilities in each stage.
Chiming in to agree with you - where I work there are different coding competencies that are assigned per interviewer (e.g. good understanding of data structures and algorithms, good problem solving when dealing with ambiguity, knows how to write code that's readable/maintainable/extensible for future use cases without overengineering, etc).
I explicitly tell the candidate what I'm looking for at the very start of the technical portion of the interview. I'll say something like "this is a difficult problem and you might not finish, but that's intended since I want to see your problem solving skills", or "this problem is meant to be a little simpler so don't overthink it, I'm focusing on how you structure your code to be readable and easily modifiable if we want to change behavior or add a feature". I suspect that knowing exactly what I'm evaluating, instead of thinking they need to excel at everything, takes a bit of pressure off and allows the candidate to perform better.
That's a really good point. I honestly don't remember if we gave the candidate a heads up in that regard. In general interviewing is extremely imprecise. It's an unsolved problem unfortunately, but I feel most companies "fix" the problem by just over-interviewing and being ok with false negatives.
> would they evaluate technical ability or culture fit more?
Why not ask them that? "What are the roles in this roleplaying game? Am I a driver and you are evaluating me? Are you a client? Are we colleagues?" Or even more directly what you asked: "Are you evaluating technical ability or culture fit more? "
I don't think there would be a point in keeping that a secret.
Yeah, from a candidate perspective that's a good idea. From an interviewer perspective, though, it's important to remember that many very good candidates will feel uncomfortable doing so.
It turns out that they were really focusing on how I interacted with the fake "client" and whether I had the proper "consultant mindset". Didn't see that coming (perhaps that's my fault). Though I feel I ended up dodging a bullet in not getting an offer from them.
My suggestion is to try to give candidates specific times to demonstrate various competencies (technical, colleague interaction, customer-facing), and tell them which ones are the focus of each exercise. Yes, that's not how the real world works (you need to utilize them all at the same time, of course), but an interview setting is hardly the real world. And you'll still see glimpses of their overall capabilities in each stage.