Honestly how many times do I need to rehearse these dumbass algos (blah blah blah, so I'll optimize for space with blah blah blah) okay already. I would much rather show you real world code that I've built, or passion projects I spend my free time on. I want to bring me to your company/projects, so get to know what I'm about holistically as an engineer. I think you can best understand that by looking at actual work done and judging whether or not the person is capable of contributing to your needs.

Whenever we face challenges, we learn from them. At scale, we learn everyday. So just hire people who are passionate about facing challenges and learning from them. Not someone who can spend 8 hours a day like a college student playing leet code instead of building something useful. It really isn't that hard to memorize a dozen essential data structures and algorithms. But then what? So cringe.

If all your career experience so far has been at the NSA, yeah, you might want to do a side project before looking for work.

So I can't say what the code specifically does or who it's for, but I can talk about how I worked on a c++ engine wrapped in a Java server that was responsible for coordinating a large group of non-homogeneous assets, as well as various architectural details (what libraries did we use, database, messaging setup, etc). So it's not like I have nothing to talk about in an interview, plus the fact that I can't get too specific adds mystique. It's kinda fun, because in my experience there's an assumption most engineers/engineering managers make about what I've worked on from that first statement, particularly if they know where I work, and it's actually not that. :)

So I can talk about architecting a system, about customer feedback and redesigns, but I can’t talk about work I did that will span another three years.

I think at FAANG it’s usually fine — most people have good enough examples even without their full repertoire. But at mid enterprise or F500 companies I could see this being a bit impactful.

Sorry, to be clear, I mean can you give an example of something you couldn't use as an example, but is now public.

I've worked at GAMMA, and agree, it's very doable to turn my work generic and talk about it in-depth, compare it to alternatives, etc. so that was the main driver in asking for an example.

if you have been through a CS program you have worked on probably at least 10 projects. and then there are your personal projects. if you have any experience you can talk about at least some of these apart from the stuff under NDA.

so, this approach is much better than timed coding interviews and is a much more fair system as it rewards and takes into account experience.

That's fine for a new graduate, but many of us are decades out of college. What we did then has almost zero relevance to jobs we might seek today.

Here's a basic straw man rails controller. There are a few things wrong with it. Apply suggestions. It was nice.

Maybe I started a new compiler or something at my company but didn't have the chops for it and the project flamed out. If I lie and said that all my goals were achieved and all the hard technical challenges were overcome, how can you tell?

Or maybe the project did succeed but someone else came up with the idea and led the efforts. I was there for the technical discussions and grilled the lead on why he made the choices he did, so now I can answer your questions and sound like I know what I'm talking about.

Software engineers can make a lot of money, the incentives to game the interview process are high and people attempt it often...

Everyone you talk with, has probably one or two anecdotal stories of such. "Yeah I worked with this CS grad that couldn't even write FizzBuzz" - yet we ignore the hundreds of other that do their work just fine.

And this is fought with setting up ridiculous 8-part technical interviews where you'll have to whiteboard some leetcode questions ("Given a problem, show us a working O(Log N), or preferably O(1) solution. You have 45 minutes") or design questions ("Show us how you would design slack / discord / zoom / etc.") that drags over 6 months.

For someone to be truly incompetent, or even not good enough to meet the company standards, the current system is overkill.

I asked him to explain how to do it in 2. And he was like, “what? It’s easy!” Then another person on the team who over heard it also asked.

Eventually the whole team was there, asking him to prove it, and he spent over an half an hour trying to explain it, and none of us were getting it. The consensus from everyone (except the manager) that it was an unfair question that relied on a a very subtle assumption about the data that wasn’t obvious at all, and that 3 passes was the minimum required Without the that assumption.

Of course the manager said everyone was stupid and it was a good question.

That manager remains my counter example of what a good manager is.

A friend of mine at a FAANG recently dealt with a bad hire. Their guess was that in the 6 months the bad hire was there they cost the company maybe ~5 million between wasted engineer time and delayed release schedules after people kept having to put out their fires. On the other end of companies, I've heard of seniors corrupting the database and its backup and ending an entire startup.

This is not a defense of whiteboard interviews per se, just an observation on why companies desperately want to avoid bad hires.

I've worked with one or two of those, but I've also been on interview loops that rejected candidates who went on to massively successful careers at a different FAANG.

The cost of false positives is relatively easy to quantify, the cost of false negatives significantly harder. Which is higher?

I'm always curious how people interview doctors. As bad as getting a bad programmer might potentially be, getting a bad doctor must be exponentially worse, so you'd think they'd have vetting the unqualified folks down to a science by now.

Mistakes can also be career-ending and potentially result in criminal prosecution.

It's way harder to get hired as a doctor than "do a bit of leetcode on a whiteboard".

I'd imagine software/IT might be only place where people who have delivered a couple of toy sized, half-assed webapps are now experts commenting on 'engineering challenges' and 'industry trends' with profundity.

They go through several years of medical residency under supervision before becoming a full fledged doctor and are burdened by expensive liability insurance and ongoing education requirements. That takes care of the competency beforehand.

It's not material to my point either way. I was only arguing that for some company's being very afraid of bad hires may be sensible and dismissing their concerns as hysteria does not seem obviously correct to me.

I think he was trying to point out that if some of the bad candidates had passed a whiteboard test, it brings in to the questions of the effectiveness of whiteboard tests to filter out bad candidates.

I remember a guy we hired years ago who answered all the tech questions we asked with flying colors. He was lazy and unmotivated and ended up dumping all his work on other people (me). I would have much rather had a motivated person who I had to teach some stuff to rather than him.

Who said that?

Others in the comments have expressed some legal concerns about "discussions" instead of Interviews, but I feel this is a hyperbolized fear based off a laypersons interpretation of US hiring law. The law requires you ask the same set of questions of all candidates, yes, but it reasonably understands that questions on past projects/work of course will never be the same, and these are legal and valid areas to go off-script.

So suppose in your situation you have a candidate and they talk about the project they worked on. To hear about their contribution, you ask (as you would all candidates):

"Tell me about a problem you encountered on this project that was particularly challenging from a technical perspective"

The candidate will tell you and likely their solution, and then you can start with the why's and how's.

"Why did you choose this approach? What are the benefits?"

"How did you arrive at this conclusion?" (you can further restate the question with "explain how you got to the idea that you needed to get here")

Make it a hard rule for yourself that you must be able to build the entire timeline and understand the candidate's role in that timeline. I find that even if it turns out they mostly were receiving orders from a more senior resource but could reasonably explain in their own words now why they ended up doing what they did, this is still a good candidate for me as they used available resources and took away a good lesson. If there is ambiguity, you can even just ask "why do you think your senior colleague had the right idea here?"

The idea of a discussion versus a checklist is you want to learn how a person thinks and how they approach issues. Even as a follow up to more factual check-list questions, "why" is very important as it helps you to understand the current state of the person. Just be honest with yourself that not everyone will have the same background and be honest on whether the "why" is relevant for the position. For example, some trivial linux kernel knowledge I would consider "nice to know", but it's by far not essential unless they're doing specific dev work on that field or claim to know it.

The other big catch you need to train yourself for is accepting that people will do things far differently than you do, even things that you consider wrong, but they work. The important part to focus on isn't about how close their answer is to yours, but to make sure you understand how and why someone reached that conclusion.

Why and how are very safe and legal questions as a follow up to a very straight-forward technical question.

Does it? I have never heard of such a requirement in a law (though I’m not a lawyer). That might be an employer’s decision on policy they use to ensure compliance with the law, but I couldn’t quickly find any law requiring the questions to be uniform across candidates. (It’s also a difficult set of terms to quickly and confidently exclude the possibility that one exists.)

I guess that’s a longer (and hopefully more polite) way of saying “citation, please.”

https://www.eeoc.gov/employers/small-business/3-im-recruitin...

Screen applications consistently. Apply the same standards to everyone applying for the same position.

Effectively, you should have the exact same criterion for all candidates for the same position and avoid "on the fly" questions/too heavily customizing it.

The legal discussion on all of this has basically boiled down to that you must ensure you're going through the same process for each candidate with the expected variances based on their employment/personal history (as it's relevant to the position)

I may overstate the situation slightly with that statement, but the idea is that if you ask about a networking stack for candidate A, you should ask the same probing questions for candidate B also. (Consider maybe you know that candidate B doesn't know this stack and will look "negative" on a review to the hiring managers, but you really like candidate B for other reasons)

Ultimately the idea is less about the specific questions and more that Candidate A and B are both reasonably considered in the same fashion for all of the requirements of the position, and that you aren't adding/omitting elements that may influence the applicability of the candidate in any fashion.

So let me restate: It's not required you ask verbatim the same words for each candidate (this is the lay-person over-read).

But for a given position, it should be established in advance a series of competencies that are necessary for the position and that are to be asked of each candidate. How you go about investigating it may vary from candidate to candidate, but should two interviews be reviewed, it should be expected that both interviews cover the same topics to a reasonable degree. (e.g., it is reasonable that if you ask "have you ever worked with library X" and the candidate flat out says they cannot answer questions on library X, you have reasonably covered that topic with this candidate. If you ask another candidate and they do have experience with library X and you ask some additional questions, you have not violated the spirit or letter of the requirement in this case.)

But, if you wanted to ask the recruiting team to add an interview to get more feedback, this would be a clear no.

Some of the best candidates I’ve been involved with hiring have been compelling for reasons you’d never get to in a very structured way.

For instance, a candidate who was just out of university. They were very shy about technical topics, and their tech depth wasn’t great. They went to a university in Africa that I knew taught people more stuff that was wrong than right, and they were starting at a disadvantage. To get him out of his shell I asked about things he’d done outside of academics. He told me that he had not done much because he was very busy. Busy with what?

Oh, just starting a real estate investing side business with his family where he’d been responsible for their international expansion into the UK. With no finance background, he’d spent his time understanding the financial, legal and tax implications of their expansion and successfully launched in his third year of university.

And I’d done a bit of finance so we talked about concepts there and he was able to walk me through complex topics in simple terms.

My job for that interview was judging bias for action, curiosity, and diving deep into metrics. He couldn’t show that for tech because he went to a horrible university and had been very busy with other extra circulars.

He apologized to me for spending so much time talking about non tech subjects. And here is the problem: people who are bad at interviewing are punished by rigid processes. There’s a lot of data to show that women and immigrants from certain backgrounds are more reserved on average. You have to be a much better interviewer to pull off a structured interview that lets these candidates shine.

Stripe handled this by having an extra interview baked in by default. People who were viewed as no risk hires didn’t have to do it, but the default was an extra interview to cover risks. Because it wasn’t an aberration, it wasn’t giving a candidate preferential treatment.

I honestly think it’s one of the smartest interview loops I’ve seen. I think they are one of the few large companies gambling a bit on the false positive side of things to shave off false negative points. And honestly, I was impressed with the people I interacted with at stripe more than AWS. They’re doing something right.

Making definitive statements about liability is not trivial. I can confirm that at some very large companies legal has banned this style of interview citing discrimination concerns. I'm not a lawyer, so I can't comment on whether they're call is more or less correct but it certainly doesn't seem settled.

Also working for a "large" US company and discussing with peers from others, no such banning exists. There is very clear training and a heavy restriction on who can perform interviews, but this is not the same as outright banning it.

I do a coding task first and a Q&A afterwards — because these are the requirements.

I did many interviews. My experience is: I could skip the Q&A completely. Most candidates could answer the questions just fine after reading the Wikipedia article for 10 minutes. In the interview I can see if they already read it or not. But I don't think it matters.

What matters are the coding skills. The coding task is quite simple and half of the candidates (with master degrees and 'years of industry experience') fail. But these candidates are often good talkers when they are chatting about the benefits of agile methodology and so on.

I totally agree with you that a candidate should just write any program. I believe I know after 15 minutes if they are developers or not.

But I also agree with the idea that we interviews must calm down the interviewee and remove the nervousness. I don't want to see if they stay cool in an exam situation because daily work is not an exam situation. I want to see if they can code when they are relaxed.

If half the candidates with a master's degree in CS are failing your test, that should be a pretty huge red flag to you that your test might have an issue. What question are they failing?

C++ is a particularly badly affected language for some reason. I think there are a lot of people who learned C++ once, a long time ago, then moved to Java as quickly as they could when it came out. But nobody likes to admit that maybe they lost a skill they once had, so they keep putting it down on the CV regardless. And then when asked to write something like hello world, they don't remember how to use std::cout or whatever.

These guys are applying for C++ dev jobs but are completely unable to write a program. Most of these fake programmers have studied electronics (not CS) or have studied in a country where you can buy a degree.

What are you actually inferring from the situation - how well the candidate can write software? How well they tackle stress and anxiety? How much spare-time they have to grind these types of questions? etc.

Probably a mix of them all, but there's a lot of irrelevant noise - if you're just looking for one thing.

As companies ask 'Leetcode easy' questions, there came to be thousands of online blogs, youtube channels etc.. which trained even the n00bs who can't write good code otherwise. If they train very well they can solve 2sum or write binary tree level order traversal without understanding much. Of course not all interviews can use novel exclusive questions, and these have a non-negligible chance of passing.

Now if you are hiring, you would think, "If these n00bs can solve Leetcode easy with practice, the good ones are solving Leetcode medium with same level of practice. So let's raise the level of questions so that we don't end up hiring these rote-learning noobs". And it continues.

I don't think there's an obvious solution to this, if you don't want to lose the statistically good heuristic of problem solving skills, in order to weed out candidates.

If you interview for an R&D position at (e.g.,) a biotech company, in academia, or one of the National Labs, you're usually asked to prepare a 30-45 minute presentation about your past work. New grads usually use their MS/PhD thesis defense slides; other folks often have a conference talk they can expand. Some places also do "chalk talks" where you describe how you'd approach a new problem of your or their choosing.

This presentation is the jumping-off point for the rest of your interviews. If you talked about building a compiler, someone is going to ask for details about the lexer, maybe have you do implement a very simple tokenizer on a white board. Another person will ask how you measured its correctness/performance. Yet another may dig into how you organized the team doing the work, etc.

Maybe, as you propose, someone else helped with parts of the work. This wouldn't necessarily weed that out. Nevertheless, I'd argue that being able to justify the decisions that were made--and cogently explain when/why they might be different--isn't actually BS; it's understanding.

The point is to have a conversation about a project that the candidate understands well and is passionate about rather than asking them a bunch of questions that we already know the answers to.

Before the interview we review the code base and try to understand what it’s doing by the documentation provided in the readme. During the interview we get the candidate to demo the project and any questions that came up in our code review we ask at the stage of execution in the project demo. The interview lasts for 2 hours and we’ve had several rounds of candidates put through this process.

Both we as interviewers and the feedback from candidates has been very positive. The interview ends up being a day-to-day normal experience within the team and this really helps us to gauge the team fit. I think this helps to hire people that compliment and expand our skill set rather than hire people who are basically ourselves.

Contrast it to giving a candidate some slightly broken code in a framework related to the role and then asking them to a)fix it and b)implement a new feature of their choosing and document it.

The advantages of the latter approach:

* The interviewer doesn't need to prep beforehand as they already know both the problem and the codebase. This means they can ask much more interesting questions and don't have to invest significant time in reviewing a project that might be in a field in which they have no experience. This in turn leads to better discussions which in turn leads to better interviews.

* The candidate is given a much tighter problem definition and isn't required to come up with something a)novel, b)not covered by their current employer's NDAs.

* The candidate's time is respected because they can be told how long it should take them up front.

* Each candidate gets a standardised problem and so it's easier to compare between them.

When I give take-home assignments, I'm just looking to quickly confirm that someone has a working home dev environment (i.e. they don't just code inside environments that other people give to them) and that they can understand a small codebase and write clean code and document it. Everything beyond that I can find out by talking to them during the interview.

When I actually used this technique in interviews it was interesting to see how many supposed senior engineers would reply with things like "I'm getting a %JAVA_HOME NOT FOUND error, please can you fix the repo and let me know when it's ready for me to work on?".

The test is intentionally designed to filter out candidates who cannot meet or do not want to meet the technical requirements.

Yeah, no, nobody is or should be giving you their own personal work. It's offensive and probably illegal that you'd even ask.

> The novelty and creativity of your submission

Yeah, no.

"Passion projects" in my "spare time"... maybe once the kids have grown up and flown the nest...

Leetcode is easy... I memorise a bunch of stuff, do the dance and pass the interview. If i'm lucky I get a problem i've not seen before and actually have to use my brain during the interview.

I took a look at leetcode when I was interviewing and decided it was a waste of time for me to learn that dance. I was happy with my chances with the companies that didn’t use it in their interviews. And it worked out fine.

Side projects will help you learn useful skills and it is the suggestion of OP that it should be used by more companies in place of leetcode interviews.

Btw, a lot of companies don’t use leetcode for hiring. In my last job hunting season, I would guess than less than 20% of processes used leetcode. Pretty far from “most” companies.

The leetcode dance is, at least for me at this point, much lower effort than starting a side project on the understanding the code I produce will be reviewed during interviews. It's like Sudoku, once you've done a "few", you get to the point where you're able to solve them quickly.

The nice thing about leetcode it is easy to bound the time to what you can handle. I do one puzzle a week and set a timer for 20 minutes. Then I get the answer and browse the forum. It's basically the equivalent of solving the Sunday crossword for me, and it keeps my algorithm skills sharp. Probably no worse than burning a lunch hour on HN.

True, but would your employer care (if they found out) if you copy/pasted a small portion of code that’s not considered critical IP (like util functions, and an integration syncing records from your backend the Salesforce, or something tangential to the business outside the core product)

May technically be breaking your employment agreement, but I can’t imagine it would be too hard to pluck out a decent amount of code, and re-write portions of it if necessary to “anonymize” it for interviewing purposes.

Or if you happen to be interviewed by someone like me, my approach is simply “if you can’t show me the code, show me the UI and explain how the backend / frontend works, and a discussion ensues.

Who decides what is "critical IP" and what is not?

This is a terrible advice, and no, you should never copy code from a work project without being pre-authorized to do it.

It's not just common sense, but you also might trip up over unexpected legal issues (licences etc) that you may not even be aware of.

What? Yes. I expect I'd be in prison for a few years. Your interview approach selects for people who don't follow their NDAs

Sergey Aleynikov went to prison over copying GPL code.

To add some context, my work is in Financial Services, trading systems and whatnot.

I've found this technique to be extremely successful. It's possible I may have had some false negatives, but I've never had a false positive. Everyone I've recommended for hiring has been successful.

It's possible you're good at spotting good people, and rejecting bad people, but it's also possible that hiring is just easier than you think it is, and most people are capable of doing the jobs you hire for. You have no way to tell if you'd have the same result just hiring people by picking random resumes. Maybe negatives are just rare.

This is the impossible problem with hiring. Every interviewer wants to minimize false positives (they're expensive!), but every interviewee thinks they're a false negative.

I don't think this could be true? If you think you don't belong/deserve the job (imposter)... why would you think you deserved the job (false negative).

The fear of BSers seems to be what holds people back from this approach though, and I can see that it wouldn't work for certain non technical fields.

I've been on the hiring team for every employer I've had for the last 15 years and have hired dozens of people.

I also just generally do not think that most people can even put together a 2-3 hour presentation of their past work that goes over well. Most of the time people can't really talk more than 30 minutes about past projects. That requires a whole different set of skills, which there are certain contexts where I'd value that more highly. But my initial impression is that if we set up our interviews this way, we'd also end up filtering out a lot of people who'd be great, since not picking the perfect project to demo basically dooms the entire interview to be a flop. With multiple interview types, you increase the chance that there's an interview they really shine in.

My experience has been that a lot of candidates can't even fill 45 minutes. I ask "what was the most interesting part?", "what was most technically complex?", "what would you do differently if you did it again?", and they just don't have nontrivial answers.

Yeah, because you can have a career where you're just gluing stuff together to make business apps for, usually, simple business problems. You're looking for craftsmen but you're interviewing plumbers.

Most people look up the local business listings, pick anyone advertised as "plumber", and call to make a service appointment. Or they use a general contractor that already has a list of approved subs. Master plumbers don't have to answer little trick questions about brazing copper or about finding lead pipes in an old building. People somehow trust them to know what their job is, and do it.

Rarely, one might encounter an unreliable plumber. They might not get paid, and any other plumber is usually able to fix their botched jobs without hurting the budget much. Review sites exist to track building-trades business reputations.

But the analogy breaks, because no one trusts software and IT folks to do their jobs competently. The default assumption is that we are all know-nothing hacks who could destroy the company with one keystroke. All our knowledge is assumed to be tightly siloed, and does not transfer between similar technologies. C++ people can't do Rust or Go. Java people can't do C#. Desktop people can't do the cloud. Back-end people can't do UI. CMMI people can't be Agile.

It's madness.

There are a million reasons why this approach doesn't work. Maybe they just don't like talking that much. Maybe they don't like the technology you're discussing. Maybe they just don't care enough to have an opinion on it.

None of the above are good reasons to pass on a candidate.

Not really. If someone can present solid reasoning and argue their point well, they don't have to think like me. Three's more than one way to skin a cat.

> Maybe they don't like the technology you're discussing. Maybe they just don't care enough to have an opinion on it.

You mean the technology they are being hired for? Yeah, hating, not caring about it is probably not a good motivator to go for the job then, wouldn't you agree?

> None of the above are good reasons to pass on a candidate.

Beats the whiteboarding. I was hired like that in my last 3 jobs and I have used the same approach myself with rather consistently good results.

I've learned the hard way to be very reserved when giving opinions about technology X because interviewers sometimes get really defensive about the thing they like about technology X.

Also, putting the emphasis on the interviewer understanding the candidates code instead of the other way around is never going to be popular ;-)

It's also hard to scale it, make it objective, and keep the efficiency high (most developers prefer not spending their time on either side of the interview table)

There is another advantage to the "algos". If you can learn algorithms then perhaps you can learn other things as well.

Every new job I've had involved quite a lot of learning in a very short period of time.

Learning whatever language, and whatever standard library is almost always trivial. They're usually not all that different, well documented, some with textbooks even.

Learning to navigate and reason about the huge number of undocumented, often arbitrary, system architecture decisions, design decisions, code layout, etc. etc. that make up real code bases.

That's hard. Really hard.

Show me something you've done that you're proud of. Take me through it in depth. Answer some questions on it.

Doesn't have to be a free-time project. If you don't have a free-time project and you're too NDA'd to discuss previous work, then present some language feature or something.

Even though this advice sounds awesome, I will be cautious of putting it into practice without thinking through the bias problem.

I do remember reading multiple research papers on this, but unable to find them at the moment. From anecdote - In the last company I worked in London, only one team (DevOps) did not follow scripted interviews. It was the least diverse team, not just in terms of representation, but in terms of diversity of thought. Most of it was comprised of "tech-bros".

Scripted interviews do not mean you ask through a basket of questions. It just means that you stay within the guardrails of a set of topics and you go through all the topics. With in a topic, you have fair amount of flexibility. For example if you are hiring for a mid level Java programmer your topics may include - Java 8, Testing pyramid, Functional programming, type safety, developer safety(CI/CD/Rollbacks/Code reviews/Pair programming etc), some domain specific knowledge and so on.

Did this result in poorer job performance for the DevOps team, or any other negative business results that were specific to that team? If not, who’s to say which interviewing method was better or worse?

As an interviewer I’ve always felt very constrained by scripted interviews and “approved question lists”. I always struggle to really evaluate a candidate when I’m asking pre-selected questions without knowing why I’m asking those questions.

It did. And even if it had not in this particular case, it will hurt the company in the long run. There is not even a shred of doubt in my mind that diversity (of thought) is the best investment that leads to success.

I have been using scripted interviews, the same method I mentioned in original comment, for more than 5 years now, hiring more than 200 engineers in three continent and I am super happy with my results.

Honestly, having been on plenty of interview panels for a decade now for DevOps and SRE roles, I'm not sure structured interviews, no matter how awesome they are, can solve the recruitment diversity problem. The war was already lost the moment the job description was published, IMO.

And the process I follow is based on having an exhaustive list of questions covering all areas of the role but during the interview if something else comes up, I don’t mind pursuing that and going unscripted. It often helps me add more questions to my list so my list of questions keeps improving.

I’ve been following this process for last couple of years and the structured process has helped me hire some of the best people I had the privilege of working with. Also, I feel more confident that I’m hiring the right candidate. But of course, there could be an element of bias in there.

Side point - I’ve just finished a SEO hiring guide that covers my whole process of hiring an SEO person (both junior and senior roles). Will be publishing that in the next 4-5 days. If anyone is interested in purchasing a copy, my email is in the bio.

FYI, there's plenty of nuance in the research that people like to gloss over. My understanding is that diversity of background / experience improves team performance. But having a diversity of values amongst your team decreases performance.

For example, if you form a diverse team where some people care about profits above all else, and other people care more about doing good in the world, the team will become less effective. Its really hard to use this research when hiring because a lot of values questions (like "who did you vote for?") are somewhere between creepy and illegal to ask.

Source: I used to work with someone who had a PhD in psychometric assessment. People saying "diversity=good" was one of her bug bears. I haven't read the research myself.

Then I would ask what her political leanings were next.

Studies seem to show that diversity initiatives fail mostly because of a lack of comprehensive diversity and inclusivity effort, e.g. companies talking big about it, but not backing it up with action, money, people, etc.

> Research shows more diverse teams deliver better results. You’re asking something that can’t be proven though: is this thing that is occurring better than the thing that didn’t occur. We can’t know the answer to that.

Research is based on the cumulative results of many recorded events, in which we try to garner a pattern of relationship.

A singular event might have two inputs: we either do the thing or we do not. We can observe the results of what we do, but we cannot observe the results of what we do not. With the aforementioned research, we can _estimate_ the results, and, in some cases, we should make decisions with those estimates in mind, but these are two entirely different concepts. Macro and micro.

Seems like a bit of a sacred cow - how much better? Better enough to sacrifice, say, experience, contacts, or unique business knowledge for? How much, exactly?

When I train interviewers my advice is: have a script, but be happy to go off it. If interviewers keep an authentically engaged conversation, the questions will blend into the background.

Scripts helps in several fronts. They help to reduce bias, standardise calibration and practices between teams, and provide a fallback to get the interview on track. However good interviewers should keep a living conversation with candidates, and take interesting cues off script.

On the other hand someone answers in a way that you have decided before hand is just the misguided answers of a particular tribe can be lack of diversity of thought, because you happen to be right about the preconceived ideas of said tribe, but pretty hard not to see that as the result of bias.

If on the other hand someone says some things you don't understand, espouses ideas completely alien - insane!

on edit: clarification

> Having unscripted conversations is one of the best way to be swayed by unconscious bias in interviews.

Yet, lack of diversity of thought is “easy to spot”. Without being swayed by unconscious biases?

Also interview processes are not just interviewer protecting against his/her bias, but also against bias of interviewee which seems to be somehow lost in these discussions.

That’s too bad, because the obvious followup question is whether those papers survived the replication crisis.

I treat the interview as my chance to help the candidate pass the interview. This bent of thought may seem subtle, but makes all the difference in meeting the candidate in their terrain, and seeing the world from their point of view. Consequently, the case studies given explicitly state that if the candidate finds something else that intrigues them, I am happy to take that as a case study instead - gives them something to flaunt and for me to learn about.

Some candidates are shy to open up, or just not comfortable conversing with strangers, or misread the power asymmetry in the interview and get anxious - I spend a fair bit of time just conversing human to human.

For the really uncommunicative candidates, I make a slight of what they built (fake slight) and this gets the conversation going like a star. The good candidates exhibit a great amount of "Builder's Pride" and defend what they built. They really good ones admit to the possibility that there were other better ways to have built, or explain to me the constraints under which they made the choices they did.

Yes! This is exactly how I treat interviews as well (as the interviewer) and would make a good addition to this already solid article. It really opens candidates up when you treat them like you're in an actual scenario at work. Let them ask for help, google for ideas, ask about different solutions, and ask them what they think of your own solutions. How they deal with these things are the real markers of what they'll be like to work with, not how quickly they can remember how to reverse an array while you sit there staring at them.

Does anyone just clam up because you are making slights about something where you don't know the constraints and make inferences about you from that? Most of us have dealt with utterly terrible, opinionated, under-skilled, supercilious and rude interviewers. How do you avoid coming off at least a little bit like that? Generally speaking there's zero point picking fights with an interviewer no matter how wrong they are.

Hopefully, the first part where the human<>human chat happens, the candidate is able to see that the intent is to have a genuine conversation, albeit with a more senior person (aka older) person on the other side.

Where I differ from you is the zero-payoff framing. There have been cases where the candidate put me in my place, rightfully so, and ended up getting hired.

This is subjective territory, but I actually prefer a candidate that spars. It gives me a chance to explain my position as well as hear his/hers.

So yeah I think that's pretty productive and positive as these conversations go. I don't see you adding anything of utility or value to it by suggesting bad faith or lack of generosity that I don't think exists on either side. "Surgical and precise" slights by definition can't be either from a position of ignorance, about the person you're slighting or the context and constraints of the work they've done. But you might get lucky. Even without luck it might not matter, maybe nobody else in the world would tell you to boil your head other than me? And nobody whose opinion or work you'd care about? YMMV.

Recently I have been looking for another team internally to my company. An interesting fit is that I went through 3 interviews. I'm an engineering manager. Two interviews were focused on system design, one was coding. The only non coding question I received was around how I coach people. The three persons who interviewed me I asked: "what does the team need to do better", and they all answered a variation of "it takes a while to get stuff to prod once it's built. We need someone who can help get better at that". Yet not a single question for that. I guess the lesson learnt is that if you are looking for a particular skilk, maybe focus on that as well.

If they knew enough about the problem and its solutions for it to filter down to people doing interviews, they wouldn't need to hire someone to teach them how to do it, right?

- Can you tell me about a time when you had an issue delivering things to production quickly? What did you do to optimize? How did you know there was an issue? What alternatives did you consider? - Can you tell me about what you have done in the past to optimize they lead time to production? What strategies did you employ? How did you

While you won't be able to tell if what they did was optimal, that can at least tell you if the person in front of you has some concrete experience in the topic, and some depth in their understanding. While that's not perfect, it's certainly better than flying blind

Paradoxically, I think I interview a lot better. I can steer conversation towards stuff I care about, and if they insist on being annoying, just thank them for their time and leave. Though this might just be a result of being pickier about who I interview with.

If nothing else, it's _amazing_ for negotiating. "honestly I'm really happy where I am, but every man has his price, what can you offer?" does wonders.

I experience this too as I not just as I progressed in my career, but even within one batch of interviewing. I've always tried to batch as many interviews as I can. By the third interview I am feeling much less anxious and just perform much better and by the fourth or fifth I am nailing them to the wall - performance seems to be inversely proportional to how much I am worried about doing well in this particular interview.

When I'm interviewing people, I work very hard to put them at their ease and discount interview anxiety when I see it. Too many interview processes are tuned for confidence, not actual skill.

No, conscientiousness doesn't correlate that much with neuroticism. You can care about the results without getting cripplingly anxious, the anxiety isn't a good thing.

If you have evidence that software developers and sysadmins are no more anxious that the average person, I'd be interested to see it. But my experience is definitely different.

And their strengths are what I care about. Everybody has weaknesses, so finding them in an interview is uninteresting to me unless it's something truly fatal. I want to see them shine, as a) that gives me an idea of their capacity for growth, and b) that's what I'm going to play to when pointing them at work.

Many people with social anxiety are excellent writers and I make sure we always have a written portion of our evaluation to give them.

Is the written portion the way to offset this in your experience? In addition to being hard to squeeze into typical 45 minute interview chunks, I’m not sure that would calm my anxiety personally. But I’m certainly willing to try.

I’ve also had people who did great in the verbal portion completely make a fool of themselves in the written portion. They have the confidence and social ability, but they often show they’re missing the skills. They’d probably make better sales people than engineers.

I haven’t yet had someone who totally bombed the verbal do a good job in the written portion. All of the ones I’ve had were just overall poor communicators. If you can’t communicate an idea verbally or written, it’s going to be tough to work with a team of people who like to self-organize.

I don’t put the written portion in any time-block, if that’s what you’re saying. I normally give it via email and give people a week to get back to me.

I find it really insightful to give people a simple exercise to complete and then ask them to talk about it. The answers to the questions are almost more telling than the code itself. You can easily see who is struggling to understand the concepts, versus someone who just didn't have much time to complete it, just by reading their responses.

Even for the same exact code, someone might answer "I completed all of the requirements", and another person might answer "I wrote this in a hurry and it doesn't meet requirement [x] all of the time, but to handle circumstances like [y], I'd implement [z]". The latter person is always a better engineer.

> for the same exact code

And I’m in no position to expect candidates to treat a take-home code test like a real job. These are all experienced engineers that already have good jobs. They don’t need my job.

I 1000% prefer someone who didn’t spend much time on my code test but knows what they’re talking about, to someone who spent a bunch of time and is a bad engineer. The time constraint will be resolved when they quit their existing full time job.

This strategy served me well on many fronts: confidence in my skills and my options, familiarity with evolving interview trends, and networking opportunities with team leaders in a tight knit industry. It also allowed me to chase roles/positions which I wasn't immediately qualified for without feeling stress and anxiety, and was immensely helpful later on as an engineer turned startup CEO to know what product/project management interviews should feel like.

It really all depends on how you frame it. I probably used terminology like “taking meetings with” instead of “interviewing at”. Just make sure it’s not with competitive companies and maybe more importantly not with sister companies under shared VC funds.

Honesty goes a long way, and if your employer is going to fire you because you are keeping tabs on your market value then you need a new employer anyway.

You can keep this shape up to some extent by doing a lot of interviewing of candidates at your current job. It really helps you understand what interviewers are looking for, what they expect, how a good interview should feel and flow.

But there are just some things you don't practice until your own interviews. So you'll want to have your answers to questions, and stories, and narratives all planned out. And make sure you practice them out loud, multiple times, until it's flowing naturally and easily.

And I just accept ahead of time that I'll fail 60% of the interviews that I apply for that I'd be perfect for. Sometimes things click and sometimes things don't. So make sure you've got warm leads and schedule your first rounds at an appropriate number of companies.

Hard pass.

Just like I wouldn’t want to be written off for a place for being nervous I wouldn’t write off a place for doing the standard tech interview day-of-45-min-white boarded-questions.

Younger me would have been very surprised how much i actually enjoy these conversations with manager types. In my experience, _most_ of the VP GM and CEO types are much broader and more interesting than most engineers tend to believe, at least younger me.

This is definitely surprising to younger ICs in the industry - who seem to want to become engineering managers any way they can.

The ceiling of genius you can possibly spike to as an engineer is far higher. I’ve seen single engineers at smaller startups perform the work of entire teams at big companies. And these folks get paid maybe 3x-4x the standard engineer salary. Huge savings. But hard to hire these folks.

I like this approach. Stops you from being painted into a corner, and if you are, you can still leave with your dignity. Some interviewers can be on such a power trip, which can make things feel pretty horrible for the interviewee.

Talk about wasting time. You are bothering to do the interview because for one reason or another you're interested in the job.

It's weird to feel good leaving the interview where you somehow saved face for yourself by not answering any of the questions. You just guaranteed that you neither get the job nor learn anything useful for your next set of interviews.

Also, the lost earnings argument doesn't matter much if it comes at the cost of your mental health.

A bit unrelated, but there's an old joke my boss once told me, highlighting how poor interview questions/criteria can be at assessing one's ability for a particular job:

A guy goes for an interview. The interviewer tells him "forget everything you learned in your degree, it won't be useful here". The guy says: "actually, I don't have a degree". The interviewer responds: "in that case, you're not qualified enough for this job"!

Two magical things happened - during this interview, I realized that I missed a huge opportunity at my previous job (in that case, the step I fell short of was escalating to the CEO) but more importantly, I got the job.

I learned later that the company put a huge emphasis on their employees, especially managers, to be self reflective and not shy away from self examination or probing by others. The point of this interview wasn't whether I could come up with an answers but whether I had the stomach for looking critically at my own failure.

So, I got the job. It paid a lot because the company had to pay for people who could pass these kinds of tests. And it was great to work with people who knew that "their shit stinks too" because they had all passed this kind of self reflection bar.

If I walked out of the interview because the questions felt uncomfortable, I would have missed out on all that.

I can only imagine how difficult that company finds it to hire though, as a lot of people I encounter in professional settings don't seem to have this ability when applied to ther own actions.

Wasting their time would be to continue the interview process and answer questions you think are pointless for the position at hand. It's something you can do when you already have other offers or your current position is good enough (meaning it is better than what the interviewers can offer).

When you know you won't take the job, it's okay to cut the interview short.

I'm interested in learning how other companies do things, what technologies people are looking for in the industry, what sorts of questions are being asked in interviews, and what kind of problems people feel are in the domain of a particular job title. I'm also interested in compensation trends in the industry.

I can get all that information from a decent interview even if I walk away. Politely, of course! It's a matter of respect for each others' time.

Though I was often asked "So why are you leaving?". Who said I was leaving?

But interviewing every now and then has other benefits:

- it has actually helped me become better at interviewing candidates

- a competing offer helped me get a better salary at a job I liked, so I didn't have to leave

- it kept my interview skills sharp for the next time I decided to interview

It didn't have anything to do with my career though. It was early, I had an okay position and just encountered something really interesting and exciting.

So perhaps the learning is more about how and when you choose to go explore something that excited you rather than other reasons to find a new job.

I've interviewed people who have resched this point. It makes for a chill interview. In some cases, the interviewee is overly "chill" and is bored with the challenges described in the job. I prefer to hire people who are EXCITED about the challenges they will face in the job.

When you have 20+ career, filled with success and failures, from which you can learn how to see and seek BALANCE, the last thing on your mind is excitement. You tend to see the world with realistic and moderate lens. And this is not only good for companies, this is a golden opportunity.

Sadly, more and more I look at the startups as a kindergarten party with serious consequences. Some VC's are playing the game and some lucky kids are taking this as a validation for knowledge and expertise. Selecting teams with "cultural fit" and "excitement" to fulfill a hollow visions of "changing the world".

P.S. I don't feel excitement. I consider myself a craftsman with a passion and deep love for my work. And your way of thinking is positioning you in the group of "thank you for your time" crowd. Automatically.

What excites me are things like enjoying time with my family, the idea of making enough money to retire in my 50s instead of 70s, working on my hobbies, and so on. Not my JIRA tickets, sorry.

I also think that excitement =/= passion. For example, if you look at the top chess player - Magnus Carlsen - he always looks bored talking about chess, and yet that's what he does all the time and is the best at it.

Serious question: Does Magnus have a team of people that he works with? Do they enjoy working with him? Do they think he is boring? Is he the type of person you want to work along side?

Whenever I've observed him seeming bored I've assumed it comes from him being mentally so many steps in front of the interviewer. He's already evaluated all the positions, variations ad nauseum prior to being asked about them. Even if he did have something interesting to say it's unlikely he could put it into words that would resonate with the audience. I assume if he actually didn't have passion for chess he would've retired by now as his record is already good enough to be one of the all time greats.

Take the above with a grain of salt though, as it's probably just me projecting from my own experience as an IC that spends a lot of time working in a fairly specialist/abstract role.

In this case your pipeline is optimized for hiring great actors.

I’m outwardly very dull, but also very willing to have a thorough and deep conversation on just about anything.

It just happens that…well…I happen to sound like Ben Stein from that scene in Ferris Bueller when I talk.

I absolutely agree. I like to work with interesting people, too.

If I might ask, what do you do to excite your candidates about the challenges involved?

Anybody who looks excited about things they they have no stake in is just pretending and is likely a ace imposter.

I'd be very vary of hiring such a person.

I would be very surprised if you say literally this and get results. No self-respecting company or manager is going to invest in talking to you if you describe yourself so overtly mercenary.

Obviously, when you're happy where you are, money is part of the equation to get you to move, but making it seem like the only motivator is super gauche and culture-centric companies (which are the good ones) would hang up on this answer.

So curious - are you actually literally saying this and people aren't hanging up on you?

I tell every recruiter my price right off the bat and get plenty of interview offers. Unconditional devotion is reserved for my wife, who actually deserves it.

I wouldn't respect a company that expects me to stick around at below-market pay for "the culture". Tech companies already appropriate your labor on the cheap and keep the (enormous) difference. No culture makes up for that. They should at least pay market rate. I'm not a dupe.

Mature managers and owners are well aware that hiring is a commercial act and that commercial acts are about money. Signaling that you’re willing to walk from the negotiation is key to getting good compensation. Strategically, you wait until they’ve already invested thousands of dollars in labor costs interviewing you first.

I think lots of people get hung up on thinking of business interactions as personal ones, just because they have been interacting with people. Small businesses may act in more personal ways but most larger companies will generally be pretty faceless and make decisions they think are good without bearing grudges because people treated them like a faceless corporation trying to make good business decisions.

Expect to receive a flying chair. If you get out of it alive, you’ll get a better salary.

And if it is below what I am seeking, I say "Thank you for making me aware of this opportunity. At the compensation range you stated it is a hard pass from me. If you can come back with a realistic number I might be open to a discussion. Good luck in your continuing candidate search."

Or if they ask me what I want, I say: "I'm currently making $230k base. Can you come up with a number higher than that which would convince me to move from a job I am happy with?"

It quickly removes the time wasters and the starry eyed dreamers and the cheap skates, and the people who are serious, then we have a conversation and see where it leads. And I have plenty of conversations every month. Yes, it's effective. It's a business negotiation, and the person on the other side of the conversation either understands that inherently or is trying to convince me that my labour is worth less.

If you're comfortable in your existing job then you are ALWAYS in the commanding seat during negotiations. If they want you then they will need to make an offer good enough. Otherwise you walk straight out and nothing changes.

Also, this isn't something you'd say on the phone. It's what you'd say during the interview, when they can't just hang up on you. You've done the interview, they ask "So, what sort of compensation are you looking for?", then you hit them with that.

If you show a weakness (a job you are actively trying to leave) then you will get lowballed. Showing that you have nothing to lose is a wall they have to scale and puts you in the best position possible for negotiations.

A mercenary working for another mercenary can be a very educational experience, and I have found that it is far easier to work with other mercenaries because they can be focused and aligned quicker than people that need to be inspired.

Honestly, if I was a hiring manager, then I'd try to only hire mercenaries keeping things professional.

This depends on what you are looking for.

I am open to learning other sides here because this is so foreign to me. What are the cases where you don't want to work in a place where people care about the mission and culture and want coworkers who do as well?

What are the cases where you're happy working for the company whose attitude is "we don't care who you are and what you value, as long as you have the basic skills and are willing to take what we pay, welcome aboard?" Do such companies become great places to work and if so how?

I am asking genuinely curious because I've always looked for high culture high mission companies because that's what I am like.

The most obviously mercenary industry is finance. I did that for a few years and then got the fuck out. But I've met plenty of mercenary people in tech, especially in startups and BigCos. For plenty of people programming is a job like any other; they're going to come in, do whatever the boss wants (however little sense it makes), and go home. I would die from that, but a lot of people either find their meaning elsewhere or just don't care much about meaning.

I like that. I'm probably one.

A few years ago, my company finally wound up my team, and brought all their development over to The Old Country (I am in the US, and it was a Japanese company). The jury's out, on whether or not it was a good idea for that company.

For me, I had plenty of investments and savings, that I didn't need to work, if I didn't want to, but I wanted to work. I love working on teams; the more eclectic, the better. I have a fairly unusual confluence of skills, experience and character that I know is quite valuable, and quite rare. I was a manager for a long time ("discussion" interviews were the way I worked). That means that I’m quite aware of the value of my skills. I’m not God’s gift to SWE, but I’m no slouch.

I also have a very big portfolio, to prove what I say. I'm not blowing up my CV with padding and BS. In fact, I had to remove a great deal of stuff, in order to keep it to a couple of pages.

I know that not everyone has a portfolio like mine, but it's what I have. It's dozens and dozens of completed, ship-ready, ultra-high-quality projects that can easily be examined, installed, built, run, and, in some cases, submitted to the App Store. There’s at least a decade of commit history, across these codebases, and a ton of documentation. Anyone can look at it. I make it very easy to find.

Couple that with many, many blog postings, training modules, essays, tutorials, etc., and you have a pretty damn good idea of who I am, what drives me, and what I can bring to the table.

You really can't get more solid than that.

In my experience, this was completely ignored, when I was searching for work. I understand the excuses that many managers use for this, and, in some cases, I can't argue against them, but, in other cases, they really missed the boat. I could have actually made a significant difference to their bottom line; especially a couple of smaller companies.

In the end, I just gave up looking, and went to work on my own. I found some folks doing nonprofit work, that looked like they could use some help, and I've been working with them, for free. I'm also making that "significant difference" that I mentioned earlier.

I have absolutely no intention of looking for work in the corporate rat race anymore. I'm quite disappointed in the zeitgeist of the modern software development industry, and don't want to darken my spirit.

It took me a few years to realize just how bad it was for me, and how much better it is, now. It would be nice to have the extra money that a continued paid career would give, but I've become used to feeling good about my work, and I work a lot.

It's like the scales fell from my eyes.

Where as working at a bank or a retail company no one is there because they are super into banking or selling generic household items. They are there to do good work and leave at the end of the day. They are professionals.

Why just "basic" skills? How does that fit in with the rest?

that depends entirely on how badly they need you

I'm in a similar job and haven't interviewed for years but want to start and not sure how to go about it.

One surprising thing I learned from the initial interviews with my lower priority targets was that my prioritization was wrong, in terms of challenges, interests and compensation.

Also I'd like to work on something related to tackling climate breakdown, so I'll almost always chat to companies in that space. Sadly I need to pay off some debt before I can take a paycut to work on these problems (or risk a startup).

Your compensation formula is completely void of the value someone can bring to the table. Young = less experienced, period. In negotiation, sure, but also in the on-the-job skills/experience/maturity. So of course they make less.

If you're a kid out of college competing with thousands of equally green kids, what would be your negotiating leverage? If you are someone 20 years in the industry with unique and proven experience, you can negotiate because you have something to negotiate with - there isn't another you.

It's important to remember that a lot of execs get measured not by results, but by the resources they control. I've done contract work on projects that I could have built for 10% of the total budget. But nobody cared, because actual results were like the 4th priority, with the top three being 1) make the blowhard in charge look good, 2) make the project look big and important, and 3) provide a dramatic finish with 80-hour-weeks so said blowhard could look like he was managing very hard.

So it's no surprise that plenty of companies want cheap bodies more than they want the value of experience. And that's before we even get to the companies whose whole business model is based on hiring a lot of newbs and renting them out as highly competent bright sparks (coughaccenturecough).

The only difference is that I would drill into specifics in certain areas, but keeping it conversation-style so that it doesn't feel like a pointed question. Usually I found it to be pretty easy to see a persons level of knowledge because they tend to hit a certain depth where they aren't able to keep the conversation flowing, so you have to pull back up into their more familiar territory.

The only drawback with this approach is that I have to be really mindful about potentially being biased. Pointed questions aren't as much fun but they're easier to approach from an objective viewpoint.

I like your phrasing. I'm not in tech and don't interview folks for tech jobs, but do interview plenty of people for technical roles in financial services (for me: insurance industry). There are lots of ways to dig into their technical skills - white-boarding, take-homes, etc. but I am often digging instead for fit.

By fit I don't mean whether a candidate has the skills. I ask questions about interests. What does the person do with their free time? Is there something they are obsessed with? What would their friends tell me they talk about way too much? What topics eat at their brains at all hours regardless of time of day. What do they do to feed their interests?

Have had the most success with candidates who come alive (or more alive) answering those types of questions. They show their drive and passion with their hand motions, facial expressions, tone, and more. It's awesome, but importantly, it IS a discussion. It's no longer Q&A to me.

And if those folks fit my other needs (e.g. tech skills, requirements for job), I hire them.

You can absolutely get a good sense of their tech skills from just chatting and you can also get a decent feel for how they communicate in general which IMO is more important than tech skills once you reach a certain point.

He's so passionate about the subject that he created a Youtube channel. It's aimed at both interviewers who want to do a better job and interviewees who want to influence their chances of success. https://www.youtube.com/playlist?list=PLhCnsRMXhadbiHsTcxMCg...

How can ANYONE confidently proclaim that they're great at interviewing without some way of measuring the people they reject?

But I agree, I interviewed some at Google and we can see the stuff that happened in every other interview. And I was really surprised many times, ultimately I realized that I can't really make good judgements based on an hours worth of data and stopped caring.