I've done a similar test. I didn't finish it in the time allotted, but a few other candidates did and they hired one of them. The fact companies can do these kinds of demanding frontend tests and still hire people is what makes me think there's a glut of web developers and not a shortage like people keep saying.

I had an onsite recently that told me were were going to be building something that had to interact with a postgres database.

I built out the entire backend prior and took copious notes because getting stuck looking up things like “how do I initialize a postgresdb” (something I do maybe once a year, if ever) would eat up time.

I think I did pretty well at that onsite because of the fact that I prepped before hand and they didn’t seem to mind that.

It’s weird to me because in general most engineers are rarely if ever creating an entire stack from scratch and it can be very easy to want to do such a thing methodically and with best practices in mind.

How would I enable hot code reloading? How do I set up PostGres so I’m not just connecting as super user which is obviously a security risk? Etc etc.

To be able to do these tasks under pressure with an engineers mindset is tough and you either have to really drill down and focus and stress that in a real setting you wouldn’t be connecting as superuser but for the sake of time in this particular toy exercise you will be.

Being able to prep was essential, and even so, I still stumbled here and there with unfamiliar syntax.

Interviewing is not a solved problem.

There's a glut of resumes that have something web dev on them, and a scarcity of reliable ways to filter for the candidates who legitimately have the skillset(s) needed for the job.

I've done a lot of these tests and interview projects, passed plenty, didn't pass a few, didn't bother finishing a few, but - weirdly - every place I've been hired didn't use them in the first place.

I've started to view them as a sign of a company you probably don't want to work for.

> I've done a lot of these tests and interview projects, passed plenty, didn't pass a few, didn't bother finishing a few, but - weirdly - every place I've been hired didn't use them in the first place.

Similar here. Strangely, the places that don't do them pay about the same as the ones who do (in the same market—not talking FAANG versus Bob's Printer Service & PC Repair in Madison, WI).

After my latest search my new policy for future searches (at least until the downturn or it otherwise stops being super easy to find jobs) is at least not to do any kind of project or evaluation before a real interview, that is, without a real person from the company taking the same time I am, at the same time. The kind where they send you to some "coding challenge" website or give you some "take-home" project before even talking to you. If they're asking me to burn a bunch of my time to save some of theirs, it means I'm too far down the slush pile and/or they're too bad at interviewing for it to be worth my time.

We use a third party for the initial code screen because we feel like it’s more useful and more respectful to the candidates. We worked with them to pick questions that were representative of the type of “real” coding someone might do at our company, i.e. not algorithms or brain teasers. The company also includes a section assessing how the candidate communicates about bugs they’ve investigated. The screen allows 75 minutes and can be done any time the candidate wants, without someone sitting over their shoulder. When it’s done, the candidate gets the same feedback that we do about their performance. The feedback we get includes the candidate’s solution, a general “score,” and a series of bullets of things the candidate did and did not do well.

Compare this to what we did previously, which was to get one of our engineers on the phone and on a pair coding website to ask brain teasers. The candidate got no followup feedback if they didn’t do well. On top of that, because the people doing the interviews knew that if they said “no hire” it would stop the process in its tracks and the person would never get an on-site, we had something like a 95% pass rate. We evaluated several third parties for this, and felt the one we chose was the most useful and the most respectful of candidates’ time.

I understand that the process isn’t for everyone (no process is), but we’ve gotten generally good feedback from applicants so far about the company we chose (Woven, for the record).

Edit: also just noting that we don’t do any sort of hard cutoff for the scores they send us. We try to evaluate each candidate’s performance in light of their experience, looking at what they were and weren’t able to get done, etc. We had someone recently do amazingly on the first half but not even finish the second: presumably they ran out of time, but we brought them on anyway because we felt like their performance on the first half deserved earned them some further consideration, even though their overall score was quite low.

I figure if my initial contact & a work history that fits very closely with what the posting company's asking for (as it usually does when I personally apply somewhere, rather than letting recruiters bring me stuff) wasn't interesting enough to get someone on the phone and instead gets me directed to a screener robot, they're just Not That Into Me. If/when (next downturn, probably, when the number of applicants vastly outstrips the available jobs) everyone's doing it I'll just have to live with it, but for now, not worth an hour plus for a lottery ticket to maybe talk to someone. Others are excited to talk to me on seeing what I've done and what I might do for them. No need to buy lottery tickets with hours of my time.

Plus, I mean, I'm not as good at coding challenges as I am at actual development work (which is a lot of stuff, some of which is tappy-tappying code into an editor) so if that's the first gate I have to pass before anything else, that lottery ticket I'm buying with my hour typing at a robot ain't likely to pay out anyway, even if I'm actually the best candidate for the job itself, which isn't guaranteed. Luckily there are so many fish in the sea (job offers paying roughly the same) that it's not an issue for now. Hopefully I've fully "leveled up" into titles for which coding challenges aren't the norm in hiring before the next downturn, otherwise I guess I'll be spending some time drilling to preen those peacock feathers, which is what I'd have had to do this time if it weren't such a seller's market for devs.

That all sounds very sensible to me.

You might have seen the same thing, but any title that sounds managerial in some way - even if in the day to day work, it's completely meaningless - goes a long way toward jumping the line and convincing non-technical people involved in the process that you're a good bet.

Some indication that you have the soft skills and social skills to get along well with non-technical staff helps short circuit the whole thing, as well as convince the type of technical staff that put too much emphasis on coding challenge questions, etc., that they need to cool it and consider the whole candidate, not just a score on a quiz.

Yeah, my curse and blessing is that I'm (apparently—I'm just going off feedback and where I've observed myself contributing strongly to projects) really good at parts of software development that don't really come through in coding challenges, even most of the ones that try to be like "real work", and that are also kinda hard to talk about directly without seeming like a bit of a prick. Good taste, good intuition, broad knowledge, a tendency to think in and about whole systems, and (frankly) sufficient general intelligence that connections among all those things just pop into my head. I also seem to be above-average at explaining concepts to others, at writing generally, and other developers on my teams are usually really happy with library interfaces, APIs, and similar things I design & build (see again: good taste).

Meanwhile, during the act of writing code, I crib off existing code to remember how the hell method invocation looks in the current language I'm writing and similar important details. If I haven't touched a language in 3 months I very likely can't FizzBuzz without syntax errors or using the wrong function somewhere or whatever. I put Go on my résumé because I have written quite a bit of it, and can talk to someone fluently about the parts that are likely to seem odd to a newcomer and some of its strengths and weaknesses, but without some cramming ahead of time or a cheatsheet I probably can't demonstrate a bit of that in code. I'll likely mess up keyword order and all kinds of basic things just trying to "hello world".

In short: I've realized, later than I probably should have, I need to get the fuck out of development per se and into something that's still basically developing software since I'm actually pretty good at that, but doesn't judge me on whether I can, in the moment, recall what a print statement looks like in a language I was writing literally yesterday (it's entirely possible I'll blank on stuff like that!). In my defense I've been easily landing jobs doing this stuff since I was 15 so my blinders were, I feel, well justified—it'd just not occurred to me previously that I might shine better, at least in interviews, which is kinda important, by seeking roles a bit adjacent to development rather than in the thick of it, until I got mumble mumble years in the field and started to think about what I'll be doing when I'm, you know, not even sort of young anymore.

[EDIT] and yeah, of course I could put together Anki decks and do daily drills to get better at the parts I'm bad at and it'd certainly help, but another part of this series of mid-career revelations I've had is that if you find yourself wondering why other people are having trouble with something that's easy to you, that's what you should put your effort into, and conversely, if you can find some way to ignore the parts you find harder than most people seem to (not always possible and sometimes you just have to work on things you suck at, sure, talking about ideally) then you should do so, to free up more time for the easy-to-you-but-not-others stuff.

I think the shortage is _good_ web developers, and it sounds like this test doesn't really help identify those.