They're not necessarily "lying" in the strictest sense. I've seen this problem in very large defense contractors like Lockheed and Raytheon that combine two things: 1) a need to hire a lot of cleared developers very quickly when a new contract is won and 2) internal HR processes that make it nearly impossible to fire someone. The former makes it possible for underqualified or even flat-out incompetent people to get hired in the first place just because they're cleared and live close enough to a SCIF and the latter means they stick. At best, they'll do literally nothing and just silently sit through meetings and complete extremely menial tickets a toddler could do in their sleep. At worst, management insists they need to do real work to justify charging for their labor hours, but then they suck so much energy out of everyone else who has to hold their hand and cover their mistakes that their net contribution becomes negative. Nonetheless, it is very easy to transfer them to another team or another project that doesn't know any better and has to take whoever some higher layer of management tells them to take, and much harder to take the time building up the case to fire them, so they stick with the company for a long time. In doing so, they can build quite a long list of bullet points in terms of projects they've worked on, technologies they've used, and what not, even though they never actually learned anything useful from any of them and could not explain what any of the software they worked on actually did or how.

This tends to be especially bad in companies that use a matrix organization where your department or career manager is in a totally separate hierarchy from the project-level managers you actually receive taskings from because the latter know your performance traits but the former decide where you get assigned to work and whether you can be fired or not, and they tend to have a huge incentive to staff critical programs, even if "staff" means fill in with net-negative warm bodies, because that's at least faster than filling with new hires, which also may not even be possible when you have very specific location and security requirements.

Sometimes I flip on obvious intreview questions while solving same problems during the flow of programming almost instinctively.

I hate the way cheaters spoil the well for people with legitimate issues. I always want to be considerate of people with anxiety, processing issues, etc. but I have also seen multiple candidates where everyone thought they were using someone else’s resume (e.g. 10+ years of experience, couldn’t describe _anything_ they did) and were trying to pass it off as ESL issues (our interview panel included a fluent speaker of that language who did not find this credible), and I wish they appreciated that they are making it harder for everyone whose resume looks like the one they used

> I don't think very many players did all the Starfighter trading challenges. patio11 or tptacek might be able to confirm, but I vaguely recall being one of like 100 or so to do all six levels of out of 10,000 that started

As far as I remember, at the beginning and for quite some time the Stockfighter trading challenges had reliability and performance issues (server capacity). So I decided to wait a little bit for the product to become more stable, and then get into the challenges. Unluckily, at that time Stockfighter was already decided to become shut down.

Invisible, forgotten third partner here... more made it through all of them than you might think. They even set up their own Slack community to help each other out.

Apologies! Not invisible or forgotten!!

Do you remember any rough numbers on started vs finished on the two CTF tracks y'all put up?

FYI, the six stockfighter challenges were some of the most fun I ever had hacking around. I enjoy puzzle games like Exit and happily spend money on them. I would probably have paid to play the starfighter challenges even though the embedded one wasn't particularly my cup of tea. I wonder how many other nuts like me might have happily paid a monthly subscription for a few releases per year . . .

I also remember you saying in an early thread that clients' hiring processes were a mess and I think implied most treated Starfighter as just another candidate funnel. Did any of your clients take up a challenge/CTF process after working with Starfighter?

I know Thomas says hiring with challenges/CTFs is a competitive advantage but I'm not so sure. If it was true, wouldn't we see more of it out in the real world?

No, we never really got far enough that we were able to collect that kind of data. I'm glad it was fun; certainly doing it for a living is entertaining. Mostly.

IMO, the messiness of hiring processes and the lack of adoption of work sample testing have a lot to do with each other. One of the questions I regularly ask in interviews (and have for a long time) is, "What are the first 5 things you'll have me work on?" During the last round of interviews, no one I talked to could answer that. I even had a couple of interviewers who didn't even know the name of the hiring manager or the position. Then you have other artificial filters like preferred certifications where the assumption seems to be that whatever training was required to pass can be mapped onto every organisation... or that simply possessing a cert magically grants insight into how most companies do stuff. The point is that companies don't really seem to have a good idea of what they're actually hiring for, but they get by hiring more senior people who have enough experience (hopefully) that they can figure it out on their own.

One of the things we discussed early on was avoiding having Starfighter turn into a certification process and maybe that's one of the reasons why we stumbled around a bit. I am aware that certifications have their place, but they're not really a substitute for industry experience and they definitely aren't a guarantee that a candidate can quickly figure out how a company's tech is glued together. IMO that's the power of work sample testing: you're giving a candidate a challenge where they can demonstrate aptitude. But if you don't know what actually needs to be done, you can't create that challenge, much less a rubric for describing expectations.

The number of cs majors I've had fail a for loop function floors me. The company I was at was paying 105k starting at that time 4 years ago for junior devs fresh out of college, so not FANG tier but plenty good for the area.

At least 30% bombed writing a for loop and if statement though. And we let them pick any language they wanted, so that wasn't it.

I taught an entry-level programming class in a university setting. There were weekly programming practice "labs" and a more involved independent project with several check-ins with me over the semester.

I gave an open-book final exam that asked students to write a for loop on one of the questions. I would say around 20% bombed the question despite the very book sitting beside them having a chapter on looping. With examples.

After the class ended, I ultimately concluded a non-trivial number of students probably didn't do any of their own coding during the course. I'm pretty sure I have worked with at least two or three people in coding positions who couldn't actually code and relied on outside (spouse? out-sourced?) support.

I've only done interviewing for high tier companies, so my bar was pretty high, I wanted candidatws to be able to demonstrate knowledge and ability of a loop with a loop inside.

Our screening was pretty good, so most (but not all) candidates could demonstrate one loop, but an unfortunate number couldn't do the nested loop. shrug