Lawyers who lose cases, physicians who couldn't save their patients, and pilots who have been in plane crashes still get to work in their occupation after the fact. Experience rules over all.

In your examples all those people never got to practice their craft. Lawyers can't practice law without a bar, physicians can't perform surgery without a medical license, etc. But software development isn't like that. What if there was a surgeon who has completed 10,000 successful surgeries but never went through med school? What if there was a lawyer who won major cases with no law degree? Or a commercial pilot who successfully flew for years on a fraudulent license and only stopped flying because he got caught? (That last one actually happened a few months ago).

If you could choose, would't you pick the surgeon with a wildly successful track record over the recently graduated med school student who has never done a surgery outside of residency?

> pilots who have been in plane crashes still get to work in their occupation after the fact.

Most of them are too dead to continue to work, and the rest usually get fired if the crash was due to pilot error.

And no, I don't believe you're going to fly with a pilot who flunked out of flight school.

Notice that I said “pilots who have been in plane crashes” and not “pilots who caused plane crashes.” This goes along with the theme of the lawyer losing the case, the surgeon losing the patient, etc. Sometimes you can do everything right and still everything goes wrong. Which is to say that the experience gained from these events makes these people better practitioners; it doesn’t matter what school they went to or what test they took if they gained real world experience practicing their craft.

No, I wouldn’t fly with a pilot who flunked out of flight school, but I would certainly fly with a pilot who’s flown for 10 years without a license over a pilot with a brand new PPL and <2000 hours flight experience.

Maybe for better perspective we can try this exercise: let’s say there’s a world renowned brain surgeon in, say, Norway who has worked for decades and performed tens of thousands of surgeries. No mistakes! Tens of thousands of happy stories!

But wait. Our intrepid surgeon can’t practice in the US because she didn’t go to medical school here! If she wanted to operate on a patient in the US mainland, she’d lose to a doctor who just exited residency!

If you were the patient who needed brain surgery, who would you pick? The world renowned expert or the guy who just finished training? Your logic dictates that the guy with the credentials wins out for no reason other than they went through whatever process to obtain those credentials, regardless if there are better metrics or methods to judge occupational success by.

> Our intrepid surgeon can’t practice in the US because she didn’t go to medical school here!

That's not my position. My position is if he was that good, he would pass the medical examinations.

> pilot

I'm not a pilot, but my dad was, other family members are, and I worked at Boeing designing the 757. Actually flying an airplane is rather simple. Most of pilot training is about handling an emergency.

These days, emergencies in the air are very rare. You may never have one in a career. But if you do have one, trying to learn how to deal with it on the job is a good way to die.

If someone really did fly 10,000 hours, but still cannot pass the certification test, he's a fool and you're a fool to fly with him.

Also, please do not confuse "did not take the test" with "flunked the test".

P.S. I have some training materials my dad had to learn. You're not going to learn that stuff by just flying around. Much of it is things that are learned about the airplane by expert test pilots, like how fast can you fly it without tearing the airplane apart. Do you think a pilot ought to know that speed before he gets in the cockpit? How about questions like how much runway do you need with a specific amount of airplane gross weight and altitude of the runway? I remember getting on a plane in Colorado and the pilot threw our luggage out, saying it was too much weight for the altitude, and it would arrive with the next flight. Do you want to fly with the pilot who couldn't make such computations? It doan' matter how good at flying he is if the airplane won't lift off the runway.

I don't buy for a minute you'd choose a lawyer who didn't pass the bar, etc.

I would certainly choose a lawyer who didn’t pass the bar provided they have won 10,000 cases in the past. Although I would settle for 1000-2000, etc.

I don't believe such a lawyer couldn't pass the bar.

Take all 8 billion human shapes, four right angle corners and straight sides is the test, all sides the same length is expertise. You're saying that all squares are definitely rectangles, which I agree with. I'm saying all rectangles aren't necessarily squares. Then it feels like you're saying there aren't many more rectangles than squares, citing that there are unquestionably squares. Unstated I'm suggesting there are a lot more rectangles than squares, by Sturgeons Law 95% of tests are crud, so there are far more people who passed a test than there are people who passed a test and have expertise. There are so many people in IT with coworkers who passed MCSE or A+ tests but cannot do what the test said they can do that it has become a meme, for one example.

That 5% of tests are rigorous and accurately identify squares is not really something I question, but I might question that the people who pass them were "taught to take the test" and nothing else.

(Would you say 7 years of degree, medical school, hospital residency, rotation around departments, counts as "teaching to the test", i.e. "an unhealthy focus on excessive repetition of simple, isolated skills ("drill and kill") [which] limits the teacher's ability to foster a holistic understanding of the subject matter."? - from https://en.wikipedia.org/wiki/Teaching_to_the_test )

If you cannot answer questions about a subject then you haven't mastered it, holistic or not.

Again you're repeating the assertion "if you have expertise you can answer questions", "squares are rectangles", which isn't the point of contention.

It's the other way which is in question, "if you can answer questions, does that mean you definitely have expertise?". Is being able to answer questions the same as having expertise? Are there people who CAN answer questions but have no expertise? If so, how many?

Referencing your other pilot comment, is being able to say what action you should take in an emergency, the same thing as having the calmness and presence of mind and judgement to actually take that action in an emergency instead of locking up and forgetting and panicking? Certainly the pilot who does act well in an emergency could tell you about it, but can all the people who can tell you about it, do it?

Please read carefully. I never said passing a test is proof of mastery. I said failing the test means no mastery.

Putting it in terms of logic, (A implies B) does not mean that ((not A) implies (not B)).

You absolutely did say passing a test is proof of mastery, and you cited lawyers and doctors passing tests as evidence, and you said "there are bad tests, but there are good tests and they work", with emphasis.

I passed German at school, despite being unable to speak German. Anybody who spoke German could have tried to have a conversation with me and see that I couldn't, that would be a good way to judge my supposed skills. Any test which is not that, any test which is a proxy to that, or a way to judge that with less effort, is worse than doing that. Teaching me to answer the questions they were going to ask on German grammar, or recite the sentences they were going to expect me to recite, is even worse.

Of course people with absolutely no skill fail tests more than random, who would question that.

> For example, would you hire a lawyer who flunked the bar?

It's not about hiring a lawyer who flunked the bar. It's more like, how many lawyers are bad lawyers despite passing the bar.

I don't know about law specifically, but in my experience a lot of people who pass tests don't have deep understanding. Vice versa may also be true depending on the test.

It's about the virtual certainty that those who didn't pass the bar are bad lawyers.