Something I worry about is that if SD became normal, then people would never get the experience of thousands of hours of driving a car in countless situations that is needed to develop good judgement, much less quick reflexes. And so when a rare situation arises when they need to take over, they won't be able to do it well.

There's a really great YouTube video called "Children of the magenta" that's part of a lecture the chief training pilot for AA gave about 20 years ago or so as part of their continuing education. He goes over incidents and situations and the essential conclusion is that pilots are getting too used to turning dials and flipping switches when in many situations they need to just take control and fly the plane.

Airline pilots already face this -- it's hard to stay engaged in flying when the plane can fly itself. By the time something bad happens and the plane gives up and hands control back to the pilot, the pilot lacks the full situational awareness he would have had if he was flying the whole time.

With enough reliable bandwidth a decent workaround is to have the system fall back to human control by someone other than the local driver. I'm imagining a Car Traffic Control Center where your onboard robot driver sees a situation it doesn't understand and throws control to a remote driver wearing a VR rig with your car's video feeds as input. The remote human driver assesses the situation, steers you carefully past the weird obstacle/issue then returns control to your robot.

A system where robots drive automatically, say, 95% of the time while human remote drivers handle special cases 5% of the time still seems like a big improvement over the status quo - there's a market for that.

Throwing a remote driver into a dangerous situation with no context sounds like a terrible solution to me. See also: Cpt Dubois from AF447. Doing that deliberately and repeatedly just multiplies the chances of catastrophic error.

And the VR driver job would be so stressful that there may not be many takers. Who would take responsibility if they made a bad call and caused a crash?

But there has to be an assumption that this is a rare event and that it takes place in a context where a VR driver has time to establish some situational awareness. (Oh, and in a lot of situations, there is no "just pull over option." I've gotten into some bad weather situations but there are often only sub-optimal options at that point. Pulling over can also be dangerous or may not even be an option.

We might be thinking of different situations. I'm mostly imagining a car or truck that does great on the freeway but poorly on surface streets or poorly on particular KINDS of surface streets or even particular KINDS of weather...and we KNOW this and can recognize those situations. The remote driver typically jumps in BEFORE the part that is actually dangerous.

This isn't a new problem - consider a big ship that delegates harbor navigation decisions to a harbormaster and/or tugboat, or a big plane that delegates final runway approach decisions or parking at the gate decisions to a control tower and/or local guy driving a tow vehicle or waving directional flags. You could slice the world up into "regions we can reliably navigate without help" versus "regions where we still need a little help", with the latter group shrinking over time as technology advances and maps get better and edge cases are better handled.

The initial product offering might be for long-haul truckers - the truck drives itself for hours on separated freeways and then throws to a handler when it needs to navigate unfamiliar local surface streets for a delivery. But once you've GOT that sort of infrastructure - basically a map with geofenced areas where remote drivers step in - it's a logical next step to make the help areas dynamic and mark slowdowns or detours due to an accident or a landslide or a cow on the road for similar handling.

95% of that job would not be stressful. I'd be more worried about it being boring...but then, so is normal in-person driving.

I'm ready to guess what it looks like: Car on autopilot going through a residential neighborhood, playground ball bounces out into the street from between two parked cars, car does not brake in anticipation of a 6-year-old that is not currently visible.

When you're going 45 in a curve driving along PCH, and a sudden fog bank obscures your cameras and LIDAR and the computer says "your controls, good luck!" you have maybe 2 seconds to react, if you're lucky. It might be a lot less.

Humans make really dumb decisions sometimes, but we are also outstandingly capable of reacting to novelty.

Whether it's an illusion of safety, a letdown of attention, the general idea is that humans should never trust that things will go well when there's real probability that they don't. I think it's not in the amount of automation, as you explained well, but rather in focusing users on the critical parts that they should watch out for — and there clearly automating helps us remove the unimportant from the equation, and also make us more responsive, more accurate for the important parts. But it's a lot of great UX, and that's one field where e.g. the military is usually great but commercial companies are abysmal if they can get away with it (read: sell enough to justify not spending a dime on more quality). That's worrying when security is involved, but it hasn't proven a moral or ethical problem for most industries absent of regulation (forced ethics, ha!), so... I think there's valid concern by OP.

As for passenger jets, the Airbus A320 (late 1980s) was the first commercial plane to have a "full" autopilot; all systems were electrical¹ (manoeuvering, thrust control, etc) which allowed the computer to integrate and manage it all. :)

It was tested a number of times by pilots for fun, from taking off to landing entirely on autopilot — ofc they're standing right there ready to take over if anything goes wrong but I've seen it first hand many times. We're talking commercial flights with passengers, it's 100% safe and actually quite "smooth" because the computer is so accurate.

Honestly, the problem is much, much, much easier for planes: a good GPS and it becomes quite the closed problem, and obviously 100% of autopiloted planes are simultaneously piloted by real humans... ready to take over. Yet a plane could technically land itself just fine if pilots were incapacitated, it really could. I suspect it did more than we know for many reasons. And when flying by instrument (means you see s__t), an autopilot is basically just a computer doing what a human would do slower by reading the same data (and maybe cross-checking with physical/manual instruments, but an autopilot doing the grunt work of stick-holding gives you more time to double, triple-check everything incidentally).

_____

[1]: Note that all systems are also doubled (even tripled) with mechanical (hydraulic etc) failovers, because obviously you can lose electricity in catastrophic situations, hence why it always seemed crazy to me that a planed requires software to fly properly instead of plain old good physics and mechanics).

I think we'll see 100% automation for freeways within a short time. I think the only way we'll see 100% automation for arbitrary point a to point b in 50 years, that a standard human could "safely" do, is if we get flying transport.