I worked at Starsky Robotics as a perception team intern after graduating high school. I will always be grateful for the team for the opportunity, it was a fantastic first job and everyone who worked there was very kind (especially Stefan).
Unfortunately, Starsky had effectively had no machine learning in 2017 (when I worked there), using solely classical computer vision techniques. This didn't match the company's ambitions of not using LIDAR and there was a strong stigma against switching to a deep learning approach. At the time, very few object detection models had public implementations and I spent a lot of time trying to get a YOLO9000 and RetinaNet implementations running at real-time speeds. Frustrating, as a small startup the labeling services kept screwing us over by returning poorly annotated images.
I think what I took away from the experience is that deep learning in domains with long tails requires a enormous investment in a labeling pipeline - dwarfing the computational aspect - to get decent results. I don't think any solutions are on the horizon that will allow us to bypass this reality. You don't see improvements between Comma.ai and Tesla because it's about the improvements far out on the tail.
For others like me who were as amused by the name, YOLO9000, it's a real thing, "a state-of-the-art, real-time object detection system that can detect over 9000 object categories".
Just to clarify a little bit... "At the time, very few object detection models had public implementations" - this is wrong. Almost all object detection models had public implementations starting from 2014, most notably Detectron (Caffe), GoogleNet/SSD (Tensorflow and matlab). Post 2015 when TensorFlow was released, one can find even more implementations.
Data is the problem. Everyone has the algorithm but not enough people have data (especially labeled ones)
Detectron was open sourced in 2018. R-CNN didn't have any public implementations (there was later a Keras implementation that didn't get the same performance as the paper reported). TensorFlow models added some object detection models a few days after I started my internship, but had various issues at the time. SSD and YOLO both had public implementations, YOLO's being in it's own C based framework.
It's a completely different landscape three years later.
I don’t want to be mean, but since you mentioned RCNN - no, you are dead wrong. RCNN was open sourced in 2014, check the repo: https://github.com/rbgirshick/rcnn
Not to mention that nvidia has thrown numerous open source efforts over the years. If SR was under the impression that 2017 was a dry year for open source deep learning vision systems - I can understand why it didn’t do very well technology wise.
Disclaimer: have been doing deep learning open source and research over the years. Have touched all major frameworks in the market.
It doesn't matter which ones had public implementations. If you are stuck to this comment it is obvious you never tried to productize any implementation of whatever
Thank you for the great writeup. It sounds like you took a responsible, valuable, and even economically viable approach, but the market doesn't want to hear it.
I wonder when the market will start listening to people who actually WORK on the problem.
> which means that no one should be betting a business on safe AI decision makers. The current companies who are will continue to drain momentum over the next two years
Sounds about right.
-----
Here's me over 2 years ago simply quoting people who worked close to the problem:
Here's my negative scenario: self-driving is "AI-complete"; you can't really hit all the edge cases without solving AI in general, which is more than 30 years away (Kurzweil is the wild optimist and predicts 2045).
You CAN use self-driving in limited circumstances, but those limitations are precisely the ones that make driving yourself around more attractive. The expense doesn't go down as quickly as anticipated because of this. They are a niche technology for DECADES.
This looks about right, and I'm not claiming to be prescient, just basically saying what Chris Urmson and Bill Gurley already said years ago. It's weird to me that there's still so much money chasing this pipe dream.
It was a sign when Google spun out Waymo. If they really believed in the product, it would be called something like "Google Self-Driving Cars", not "Waymo".
----
edit: it's easy to be negative, so on the positive (and contrarian) side, I believe rideshare is undervalued, and has a long way to go. I bought both Uber and Lyft stock at $16 yesterday (they may go down again, but ping me in 2 years). This comment in this thread is a nice take and makes me even more optimistic:
I still think that AI shuttles and buses driving predetermined routes and schedules would be tremendously useful. Especially if they could operate continuously 24/7. And that should be possible, with maybe tele-operators dialing in to override the edge cases or sending dispatch crews to takeover for any issues that come up.
The point is that we already have self-driving vehicles. You press a button on your phone and somebody drives you around. Whether it's a premium ride, a shared ride, or a bus is an orthogonal issue to whether the driver is human, which is a matter of cost.
The fact that it's a cost issue makes it look even worse and further out for driverless.
If there is demand for a shuttle along predetermined routes, it would already exist with a driver, and that's what Lyft Shuttle is if I understand correctly. I haven't seen it in SF or heard of anyone using it. It's definitely possible that they could figure out the product and it will become popular, but that has little to do with self-driving.
> The fact that it's a cost issue makes it look even worse and further out for driverless.
I see it the other way, they slot immediately and easily into already existing businesses and if the only difference is cost consumers are likely to swap over.
This is true, but essentially every transit system in the world (minus Hong Kong and Singapore's) operates at a loss. Maybe you can save money vis a vis unionized drivers, but I'm not wildly optimistic.
This is untrue. Tokyo Metro is one of the largest systems by ridership in the world, and operates at a profit. It does have sizable capital investments that take years to pay down.
This also completely ignores the halo effects of these systems that vastly increase the economic output of the region. This is why it is often logical for the state to pay for the system, since the costs are centralized and the profits are distributed.
> Here's my negative scenario: self-driving is "AI-complete"; you can't really hit all the edge cases without solving AI in general, which is more than 30 years away (Kurzweil is the wild optimist and predicts 2045)
If you are correct about that, isn’t it strange that we would give birth to a intelligence and immediately assign it the job of Taxi Driver? Surely it would be an ethical minefield?
This looks about right, and I'm not claiming to be prescient, just basically saying what Chris Urmson and Bill Gurley already said years ago.
I'm curious what specific statements by Chris Urmson you are referring to. When he was in charge at Waymo in 2015, they announced that their solution is good enough to drive a blind man around town. And this was before they acknowledged they're using teleop.
The links to the articles are in the 2018 comment I linked.
The Urmson one is from March 18, 2016 talking about a 5-30 year time frame, and the Gurley one is from April 2017 talking about a 25 year time frame.
Without any more context, "drive a blind man around town" has about 3 decades of wiggle room in it... If you look at Waymo's recent blog posts, they have similar words that sound good but leave wiggle room, and they've been doing that forever.
And reading back over those articles, they hit it right on the head:
Over the next five years, hundreds of companies will claim to have successful self-driving cars, Gurley said. But he said he’s reading between the lines of press releases, which often tout tests in very controlled environments that do not totally reflect the real world.
“The part we haven’t figured out yet, the last 3 percent, which is snow, rain, all the really, really hard stuff — it really is hard,” Gurley said. “They have done all the easy stuff.”
-----
Urmsson was trying to having it both ways, but if you read between the lines it's obvious what he was saying:
Urmson put it this way in his speech. "How quickly can we get this into people's hands? If you read the papers, you see maybe it's three years, maybe it's thirty years. And I am here to tell you that honestly, it's a bit of both."
He went on to say, "this technology is almost certainly going to come out incrementally. We imagine we are going to find places where the weather is good, where the roads are easy to drive — the technology might come there first. And then once we have confidence with that, we will move to more challenging locations."
In an interview, a Google spokesman agreed that Urmson was describing some aspects of the project differently than the company had in the past. "Yes, there was some new stuff in there," the spokesman said. "That was obviously the intention of the speech: To say some new things."
-----
That was OVER FOUR years ago. People didn't get the message and kept pouring money into it. Prediction: if we get economically viable level 4+ self driving in the next 10 years on a $1B+ dollar scale, it will come by way of a breakthrough, not by an extension of existing tech. That is, by throwing out tens of millions of lines of code that's been written and billions of dollars in hardware that's been designed and manufactured, and taking a different approach.
> There are too many problems with the AV industry to detail here [...] The biggest, however, is that supervised machine learning doesn’t live up to the hype [...] It’s widely understood that the hardest part of building AI is how it deals with situations that happen uncommonly, i.e. edge cases. In fact, the better your model, the harder it is to find robust data sets of novel edge cases. Additionally, the better your model, the more accurate the data you need to improve it. Rather than seeing exponential improvements in the quality of AI performance (a la Moore’s Law), we’re instead seeing exponential increases in the cost to improve AI systems
This is exactly the problem with data hungry machine learning approaches, specifically deep learning (and that’s without even mentioning the compute resources necessary to learn). The only way to circumvent that is plausibly apply better inductive biases, and fundamentally rethink what the field considers important.
I think the obvious problem is that induction (which is what learning from data is), is simply only one tool in the huge space that is intelligence, and it will never be enough to emulate the skill of a human driver, which is more or less what is necessary for autonomy in an open environment.
The "induction" that machine learning algorithms do also isn't the same as the induction that humans preform. We induce new concepts from experience (data) -- The description itself assumes consciousness in both "concepts" and "experience".
Thinking of computers as getting more "intelligent" like humans is a category error -- computers are dumb matter configured in an intelligent way by actual intelligence (humans) to preform certain tasks for us. We get better at telling them how to preform certain tasks (software), but there's no reason to think we're moving along some continuum of intelligence towards us.
That's the premise of deep learning - inducing high level concepts from experience, without manual feature engineering.
DL models are good at induction, what they can't handle is generalisation (being accurate outside of distribution). And self driving has a long tail, that's why it's so hard.
> and it will never be enough to emulate the skill of a human driver
It may never be enough to beat the best human drivers, but it only needs to beat most human drivers to be worth it. We're not that far off.
Your objection reminds me of the skeptics of spell check and grammar check. In principle, a perfect spell and grammar checker would also need general AI to fully understand a language and what you're intending to express. Fortunately, imperfect spell and grammar checkers are all that most of us need.
Most of the engineering team has spread throughout the AV industry, with most folks going to our neighbors and fellow YC company Cruise Automation. Some are at Waymo, Zoox, AutoX, and some purposefully exited the AV space entirely.
I joined Applied Intuition to help build out Simulation and Infra for other companies producing AVs/robotics.
There are a few folks I know of who are still looking for their next roles in the BizOps/PplOps side, which has been especially hard during COVID-19 season if anyone wants to do some linkedin stalking.
Hey Dan, could I ask you a few questions about what you learned at Starsky? I have a robotics background and what they tried to do is very very interesting.
Hey Stefan it’s very nice to meet you! I spent many years at GRASP lab at UPenn and most of my friends from there are at Waymo/Nuro/Cruise now. I do not work at a self-driving car company, but I 100% agree with your sentiment regarding the importance of assessing safety, this has obvious implications in how these cars are insured, and therefore priced and financed. On a separate note, I met a long haul trucker/Russian immigrant in a hacker hostel in Mountain View, and his story was eye opening. He said it was the worst job he ever had, and he was “treated like an animal”. So alleviating his problem is very worthwhile in so many ways. As for questions:
1. Referencing page 5 of your VSSA (1), how were you able to quantify "When weather, traffic or other conditions made driving unsafe." It seems like a tricky problem because this region of the ODD is not discrete such as {warehouse, ramp, highway} but rather depends on the quality of your cv stack as well?
2. In hindsight and with the company behind you, do you think it make sense for each car companies to do self reporting? Or should there be some sort of government oversight/technical validation process done by external 3rd party?
3. I would love to chat more! It must be a very emotional time for you, so I would understand if you do not wish to speak about it. But if you do, could we set up a time at lingxiao@seas.upenn.edu?
3) Sure! It isn't too hard to figure out how to email me, so send me a note or DM on twitter or LinkedIn or whatever.
2) AV has been a great testcase for federalism, and I think we've seen different regimes work differently. For the stage that industry is at, I think that insurance requirements is sufficient (insurance carriers have the sophistication to figure out if you're being unsafe). California's regime is overly-prescriptive in such a way that it confuses things more than it improves safety.
To be clear - if we were closer to deployment I'd advocate for a way tighter regulatory regime.
1) We figured out a programmatic way to measure ODD. Trying to specify all versions of ODD is a fools errand, and you're right that it would change a lot by how good you stack is.
ODD = Operational Design Domain. The conditions your system can drive safely in - rain, snow, sunshine, glare, high traffic, no traffic, pedestrians, etc ad infinitum
TBH - In the early days the "cool AV people" seemed to act as if teleop was an admission of bad engineering (your team isn't good enough to perfect L5). If everyone else is right on the cusp, and you're not trying, you must be some sort of loser.
There are still a number of folks who think that teleop can never be safe - surprisingly sophisticated people hold onto that dogma. We wrote a whitepaper about it for investors, and I'll share that at some point as I open up more and more of the Starsky vault.
Those sophisticated people were correct, at least for now (never say never). Teleoperation can work in some limited specific circumstances. But latency, reliability, and coverage of current wireless data networks are insufficient to allow for widespread teleoperation on public roads.
Lidars break. Radars fail. Cameras run into issues. Drive by wire systems malfunction. Computers vibrate to death.
All parts of an autonomous system break. Safety engineering is measuring those breakages, and designing a system that is safe when it inevitably breaks.
If you're the operator, you can choose to only drive on routes that should have sufficient connectivity. If your remote driver is only issuing high level commands (i.e. not responsible for safety) latency starts to stop mattering so much.
The problem here is the availability bias - you've seen your phone fail so you know that telecom links can fail. As a layperson you might not think about how the rest of the system suffers from the same limitations - but they do. You engineer around them.
The situation becomes clearer if you take a look at the traffic today with a human operator in each car. Also in this model, engineering and human failures happen. Engines break, tires get punctured, brakes fail, and humans make errors. We still accept the deaths and injuries caused by car traffic, and don't prohibit cars.
So it is more than sufficient if you can show that AVs - on average - cause less harm than human-operated vehicles.
To your specific question: when the AV loses connection, it would do the same as a human driver when a tire is punctured: turn on warning lights, slow down, and stop at the roadside. Like in other car failure situations, that might cause an accident in some cases. However, that is fine, as long as it is rare enough.
(disclaimer: I'm not working in AV tech, I don't know if current AV technology handles this case as imagined)
That doesn't make the situation any clearer. Coping with mechanical failures isn't the primary concern. The issue is how to handle edge cases where the AV software is unable to decide on any course of action.
Or, you know, all the skills they have from the industry makes it easier for them to get a job in it. Personally, as a lowly peon, I don't really care if my company is fundamentally unprofitable. I care that I get to develop the skills I want to develop.
As recently as February, the founder didn't share the (current) founder's perspective on the AV industry, considering they were trying to secure funding.
Thank you for the write-up! It's a little depressing (but not entirely surprising) to hear that you couldn't get folks excited about safety -- I work in the aerospace field, and our day-to-day is all about risk management: how, why, when. It's frustrating that high-reliability systems aren't seen as exciting when really they're what makes the magic run.
Best of luck, and I'm looking forward to your next venture!
Ironically, in my experience, finance is the field that probably cares most about risk management. That and maybe healthcare (though less than you would think, surprisingly).
Disclaimer up front: I work for General Motors. I don't work on AV. Any opinions are my own. I have no special knowledge of GM's AV strategy.
> It didn’t matter that that jump from “sometimes working” to statistically reliable was 10–1000x more work.
There's 2 states of functionality:
1) It doesn't work
2) It sometimes works
The inverse, for disk drives: Failing and Failed.
Think about apps/services. You could say that your app is working, but over a long enough time period, it is only sometimes working. It's working while you have disk space, free memory, and a working network connection. It's working while your business assumptions hold true. It's working while your datacenter has power. We've developed strategies for managing all of these things; for load balancing and Active/Active hosting. But even with that, it only sometimes works.
---
With all that, I think it may be useful to think of self driving in terms of tasks.
If you can put a box around what you expect a computer to be able to do, you can define tasks that will always return a reasonable output.
The more tasks a computer handles, the easier it is for the human in control (think driver assist, like lane keeping and automatic cruise control).
If you add enough tasks, and perform them well enough, maybe you can take the human in control out of the vehicle.
I think I'm in agreement with the authors that I don't see the day when there isn't a human in control. Or the other way to say that is that if there isn't a human in control, sometimes your AV will just stop.
Just stopping is ok is ok in some use cases, but not others. It will be a long time before robotaxis "just stop" less frequently than uberdrivers, but AV trucks can already be more reliable than contemporary drivers.
I’ve been developing software for self driving (both road and offroad) the past two years and what I’ve noticed is that there isn’t much innovation in the field. It seems like everyone follows the exact same design paradigm which is using something like Autoware. It’s pretty disheartening to see how much money has been poured into this field and what do we have out of it now? GM Supercruise came out in like 2013 [0] with similar performance to the level of autonomy we’re at in 2020. To clarify, this was in a production cadillac in 2013, so anyone who knows about automotive products knows how long this must have been in development before they were allowed to put it in production.
To the author, it’s interesting to read how quick you rose and fell, but it’s good you got out now instead of holding on for years and years.
What do you think is next?
I’m honestly tired of the lack of progress in self driving, and was wondering what anyone’s thoughts were on home robotics. I feel like that’s the next big thing that VC’s will pour billions into.
Thank you for sharing this very candid article on the Starsky Robotics and generally the autonomous vehicle space. It's a real eye opener. I've been following your progress for the last few years (and I also read about your company through Reilly Brennan's "Trucks - FOT" newsletter).
I am sorry you could not get investors to believe more in what you and your team, especially as you required a lot less funds than many other companies in this arena. I also thought you had a clear business case (I worked in ride-hailing and also logistics so understand some of the problems in this space).
I wanted to ask you a question: I am building a startup in the dash cam video analysis space. We are building a large and geographically diverse dataset of road videos, where our users can annotate/label the data. We then are going to look at detecting specific events like accidents and edge cases on videos. Do you feel this type of business, the data we collect, and insights we generate would have value for a AV startup?
All the best in your next move.
Stay strong - you can be proud of what you and your team did.
We had really strict rules on what we wanted our data to look like, and were very specific about subject matter. We probably wouldn't have been able to be a customer.
Our intention to deal with accidents/edge cases was that if anything looked outside of ODD (as in, not perfect driving conditions) we'd execute a MRC (pull to side of the road, or stop at next exit). Relatively easy way to solve most of the really hard edge cases.
> Around November 12 of 2019, our $20m Series B fell apart. We furloughed most of the team on the 15th (probably the worst day of my life), and then started work on selling the company and making sure the team didn’t go without shelter (or visa status, or healthcare for the new and expectant parents).
We were able to land many of the vulnerable jobs by the end of January and I’m in the process of selling the assets of the company (which includes a number of patents essential to operating unmanned vehicles). Like the captain of a sinking ship, I’ve gotten most of the crew on lifeboats and am now noticing the icy waters at my ankles while I start to think about what I do next.
> Instead, the consensus has become that we’re at least 10 years away from self-driving cars.
I'm going to assume the founder of a self-driving truck company knows what he's talking about.
But at the same time, I have a hard time reconciling that with the fact that I sat in a car that drove itself all around San Francisco, dealing with a ton of edge cases.
Maybe we won't get to a 100% drive-anywhere-you-want-car in 10 years, but to be fair, a lot of humans aren't capable of driving a car anywhere either.
There are a lot of LA drivers who can't drive in snow, for example. I was one of them, until I got practice, and even then, I'm not that safe at it.
I think as long as we set the bar at "drive anywhere a car can go with 100% safety" we will never reach that bar.
But if the bar is at "drive as well as a human in most of the places humans drive", well, I've already seen a car that can do that.
What's hard about this comparison is knowing exactly what you saw. If the safety driver disengaged 1-3x, it might have felt like a robotaxi drove you all over; but the effort to get that system safe for unmanned regular service might take another decade.
If it was super mapped and following a fixed route with object avoidance; the difference between a car that can go on an HD-map track and one that can go point-to-point in a city is also maybe a decade.
We humans have limited empathy for what is really really really hard for computers. Many know that's true for an app, but somehow ignore that knowledge when it comes to autonomy.
Such an excellent and cogent discussion of the challenges of making an autonomous vehicle startup work.
I really resonate with the challenge they had making safety "sexy" from a coverage point of view. It seems it shares that attribute with "security" which, when done well, means nothing happens.
I had a lot of hope that we would get four lane (two each way) "freight ways"[1]. These would be paved roads designed specifically for autonomous trucks to move freight from one point to another. With sufficient freight ways between key cities on the coast and further inland, and efficient way to move goods with somewhat worse costs than rail and quite a bit better in terms of reaction time.
[1] Like a free way but designed specifically to exclusively service autonomous trucks carrying standard containers.
This begs the question, how much does it cost to lay road than to lay rail? If it's going to be exclusively autonomous, the more narrow confines of the rail should be well suited.
Another common fallacy. The US actually has a better rail network than Europe - most people just think it's worse because passenger rail here sucks. That's the case because we prioritize freight trains over passenger (in America Business is #1).
To paraphrase Churchill, trucking is the worst form of transportation, besides all the others. If you can put freight on a train, you do. It's slower, point-to-point, and less good a less-than-trainload freight than trucks are.
Many many smart people share this thought. I don't.
Freeway costs roughly $4m/lane/mile. Building a duplicate system for trucking would make UBI look cheap. Truck taxes also pay for much of current hway infrastructure, so you'd need to replace that $
I always thought that autonomous driving was more than 10 years off (if not 20) for cars in city traffic. But I thought trucking in the US had a chance for intersate traffic till the proverbial last mile, because there are less edge cases that you need to train the model. Were you off by just 3 years for interstate “driving” till the last mile?
Will investors lose their loss memory and a whole new set will invest in the space in say 3 years.
I used to live on one of the side streets in MV that Google's AV cars trained on. Saw three in a row once. They were usually the only traffic, so that annual report of millions of miles travelled was meaningless - they might as well have driven arounf the Safeway parking lot at 5 am.
I guess passenger-carrying quadcopters is next. Oh wait ...
Hard to say. Truck drivers will be hit hard by this - they can't really stop interacting with people, almost never get paid time off, are older and in poor health, and are paycheck to paycheck.
My bet is that nearly all of them get COVID, with a higher hospitalization / mortality rate. Could knock 30% of capacity off the road in the next 2 quarters, which would mean food shortages.
This is a very good and honest retrospective. He shows clear thinking, and surprising technical understanding for someone who is not technical. And most importantly he shows humility. A+
There are too many problems with the AV industry to detail here: the professorial pace at which most teams work, the lack of tangible deployment milestones, the open secret that there isn’t a robotaxi business model, etc.
Just curious, why would there not be a business model for robotaxis?
Uber: the drivers own the cars, the drivers maintain the cars, the drivers clean the cars, the drivers store the cars when not in use.
Driverless: the company needs an operation, real estate, and staff comparable to a big auto rental company to do all that. Plus the engineering and technical staff required for autonomy.
Even with startups doing autonomous shuttle buses, which works at low speed, nobody is making money in that space. It's all demos.
> Uber: the drivers own the cars, the drivers maintain the cars
One additional factor seems to be that Uber drivers tend to underestimate the depreciation of their cars, so they effectively subsidize what otherwise would be Uber's capital cost.
Add to that the fact that you are never going to get 100% fleet utilisation, will need to pay for qualified tele-operators, maintain/repair vehicle along with sensors...
I recently listened to a podcast episode of the Autonocast[1], where they interviewed a Harvard Researcher who claimed the economics of Robotaxis just don't work. Very interesting listen.
I think it will probably work at some point in the future( 20-40 years). Today the reality is that the jobs we reward the least are the ones that turn out to be the most difficult to automate. Someone with 10 min training can do better job with a thread and a needle than the most advanced robot painfully trying to stich two pieces of garmet together.If these can be overcome,then it may work.
And the important part of your observation on the thread and needle is that there are people who will do that for about 1 penny currency relative- and about 100 more willing to take the job if the first one quits. Hard for a capital intensive robot and a team of Bay Area salaries to step into that even if good at it.
Was going to ask the same question and scrolled through the comments to happily find yours.
The answers basically seem to explain why there's no robotaxi business model NOW though... I still don't buy the argument that there wouldn't be one with suitable (relatively low marginal cost, eg Tesla system) self driving tech. Yes, uber drivers work for less than min wage, but that's still expensive compared to cost of capital for a (speculative commercial) self drive unit as an add on.
High level - Uber and Lyft have done a terrific job getting people to work for less than minimum wage. I've met quite a number of uberdrivers who work in Robotaxi-Ready cities (think Chandler, Tampa, etc) who work 12-16 hours/day, 6-7 days a week, and make $20k/yr after depreciation.
While sensors are getting cheaper, most teams respond by throwing more sensors on the vehicle. It seems unlikely there will be a <$20k robotaxi anytime soon.
Yes, this is essentially the same reason why you don't see robots flipping burgers. The automated equipment would have to get very cheap to be cost effective.
My current thesis is that robots (have to sense the world, make decisions, move freely, and respond to stimuli) only make sense in markets with labor shortages.
AKA - if people are willing to do it for cheap, it probably won't make sense to get a robot involved.
I haven't heard that thought before, but it's an interesting one.
Probably won't help with any of those folks opposed to UBI because "people get meaning from work." Proponents of that view must know a hidden trove of self-actualized retail workers.
> The biggest, however, is that supervised machine learning doesn’t live up to the hype.
This is the key point. The new DNN approaches can outperform the classical techniques, but only when they can exploit vast amounts of training data. The dramatic successes of Deep Learning all depend on either unsupervised learning against enormous raw datasets (BERT, GPT-2, word2vec, etc) or games, where you can generate unlimited quantities of labelled data by playing the game against your own agent (AlphaGo, AlphaStar, OpenAI Five, etc).
As someone already mentioned here,I think it's probably not a goood approach. The way we store info in dstabases is very limited compated to what we can do in our heads. For instance we know a concept of a table.It can be made of almost any material,can have whatever shape,size, and colour we want and yet we can instantly recognise it without having some concrete data points on what it should look like. I can make a glass cube,put it in a middle of the room and people would know it's a table.How the hell we operate this way,I have no idea.
On a basic level we all:
- Know what a room looks like.
- Expect certain objects to exist at certain places and not expect others
- The brain indexes these
- Seeing one or two datapoints allows us to guess. So seeing a glass cube in that context gives us few logical chpices (it must be a table) similar to someone typing a letter in a textbox and having it filter a pre-defined list.
Thank you for sharing this. The insights and details you share here will help many future founders.
I'm sorry things didn't end up in the exact way your team may have hoped. I hope you can take pride in everything you accomplished. I wish you all the best!
I'm not sure it's intellectually honest to put all the blame on stupid investors, as a founder it's your job to deliver what the market and the investors want, or convince them otherwise. And if in fact investors and the market is just stupid, that's actually a great sign: you were just early and have a shot at doing it again in the future.
Great read! And thanks to the author for all the candor.
The business case always seemed clear to me and now reading this, I wonder if there is a case to be made for an engineer and a trucking operations veteran to build a business that requires minimal capital (or maybe even is bootstrapped?!), to take it across the finish line?
There are a couple of people working on that, notably CloudTrucks.
It's not for the faint of heart - truck drivers and engineers follow a very different set of social norms. It's hard to be the hardass boss drivers don't want to fuck with while being the cuddly CEO engineers like the culture of. That might be a future blogpost.
Not sure about him blaming the "professorly pace" of research though. That's the rate good research is done at. If you were expecting anything faster than you don't really understand what it was going to take in the first place
Thank you for the excellent write-up here. I'm sad to hear that Starsky is dissolving, but I have hope that the lessons learned here will be applicable elsewhere.
The point about safety being a low priority with investors makes sense but is unfortunate. I hoped that Google's investment in Waymo would push past this hurdle, but they're latest funding round makes it clear that they also have to deal with investor wants over the success of the technology. Really hoping to see someone get the funding to make this tech work reliably in the future and apply the same safety-first design that Starsky used.
I’m sure it was explored but it didn’t close. Stuff like this is annoying for a founder to hear because he’s like “aw yeah, why didn’t I think of that earlier?!” Of course he thought of it — but it just didn’t work out that way.
It is pretty cynical and hindsight is always 20/20, but... is self-driving automotive just trains and metros? There is a fundamental business case flaw in these startups, because regulators worldwide will never approve them go together with human traffic, which is messy, chaotic, smart in its own sense at local level (you need to know the uses of local people, not the general rules... eg. driving a car in Naples is very different from driving a car in Milan). Only dedicated lanes or tunnels then, trains!
I wonder what Elon is going to do when _his_ "full self driving" fails to materialize. Which it will. Not only you can't do this with just cameras and radar, I doubt you can do it _period_ without modifying the roads specifically for such cars, and segregating them from human drivers. And even then it will be difficult psychologically and legally to convince the public that this is "better" than a (possibly inebriated) human, for reasons that have been discussed to death already.
> It’s widely understood that the hardest part of building AI is how it deals with situations that happen uncommonly, i.e. edge cases. In fact, the better your model, the harder it is to find robust data sets of novel edge cases.
I have no idea whether Tesla will or won’t succeed. But they do have one major advantage over just about every other AV company out there, which addresses the point above. That is, their huge network of camera-equipped cars (a million and counting) provides probably the deepest, richest AV learning dataset on the planet, and probably by orders of magnitude. If accessing the dataset and thus novel edge cases is one of the major challenges in AV development, Tesla is very well placed.
Tesla makes their money selling cars, and Autopilot sales are non-refundable. They get paid even if they don't deliver, so I'm not sure how much they care other than keeping the hype alive.
Then worst case they'll get a class action lawsuit, appeal the verdict three times and then end up giving everyone $100 worth of credits towards a future Tesla. Lawyers will net a couple hundred million.
I would have thought so before but we've had multiple people killed in auto-pilot related accidents already and people still buy. Tesla will spin the news and results and probably get a settlement statement that makes them look not so bad. Especially if everyone is in the same boat and failing to get the technology working well enough. Then Tesla will simply boast at how much better they are than competitors.
Crisis PR is an interesting thing. Each incremental death from AV causes less repetitional damage. Uber had a big shitstorm when they first killed someone, now they kill someone every day and the response is crickets. This is less of an issue than people like to make of it.
Realize the phrasing may have been confusing - I'm talking about regular Ubers, not self-driving.
My quote was from a person internal to Uber. Below is a TC article citing a public report on US Data. Numbers in that are roughly 50/year, or a fatality in America every week. Every day may have been a stretch, but I'd guess you could deduce that Uber drivers worldwide are involved in a fatality accident at least every day.
*also - this is obviously a lower rate than non-Uber drivers per mile
I have a friend, commuting 6hrs every day (3hrs + 3hrs back) in a Tesla model S, using autopilot (3 days a week). He has been doing it for almost a year now. So far, so good... given the number of miles driven, it is actually pretty good.
No it isn't. As the article implies, the number of miles driven, or even the amount of time driven, isn't a good estimate of AI quality. The best standard is the number of unusual events that the autopilot could handle without human intervention.
I've had 30 hours of driving in a Tesla, and I had to intervene 6 times, twice with no warning. That doesn't mean that Autopilot isn't a useful tool, but it does require operator diligence.
When I said that it is pretty good, I base it solely on the fact that my friend is still alive, even while driving 6hrs per commute mostly on autopilot. I will have to ask him how often he has to intervene (mostly highway). But I do know that he has been watching a lot of Netflix during his commute... so your mileage might vary.
It is like Ethernet vs token ring. Not as safe and especially no guarantee. Some packet or life lost. Ethernet win for simplicity and ease of expansion/installation and surprisingly for management. Not completely safe. No driving is.
30K people get killed in car accidents in the US, and "people still buy". Heart disease kills half a million a year and people still eat twinkies. What else is new? That's called "freedom".
The issue is that if I paid a ton of money for something, I do generally expect to get what I paid for. And in this case that's not gonna happen.
[*] Hypothetically, I'm not presently a Tesla customer.
>Not only you can't do this with just cameras and radar
Humans do it really well with just two cameras. It's not a hardware problem; it is entirely software. Whether self driving is possible or not with current AI techniques is debatable, but we're not waiting on any advances in hardware to do it.
And the problem with current approaches is entirely that while we can "train" a NN to produce specific emergent behavior associated with the training data to a great degree of accuracy, we're totally unable to demonstrate that these systems have perfectly(100% accurate) predictable behaviors.
This is because we're totally unable to come up with a coherent model for why the emergent behavior occurs given the input data and NN training. We know how individual elements of the NN work, and we can describe how the training system works. But the whole notion of repeatedly letting perturbations in a control value or control values dictate the entirety of the performance of a system is nuts.
The only way to determine how a NN will perform in a given situation outside of the training set is to actually feed it the stimulus and check the outputs. Given the amount of stimulus that we as drivers routinely get, it's impossible to say with any degree of certainty that a self-driving car that is built on an NN classifying engine will accurately classify everything in all situations and lighting conditions, because it's impossible to feed it a training set large enough to encompass those situations.
That leaves the question of making a classifier that is better than humans. And whether an NN is better than a human depends very much on the situation. We could make some statistical arguments, but when you're gambling with peoples' lives here it becomes difficult to tolerate such arguments. It's easy to be blase about it until it's your child chasing a ball in front of an AV, at which point any discussion of the statistics is academic.
Some large teams have tried to overcome this by making up every such scenario, coding it into simulation, and testing their system.
What makes this challenging is that there are more such scenarios that might occur than there are grains of sand, or stars in the universe.
Those teams end up thinking they did all the necessary work and then go test, just to find out that there's a new edge case. At Starsky we called it Safety Theatre
I had a mentor at my last job who would repeatedly say "You can't test safety into a product." As in you have to actually do the work to understand and limit how the product behaves, and only then can you call it safe. If you build a product without thinking of safety, then try to test the safety into the finished product, you just end up covering all the known sharp edges with foam. Software products have limitless sharp edges.
As the article points out, machine learning has its limits. It can do some things reasonably well, but it does max out. It's easy to get to 90% accuracy, hard to get to 98%, and 99.9% is out of reach.
It's true, but it ignores the fact that human eyes have orders of magnitude more dynamic range than even very expensive specialized cameras, and it obviously ignores the fact that we haven't invented general AI yet.
It's a pithy response that undermines the challenge of a problem nobody has been able to solve even with years of effort and billions of dollars.
My husband is an optometrist. There are lots of people who are driving with poor vision. We do not need (although it is obviously better) perfect vision to drive a vehicle. We can drive in fog or heavy rain, with limited vision. I am not sure that image quality is a big factor here.
But we do have a good spacial sense (distance) + object recognition (mass/speed) + psychology/habits. I can tell at glance if an object is able/supposed to move, and adjust (i.e. watch/sample it for few more milliseconds) to estimate it current speed. I estimate its mass and deduce inertia (can it stop if need to). I make eye contact with driver, to acknowledge that he saw me, etc. Not saying that those things cannot be learnt and used by a software.
I do agree that we do well with just two cameras, and we make up for it. BTW, from our senses, one that I rarely see mentioned is the hearing (used to label and spatially locate). I do use it while driving (e.g. motorbike approaching, but not seeing it yet, or emergency vehicle).
I agree that human eyes do have benefits over your typical camera, but I don't think the phrase ignores that GAI hasn't been created yet; rather, it concisely points that out and makes it clear how difficult it is to do.
As the OP said, "it's not a hardware problem," in that the quality or number of cameras, sensors, etc. isn't the bottleneck to solving this problem.
>It's true, but it ignores the fact that human eyes have orders of magnitude more dynamic range than even very expensive specialized cameras, and it obviously ignores the fact that we haven't invented general AI yet.
That level of resolution doesn't matter at all for driving. People can drive just as well through a video feed, like Starsky was doing. Yes general AI does not exist yet, but my point is simply that the parent made a comment about the need for hardware which is simply not true.
> That level of resolution doesn't matter at all for driving. People can drive just as well through a video feed, like Starsky was doing.
I can tell you from experience this is false. It usually works, but when it doesn't you're fucked. People wildly underestimate, by orders of magnitude, how many and how complicated the edge-cases are for self-driving.
I wish he would just give up on it and make cheaper cars without any autonomous abilities at all. The world needs cheap electric cars, I don’t want any AV in my car. How much cost does all the AV tilting at windmills add to a Tesla?
If you're going to spend trillions of dollars to dig point to point tunnels for autonomous vehicles, why not only spend hundreds of billions to build subways instead that can carry 100x the number of passengers?
Because, if built like subways are in the US today ($900 million per mile), that would only give you a few hundred miles of subway. So a few city centers might have better public transit, but the vast, vast majority of the US would not. (Which isn't to say we can't do both if tunneling can be done much cheaper. It's pretty silly to think we, a mostly suburban nation unfortunately, only need high density subways and nothing else. Particularly during a global pandemic with social distancing.)
Additionally, with highway-like vehicle spacing and high capacity stretched Model Xes (as the Boring Company proposed), the throughput is the same order of magnitude as a typical subway line. With denser spacing, potentially even higher.
Anyway, the question is about what Elon's backup plan was. This is it, regardless of whether it works or not. (And there's plenty of reason to think it won't, to be fair!)
Why does every self-driving car company focus on driving like we do today? Seems more viable to make a system where the trucks dock at speed on the highway. This is where the power of AI come in to play by coordinating the two trucks next to each other, switching load and continue driving without ever have to stop. The end goal would be to also move people, but we have to start with something safer that makes economical sense.
So? That graph is about the capability of the system and how it improves with effort. This has nothing to do with which part of the system fails the most. Where do you see the contradiction?
The choice of naming here is somewhat unfortunate, and probably contributes to some confusion.
Level 0-5 is terminology used by SAE to describe the range of fully manual (Level 0) to fully autonomous (Level 5). One would expect that in an article about Autonomous Vehicles, L{0,1,2,3,4,5} would describe the level of autonomy that the vehicle provides, but the article instead uses these to describe arbitrary thresholds of a graph demonstrating the capability of an AI.
I think the plateau of productivity will be really awesome computerized copilots and safety assists/warnings. The human computer driving team could Be quite a combo of designed right.
It's well known that if the computer is handling a significant part of the task, the human will get distracted or zone out completely. The "human computer driving team" will end up being less than the sum of its parts. Airline pilots already have well established procedures to prevent this from happening when they engage the auto-pilot.
The problem is that most of those businesses don't make sense with the current cost of talent. The L2 systems OEMs are rolling out are primarily made by engineers in low-cost areas at $60-90k/yr, not $130-250k/yr in SV.
Realistically, most of the AV talent will go to industries that can afford them. Salary isn't super elastic.
Perhaps for true self driving we’ll need one more breakthrough in AI. Perhaps a system that can identify its own edge cases and request additional training or seek out the training itself.
For any engineers reading this who want to solve the computer vision/machine learning problems explained in the blog post - reach out at info@nuronlabs.com.
There was a person in the vehicle. Amusingly - many VCs who "should know better" fell for the same error. I'd regularly get in arguments with investors about whether others had done unmanned or not.
In the sleeper cabin monitoring the vehicle. The main cabin was unoccupied and there wasn't a single intervention for the entire trip. I knew people involved with that trip at the time.
The difference between "in the sleeper cab" and "outside the truck" are huge. We first started doing in the sleeper cab in Jul'17, took us 2 years of dedication to get confidence to remove from vehicle.
To take the person out of the vehicle, the system itself needs to be able to self-monitor and pull itself over if it stops working. That's the big lift.
That's it. The team at Uber did that like a year or so later but with no public fanfare because it wasn't considered noteworthy. That still puts it at ~2 years prior to this accomplishment.
This to me very much assumes no leap forward. See the start of deep learning - sudden rapid progress. I don’t see why there can’t be another jump like that. Much like the chip manufacturers mentioned in the post.
No idea what that advance may be but it’s out there I think
Perhaps there is no leap forward in long haul trucking. Yet there are many supervised machine learning products in the market and doing well (Tesla autopilot, iRobot vacuums, etc). I bet the founder can find a niche that could be marketable and can be mass produced or scaled.
Sad news. Apart from improving upon ML, can more sensors and physical modeling get us past the AV plateau?
An example of an additional sensor would be an infared camera. I don't know how well infared cameras work during the day or when they are occluded, but they seem like they can provide an additional datapoint to classify an object as "person" for example. That way, it doesn't matter if they're wearing a paper bag Halloween costume, the car still knows what's underneath.
Physical modeling seems like it would help with prediction. If a deer is idle on the side of road ahead, a car with a physical "deer" model would account for the probability that the deer will suddenly dart across the road and the speed at which it can possibly do so. It will slow down and watch for certain behaviors, such as the deer standing up from a prone position, and modify its probabilities accordingly. That is, after all, what we humans do. The ML wouldn't need to perfectly model all animals. Some animals would be put into a "mammal" class or more vague "animal" class, paying special attention to size and stature.
Additionally, depending on how long it might take to develop truly autonomous self-driving cars, we might want to develop self-driving car friendly infrastructure. I'm talking about AV-only lanes with safeguards in place to minimize variability, vehicle-to-vehicle communication, infrastructure-to-vehicle communication, and better than average maintenance. It seems like it might be worth it, given the gains to be had.
Here's an underappreciated angle to the AV-friendly infrastructure: it's a hedge against incorrect predictions. What if truly autonomous vehicles are actually 15 years away? In that case, if an "AV heaven" is developed in a corridor somewhere in Los Angeles 5 years from now, that'll be the only place where the AV dream is realized for the ensuing 10 years. The profit from "AV heaven" can be reinvested into AV development. It can also be used to expand into other areas. Moreover, AV heaven might galvanize the government, public, academia, and investors to eliminate any bottlenecks in AV development. There's no immutable stone tablet, only viewable by God, that says "AV development must take X years." We can modify the timeline to some extent. Lastly, AV heaven would allow us to observe the true value of self driving cars. What if they're better than we ever imagined? What if they radically change human behavior? What if they increase development in surrounding areas by a lot? What if they change the nature of work and where people live by more than we even expected? What if self-driving trucks + last mile bot delivery radically push down prices on everything and enable previously impractical business models or behavior? We actually know many of the changes that AVs will bring, but, critically, we don't know the magnitude of those changes.
I worked at Starsky Robotics as a perception team intern after graduating high school. I will always be grateful for the team for the opportunity, it was a fantastic first job and everyone who worked there was very kind (especially Stefan).
Unfortunately, Starsky had effectively had no machine learning in 2017 (when I worked there), using solely classical computer vision techniques. This didn't match the company's ambitions of not using LIDAR and there was a strong stigma against switching to a deep learning approach. At the time, very few object detection models had public implementations and I spent a lot of time trying to get a YOLO9000 and RetinaNet implementations running at real-time speeds. Frustrating, as a small startup the labeling services kept screwing us over by returning poorly annotated images.
I think what I took away from the experience is that deep learning in domains with long tails requires a enormous investment in a labeling pipeline - dwarfing the computational aspect - to get decent results. I don't think any solutions are on the horizon that will allow us to bypass this reality. You don't see improvements between Comma.ai and Tesla because it's about the improvements far out on the tail.