Any background info on the betting on cameras alone? It sounds as silly as betting on an artificial version of our proprioception to be implemented in cars to measure acceleration. I also don't think they went all the way regarding neuromorphic engineering with spiking neural nets and artificial retinas. It's just so random to me what was decided to be good enough for autonomous navigation.
Tesla went from very expensive cars down to cheaper ones. It would make so much more sense to do the same for perception. First go over board and go for high bandwidth input and lots of processing power and optimize later.
The betting on cameras alone is basically an Elon Musk thing. His reasoning is basically that if humans can do it AI should be able to do it. So far the software isn't really up to it but time will tell. Some stuff - https://www.engineering.com/now-revealed-why-teslas-have-onl...
I used to regularly have to make a left turn onto a rural highway on foggy mornings. Sometimes people drive faster than they should in fog. Sometimes fast enough that by the time they could see I'm in the intersection turning they would be too close to stop.
Cars going fast enough to have that problem made enough sound that they could be heard quite a bit farther away than they could be seen. I'd open my windows at the intersection and listen until I couldn't hear any highway traffic. Then I'd know that any approaching cars are far enough away that I should have time to turn onto the highway and get up to speed before they arrive.
Yeah. Also I don't know how good the Tesla cameras are but my car has a reversing camera and it's ok for going back 2m at 2mph but kind of terrible compared to looking forward through the windscreen.
IIRC I think it’s the section (1:23:25) – Camera vision
The TL;DR is that sensor fusion is really hard, and their bet was that keeping the training pipelines simpler would let them scale faster/easier, and human vision is the existence proof that it can be done without lidar.
One of the big flaws in Karpathy's logic is that it implies human vision is acceptable and sufficient for an AV. The reality, as Cruise found out, seems to be that society will demand AVs are much safer than humans.
Human vision is an existence proof for human-level performance without lidar, but Waymo is an existence proof for 10x human performance WITH lidar. Right now the latter is where the bar is, and it'll keep being raised. I don't think at this point one could get away with deploying AVs at scale that are significantly less safe than Waymo.
Also: if sensor fusion is so hard, why is Waymo able to solve it but not Tesla?
> Also: if sensor fusion is so hard, why is Waymo able to solve it but not Tesla?
I think Karpathy's point is that Tesla wants to try to avoid the "entropy" that comes from adding a sensor (senior software engineers and higher understand this concept). Every sensor (and every version of it -- sensor hardware does get updated) you add requires recalibrating the software stack, the hardware design, which introduces points of failure every time you roll it out.
According to Karpathy, Tesla does use Lidar -- but only at training time, as a source of truth. Once the weights are learned, they operate without the Lidar.
Have a full sensor suite may work for Waymo at the current scale (limited cities), but scaling beyond that poses problems.
Whereas Tesla has to work with a different set of scaling economics -- that of a mass market vehicle already deployed globally.
"I've often found myself uneasy when LLMs (Large Language Models) are asked to quote the Bible. While they can provide insightful discussions about faith, their tendency to hallucinate responses raises concerns when dealing with scripture, which we regard as the inspired Word of God."
Interesting. In my very religious upbringing I wasn't allowed to read fairy tales. The danger being not able to classify which stories truly happened and which ones didn't.
Might be an interesting variant on the Turing test. Can you make the AI believe in your religion? Probably there's a sci-fi book written about it.
> In my very religious upbringing I wasn't allowed to read fairy tales. The danger being not able to classify which stories truly happened and which ones didn't.
Thanks for sharing. You might be interested: JRR Tolkien, 'fairy tale' author, was also a/the leading scholar of Old English (Anglo-Saxon), related languages, and the culture and myth around them - including 'fairy tales'; and he was a devout Catholic.
How could he write (and study) such ungodly material? He wrestles with the question multiple times, but if you are interested, I strongly recommend On Fairy-stories, an essay based on a lecture. It covers far more ground than this question, but it's worth reading anyway. I'll append a spoiler below.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
[SPOILER]
There's more to it than this, but it's a wonderful vision:
"The Gospels contain a fairy-story, or a story of a larger kind which embraces all the essence of fairy-stories. They contain many marvels - peculiarly artistic,[1] beautiful, and moving: 'mythical' in their perfect, self-contained significance; and among the marvels is the greatest and most complete conceivable eucatastrophe. But this story has entered History and the primary world .... The Birth of Christ is the eucatastrophe of Man's history. The Resurrection is the eucatastrophe of the story of the Incarnation. This story begins and ends in joy. It has pre-eminently the 'inner consistency of reality'. There is no tale ever told that men would rather find was true, and none which so many sceptical men have accepted as true on its own merits. For the Art of it has the supremely convincing tone of Primary Art, that is, of Creation. ..."
[1] "The Art is here in the story itself rather than in the telling; for the Author of the story was not the evangelists."
Building technology might be actually kind of easy compared to finding product-market fit. The latter might be better described as "sales team - network - existing practices - existing monopolies - adoptation rate - product iteration speed - right region" fit.
Yeah - casually ruling out the moon and sun in the article's first sentence feels a bit underhanded. Beyond pedantry, what's the point of the "Planets Only" sign?
