The cool thing about math applications is just how easy it would be to generate synthetic data. That these large language models haven't attempted to supplement their gigabytes+ scale data sets with such is an oversight.
Note, you’d need to train such a model on data teaching it about the relationship of every number to every other number when run through every function. Yes, infinite synthetic data, but you’re just memorising stiff you can already generate
Or build a model that has "peripherals". Oh, I'm being asked to do math. Let's put it in my calculator app. Everything doesn't have to be in one uniform network.
Evidently the brain works that way: the cortex is built on top of older components, so it doesn't have to figure out basic metabolism the same way it has to learn to identify people.
- design certification treated as "just another 737" which ignored fundamental deviations in hardware and software, complicated by safety certification by manufacturer instead of public employee
- hardware design including a "dynamic instability" in which
airplane approaching an aerodynamic stall had a tendency to go further into the stall due to lift produced by the oversized engines at high angles of attack, which was intended to be mitigated with software
- omission of using multiple inputs, including the opposite angle-of-attack sensor, in the computer's determination of an impending stall
- a changed philosophy about human/machine interaction from humans winning a battle of the wills every time to computers winning a battle of wills in cases of envelope protection
The final item is perhaps the one most fundamental to other cases of safety critical human machine interaction designs. If we are signing over agency to machines for envelope protection, that means we need in advance to understand every potential edge case scenario where that envelope may be mis-framed. Such comprehensive foresight in some environments may be intractable. For the 737 this was exasperated by the presence of an inherent source of instability originating from hardware design.
Based on wikipedia it appears the original definition of perceptron was associated with the specific case of a neuron with a heavy-side step function activation output (which maps aggregated weights to either 0 or 1). I've generally adhered to referring to neural networks nodes in my writings as "neurons", which I believe is a catchall for any type of activation output, although have seen others use the term "perceptron" in the same usage in modern practice.
The biggest asset of Facebook for now is the facebook platform, perhaps a more durable asset will be the social graph sourced by the facebook platform. What would it mean for a durable social graph commercialization model? I'm not sure, but at a minimum it would need to be exportable and portable between platforms and ecosystems.
He was also a jazz musician (the clarinet), a somewhat accomplished juggler, a devoted unicycle enthusiast, and left behind a basement full of contraptions he was building in various states of finish - like the electronic mouse navigating a maze, a chess playing machine, and all other kinds of curiosities. His papers are coherent and still relevant to this day and follow the birth of each of these fields like information theory and artificial intelligence. Who knows what else he might have been working on at Bell labs that we may not be privy too.
Not explicitly mentioned in TFA is that Shannon was the first to apply the Minimax algorithm [1] for computer chess. The Minimax algorithm, later streamlined to include alpha-beta pruning, has been a key component in AI game playing machines ever since.
I've posted this before, a compiled list of the machines and gadgets Claude Shannon built to experiment with simple AI ideas to play games [3]. Apologies for the repeat:
It actually is, most papers written today are cut and paste jobs with multiple people writing one section. The quality and focus from one paragraph to another can change. I think the lack of computers helped a bit as well for the quality of earlier papers. There used to be a lot of effort needed to publish a single paper since computers couldn't do typesetting.
Aside from that he has what comes off as a very clear mind. If you haven't I would highly recommend reading his papers. Coherent is a good adjective to describe his work.
Just like in order to become a composer of music you need to start by performing other people's music, I think it helps in writing to start by responding to other people's ideas, dissecting and evaluating, which helps you build the competence to generate ideas of your own.
If you try to wait for the money making opportunity in order to build something it may never come. Just start building. Keep building. There are more ideas at the end of a long path than there are at the start. If you don’t keep building they will remain unseen.
I know this is somewhat of a false dichotomy, but at some point I think PG's essays started to shift from being directed at startup founders to giving advice to his children that they can read when they grow older.
There probably is some tethering to whatever his perspective is at the time. At some point, YC was a relatively new idea coming to life and he was probably constantly thinking a certain way. Now, it's something that he's been up to for decades.
That said, I think there's more of a zeitgeist change than actual change in pg's content. Things sound different 10-15 years apart. A lot of things age poorly, often: idealism, stand up comedy... most anything avante garde-ish.
Clever people spoke highly of agile,for example, when it was manifestos and such circa 2005.
Ok I just looked up the phrase false dichotomy, apparently doesn't mean what I thought it meant, probably would have been better said as "the two are not mutually exclusive", still point holds.
"False dichotomy" implies more than just the fact that "the two are not mutually exclusive". It further implies that the speaker in question has implied that the two are mutually exclusive, when they are in fact not.
I feel like a lot of it is the same stuff that you get in generic self help books, but explained in contemporary techie language and cultural references.
Not that that is necessarily bad per se, there can be a lot lot of value to reminding people of things that may seem obvious. But it's annoying when people treat him like a genius for saying fairly standard platitudes in a clever way
