Unfortunately this argument, while imho entirely valid and also frequently seen in discussion, is unable to stop the massive train that has been set in motion.
There are various types of retorts: i) the brain is also doing gradient descent; ii) what the brain does does not matter (if you can fake intelligence you have intellegence) iii) the pace is now so fast that has not happened in decades will happen in the next five years etc.
None of them is remotely convincing. I would really start getting worried if indeed some of the AI fanboys had a convincing argument. But we are where we are.
Somehow this whole episode will be just another turn of the screw in adopting algorithms in societal information processing. This in itself is fascinating and dangerous enough.
The brain is definitely not doing gradient descent. That is the biggest issue with using artificial neural networks as a neuroscience modeling tool. The fact other learning algorithms have not come close to gradient descent performance in most cases has therefore largely kept neuro theory as a niche in modern academic research -- despite its entanglement with the early development of artificial neural networks.
Not taking a stance on whether "intelligence" can emerge out of gradient descent but it's certainly not biological.
I don't think there is any strong argument for AGI (i.e., that it is somehow coming any time soon). In my explanatory framework all developments make sense as purely statistical algorithmic advances. The surprising and interesting applications involving images, language are not really changing that fundamental reality.
There is a case to be made that with sufficient ingenuity at some point people will expand the algorithmic toolkit into more flexible and powerful dimensions. It may integrate formal logic in some shape or form, or yet to be conceived mathematical constructs.
But the type of mental leap that I think would be required just to breakout of the statistical fitting straight-jacket cannot be invented to satisfy the timing of some market craze.
If you look at the broad outline of the development of mathematics there are many areas where we have hit a complexity wall and generations of talent have not advanced an iota.
Even if we condition on some future breakthrough, the next level of mathematical / algorithmic dexterity we might reach will follow its own intrinsic logic, which will probably be very interesting but may or may not have anything to do with human intelligence.
There are various types of retorts: i) the brain is also doing gradient descent; ii) what the brain does does not matter (if you can fake intelligence you have intellegence) iii) the pace is now so fast that has not happened in decades will happen in the next five years etc.
None of them is remotely convincing. I would really start getting worried if indeed some of the AI fanboys had a convincing argument. But we are where we are.
Somehow this whole episode will be just another turn of the screw in adopting algorithms in societal information processing. This in itself is fascinating and dangerous enough.