And the entire point of this article is to show what has changed since AlexNet (spoiler: a lot!), let alone GPT-3/DALL-E/Midjourney/Whisper/ChatGPT/Imagen/Imagen Video which aren't even in this article!

I wonder if what we're seeing is anxiety - that AIs are going to take my job, or create huge problems with spam/deepfakes - manifesting in frankly ignorant, jaded comments that completely ignore the huge positive potential.

People have been rewarded in the past for being skeptical, and have learned this trick to a detriment to their own intellect. It is a very easy trap to confuse skepticism with expertise on the internet, and HN is one of the worst about this - most of the comments that feign skepticism barely even understand the topic they are commenting upon anymore, and are simply generating skeptical words because that's what has rewarded them in the past. There is also unsubstantiated hype, which is not helpful either.

Occasionally someone actually has read a few hundred fundamental papers on ML and can give an actual educated response, but it is quite rare. Typically they don't feign skepticism, but rather notice there are noteworthy improvements provided by metaRL and RLHF, etc.

I work in vision. Go look up the imagenet leaderboard. Look at the results of Alexnet vs the top result today. The trend is a log line. The top contending architectures still include CNNs trained on backprop, they’ve just had a decade of tricks applied to eek out some improvements. The transformer based vision models aren’t much better.

Talk to any machine learning expert and they’ll tell you the math and fundamentals haven’t really changed since the 90s, we’ve just gotten better at scaling. Transformers came onto the scene half a decade ago and we could scale them much better than CNNs, but like CNNs of today, we’ve hit the diminishing returns limit.

Maybe look at actual data instead of being dismissive to different opinions.

I find delightfully ironic you just accused me of not having an intelligent take on AI, just trying to fake one.

That out of the way, the very term AI has been applied to automatic computation since its inception. And the current hype drive is nothing but marketing for software engineering done the hard way. You get one good chatbot by turning 8 years of internet into 1 tb of parameters on memory, costs nearly a million a day to run and... it can regurgitate semi-coherent prose. Wow. Talk about hype.

I'm not skeptical on AI to sound smart. Hell what do I get from some random anonymous account I use to read some blog aggregator? I am deeply skeptical of people selling hard some shiny new compute silver bullet that will do away with all the nasty complexity. Because it won't. We've been warned nearly 60 years ago about that.

Since you don't know squat about my background, maybe you're the one slinging snark around here.

The straight denialism around what transformer models can do has been very disappointing, though maybe not unsurprising. Acknowledging what that means acknowledging that your world is about to soon change in massive and unpredictable ways and the future you had planned and expected is not going to happen.

Vision models are still CNNs trained with backprop. Now they’ve begun to incorporate transformers into vision task, but the results aren’t x10 better. Please explain to me how vision models today are massively different than Alexnet and not just a bunch of slight optimizations and tricks to eek out some marginal accuracy improvements.

If you're talking about vision, fine. But you were replying to a comment about ChatGPT and then brought up AGI. No offence, but although some vision tasks are AI-hard, in general vision has minimal to do with AGI. That's why I quit the field. Transformers in vision may not be very interesting, but they certainly are a breakthrough in language.