Fun interview but barely any meat for those in the field. Just very general questions and answers, saying that the road is murky, but nothing about e.g. if Transformers / attention are the way to go forward, multi-modal models, reinforcement learning + self-supervised learning.
I do not question for a second that Carmack is a computing genius, but “murky” is an understatement. There is nothing of technical, scientific or business value in this interview. It’s just generic words strung together.
Why do people think he is a computing genius? He read contemporary papers in computer 3D rendering and implemented well understood concepts in a shitty dev environment (early x86 and VGA systems). When I do that for my company, I'm just a junior dev.
People really need to stop with this "Great person" nonsense. He's a pretty smart coder, and is gifted with geometry and other fields of math. He's not a genius. He didn't "master" calculus at age 15 like Einstein, he didn't invent anything particularly new in the field. Why the obsession people have with him? Why should we look to him for AI questions? What evidence is there that he has any new knowledge?
> Why should we look to him for AI questions? What evidence is there that he has any new knowledge?
He covers this in the article. He doesn't. He's just trying stuff out with a different approach than others because he believes (and is probably correct) that there is a chance that the most efficient path forward to AGI isn't the work that OpenAI and others are doing.
Even socially, it fails to plumb the depths. Mr Carmack is taking a "different path" by ... reading the relevant literature and talking to YC? This is 100% mainstream - The difference is what?
There is a comment in the article about having models watch TV and play video games, and he's talked about that before too in his Lex Friedman interview. Seems like his approach is to take existing model architectures, apply some tweaks and experimental ideas and then use datasets consisting of TV (self supervised learning maybe?) and classic video games (RL I guess?).
The video game part at least sounds like what Deepmind is already doing. I guess we'll just have to wait and see what he plans to do differently.
It seems to me like his expertise would be most valuable in optimizing model architectures for hardware capabilities to improve utilization and training efficiency. That will be important for AGI especially as the cost of training models skyrockets (both time and money). If I was a startup doing AI hardware like Cerebras or Graphcore I would definitely try to hire Carmack to help with my software stack. Though he doesn't seem interested in custom AI hardware.
It’s a puff piece in a general interest magazine. It’s not going to go into details. I also got the impression that Carmack was being cagey about the directions where he saw potential.
Yep. Does it ever get around to answering the question implicit in the title: what is it that he is going to do "different" to the rest?
Seems more like he's talking to and following up with Altman, "Y Combinator conference" and the rest. Is that "bucking the trend", taking your own "path", really?
I assume it's to know which directions have shown the most recent potential, to catch up on the techniques and literature so you can talk/think intelligently on the matter. But I see your more general point, he has to be careful not to get caught up I the groupthink he is (understandably?) critical of.
The area of AI research isn't some outlier of super geniuses that don't exist anywhere else. Carmack has often worked around some of the best talent on the planet and he has always stood out.
