Hacker News new | past | comments | ask | show | jobs | submit login
Generalist AI Doesn't Scale (daemonology.net)
2 points by cperciva on April 6, 2024 | hide | past | favorite | 2 comments



Specialist AI doesn't scale. When you train separate models, each one starts from scratch, so you get no transfer / informative priors. So your specialist models wind up collectively scaling much worse than the power law of the generalist. And because of that, your collection of specialized models winds up being leapfrogged by a generalist model which avoids being 'penny-wise pound-foolish'. In the real world, boundaries are not sharp, and everything is "hallucinations of modern artwork depicting polypeptides"; this is precisely why GOFAI approaches failed!


What's should be information transfer between journalism AI and espionage AI? They are the opposite, avoiding contamination is the point! Sometimes, subjects are actually separate and "transfer" is "contamination". I don't want postmodernist thinking on my arithmetic AI.

This does move the goalpost to having an AI to select the AI to query, but that's a much lower bar to clear, it's almost a solved problem with sentiment analysis-type engines.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: