I agree. The systems in place already solve generalized problems not directly represented in the training set or algorithm . That was, up until the last few years , the off the shelf definition of AGI.
And the systems in place do so at scales and breadths that no human could achieve.
That doesn’t change the fact that it’s effectively triple PHD uncle Jim, as in slightly unreliable and prone to bullshitting its way through questions, despite having a breathtaking depth and breadth of knowledge.
What we are making is not software in any normal sense of the word, but rather an engine to navigate the entire pool of human knowledge, including all of the stupidity, bias, and idiosyncrasies of humanity, all rolled up into a big sticky glob.
It’s an incredibly powerful tool, but it’s a fundamentally different class of tool. We cannot expect to apply conventional software processes and paradigms to LLM based tools any more than we could apply those paradigms to politics or child rearing and expect useful results.
> The systems in place already solve generalized problems not directly represented in the training set or algorithm
Tell me a problem that an LLM can solve that is not directly represented in the training set or algorithm. I would argue that 99% of what commercial LLMs gets prompted about are stuff that already existed in the training set. And they still hallucinate half lies about those. When your training data is most the internet, it is hard to find problems that you haven't encountered before
o3 solved a quarter of the challenging novel problems on the FrontierMath benchmark, a set of problems "often requiring multiple hours of effort from expert mathematicians to solve".
I’m having a hard time taking this comment seriously, since solving novel problems is precisely what LLMs are valuable for. Sure, most problems are in some way similar in pattern to some other, known one, but that describes 99.9 percent of what 99.9 percent of people do. De novo conceptual synthesis is vanishingly rare and I’m not even sure it exists at all.
And the systems in place do so at scales and breadths that no human could achieve.
That doesn’t change the fact that it’s effectively triple PHD uncle Jim, as in slightly unreliable and prone to bullshitting its way through questions, despite having a breathtaking depth and breadth of knowledge.
What we are making is not software in any normal sense of the word, but rather an engine to navigate the entire pool of human knowledge, including all of the stupidity, bias, and idiosyncrasies of humanity, all rolled up into a big sticky glob.
It’s an incredibly powerful tool, but it’s a fundamentally different class of tool. We cannot expect to apply conventional software processes and paradigms to LLM based tools any more than we could apply those paradigms to politics or child rearing and expect useful results.