A lot here depends on how you view generalization. Consider the Atari playing AI developed by Deepmind. It reached superhuman capability on a variety of different games given no domain specific knowledge. It had access to just the visual information and its score.
It's natural to claim that's not generalizing since it's in such a specific domain as well as the fact that it had access to its score. But I think you have to consider that life itself starts in the exact same way. We're little more than a vast number of evolutions starting from very simple organisms who's only purpose was to seek out sustenance. They had an extremely simplified domain with limited input thanks to fewer senses. And they also received a score. They were 'informed' of their sustenance and if it fell below a certain level, that was game over which they were also informed of --- at least in some manner of speaking.
I think the ultimate issue is that it's rather disappointing how unexciting it all is. And so we constantly shift the goal posts. Going back not that many decades ago it was believed that a computer being able to defeat humans at chess would signal genuine intelligence. Of course we managed to achieve that, but the matter of how it was done being so unexciting and uninteresting led us to shift the goal posts.
The achievements we're regularly producing now a days dwarf anything those speaking of chess=intelligence times could have even imagined. Nonetheless we still now go, "Nah.. that's not REAL intelligence either." But I think we will, up to the day we do genuinely create generalized intelligence argue that it's not 'REAL' intelligence. And I imagine we'll be arguing that's not 'REAL' intelligence even afterwards. Because it won't be magical, or exciting. I don't think there's ever going to be a "we've truly done it" moment. It's just going to be clever ways of teaching increasingly sophisticated tricks leading up to the point that a creation becomes more effective at most of any task, including assimilating and producing new knowledge and expertise, than we are.
Do we even have a good definition of what it means to generalize? I keep seeing this term thrown about and while it makes intuitive sense, and certainly train/test/validation splits are useful, I sort of question the whole premise. I mean, when we consider all possible sets, all optimization problems perform equally well (or poorly), so in the most general sense, generalization is impossible. Which means we have to restrict our sets somehow, but to do that we have to have some measurement as to how they relate, but how do you know in advance? In a sense what the machine is learning is a distance metric between these sets, but the only way to know it's working is to actually run it. To say in advance "well, this machine generalizes this well on this class of problems" seems like an awful stretch.
So much of what we mean when we say "this model generalizes well" seems like baked-in human assumptions about the data distributions that may not really have any basis in reality.
Generalize does need to be more specific in order to make sense. In current ML work, generalizability means something else than what it typically used to mean in AI work.
One way to look at what generalizability is would be like this: problem domains typically come with sets of axioms. Generalizability is the ability for a solution approach to work across different domains that don't have 100% axiom overlap. The wider the difference between axiom sets for a domain, the more difficult / impressive the generalization.
Solving two problems within the same domain, necessarily sharing a 100% overlapping axiom set, is not generalization at all.
The reason the axioms supporting the domain matter is that they (in part) guide which heuristics work, and how they should be applied. And that's the sort of generalizability that is missing from current work: some solutions can make some pre-programmed choices about applying different heuristics depending on the problem set (driverless cars are doing this now). This is the "bag of tricks" approach. But they don't typically morph in how they are applied, or the end that they're trying to accomplish when they're applied.
I have pondered this and have decided what my standards are. I realize I don't have the authority to set those standards for everyone.
GAI is, to me, when machine is able to be given a problem and then, without prompting, decides which data to consume to learn how to solve that problem. It could be told to optimize an automotive design for a 5% efficiency increase without a loss of safety features and while keeping the performance the same - and then go out and figure out what data it needs to learn so that it can solve that task. It would assemble and process that data and then come up with the answer, which might just be that it is impossible with current tech and here is what is needed and this is how to do it.
That's rather verbose and I'm absolutely not the person who gets to define it. But, when someone says AI, that is how I think of it. More so when they say general AI.
> And I imagine we'll be arguing that's not 'REAL' intelligence even afterwards.
When Data walks out of a lab, do you really think people will argue that he's not intelligent in the generalizable sense that humans are? Anyone who has watched ST:Next Generation agrees that he is.
> The achievements we're regularly producing now a days dwarf anything those speaking of chess=intelligence times could have even imagined.
I'm pretty sure the AI founders like Marvin Minsky could imagine Artificial Intelligence being generalized over various domains. I forget which one predicted the by the year 2000, an AI would be able to read all the world's books in an intelligent manner. McCarthy assigned a student to solving the robot vision problem over a summer back in the 60s.
Problem is that they underestimated the difficulties in getting a machine to perform tasks that are simple for us, while thinking that tasks like superhuman chess would be the difficult challenges. They had it backwards.
To add to your excellent point; it takes 10-20 years to fully train a human from useless baby to fully functional. And that time is filled with days of continuous exposure to data.
There is no problem domain that humans have been shown to hold an unshakable advantage. I used to be able to say Go but that is no longer even a little bit true. It could well just be time and data from here.
Programming, philosophizing, holding a conversation, playing a sport, applying concepts learned in one domain to another, and a common sense understanding of the world (Minsky's big one).
And it's not just 10-20 years of a blank slate being exposed to data. It's evolved brain structures that know how to learn those skills. That's why a young human child quickly surpasses the learning abilities of chimpanzee.
It's natural to claim that's not generalizing since it's in such a specific domain as well as the fact that it had access to its score.
Yeah. I would like to see the same approach applied to NES games[0] with the capability of beating a game like Zelda or Ultima IV with no domain-specific knowledge. It's not going to happen. These games assume the player knows about human culture, values, virtues such as courage and triumph over the forces of evil. Symbols such as the sword and shield, the heart; they have deep meaning even to a person who has never played a video game before.
To build an AI that can play a game like that, without any specific knowledge of the game itself, would be to build something that likely reduces to general intelligence, though I hesitate to claim it forcefully.
Deepmind also failed to reach human level on a a third of the games they trained it on. And I wonder about the superhuman level. Superhuman compared to average gamer, or really dedicated ones? Because there are people who have played perfect games of Pacman and Pitfall.
It's natural to claim that's not generalizing since it's in such a specific domain as well as the fact that it had access to its score. But I think you have to consider that life itself starts in the exact same way. We're little more than a vast number of evolutions starting from very simple organisms who's only purpose was to seek out sustenance. They had an extremely simplified domain with limited input thanks to fewer senses. And they also received a score. They were 'informed' of their sustenance and if it fell below a certain level, that was game over which they were also informed of --- at least in some manner of speaking.
I think the ultimate issue is that it's rather disappointing how unexciting it all is. And so we constantly shift the goal posts. Going back not that many decades ago it was believed that a computer being able to defeat humans at chess would signal genuine intelligence. Of course we managed to achieve that, but the matter of how it was done being so unexciting and uninteresting led us to shift the goal posts.
The achievements we're regularly producing now a days dwarf anything those speaking of chess=intelligence times could have even imagined. Nonetheless we still now go, "Nah.. that's not REAL intelligence either." But I think we will, up to the day we do genuinely create generalized intelligence argue that it's not 'REAL' intelligence. And I imagine we'll be arguing that's not 'REAL' intelligence even afterwards. Because it won't be magical, or exciting. I don't think there's ever going to be a "we've truly done it" moment. It's just going to be clever ways of teaching increasingly sophisticated tricks leading up to the point that a creation becomes more effective at most of any task, including assimilating and producing new knowledge and expertise, than we are.