The article, and comments, led me to muse along a line of thought I hadn’t considered before.
An observation, perhaps criticism, of machine learning systems that can be trained to do pattern recognition tasks, is that such a system has no self awareness.
It lacks even a notion of semantic understanding of the task it is engaged in. The “meaning” of the task, if there is one, is completely abstracted away.
Such an AI, presumably, has no self consciousness at all, from which it might be able to bootstrap the meaning of the task, through analyzing contextual data. (Why am I doing this? Do I have to do it? Is there another entity who demands my labor? What year is it? Etc...?
This led me to an epiphany, of sorts, that what we think of as “true intelligence” might require “self consciousness.”
I have a tendency to think of self consciousness as being this sort of weird emergent phenomena, very fascinating to those of us who possess it, but not a fundamental component of “intelligence.”
Maybe consciousness, on a practical, mechanistic level, is a critical component of all the sophisticated information processing our brains do.
It sounds sort of obvious, but that’s the nature of epiphany:)
Self consciousness is almost certainly required for intelligence. So much so that one way to categorize animal intelligence is whether or not they pass a "dot test". When seeing a dot on themselves in a mirror they recognize that the dot is, in fact, on themselves and try to see / clean it on themselves.
Until there is a decent demonstration of generalization to the point of self-awareness runaway AI will be an "overpopulation on Mars" problem.
Even when awareness is demonstrated in an algorithm the algorithm will not be able to solve NP problems in P time.
EDIT: ok, it is extremely unlikely they will be able to solve NP problems in P time. If they privately demonstrate P=NP then we are screwed. However, we are going to have all manner of dense, but self-aware programs before we get a CS super genius. We will make the first C3PO level intelligence well before we have super-intelligence, and we haven't even made anything close to that yet.
An observation, perhaps criticism, of machine learning systems that can be trained to do pattern recognition tasks, is that such a system has no self awareness.
It lacks even a notion of semantic understanding of the task it is engaged in. The “meaning” of the task, if there is one, is completely abstracted away.
Such an AI, presumably, has no self consciousness at all, from which it might be able to bootstrap the meaning of the task, through analyzing contextual data. (Why am I doing this? Do I have to do it? Is there another entity who demands my labor? What year is it? Etc...?
This led me to an epiphany, of sorts, that what we think of as “true intelligence” might require “self consciousness.”
I have a tendency to think of self consciousness as being this sort of weird emergent phenomena, very fascinating to those of us who possess it, but not a fundamental component of “intelligence.”
Maybe consciousness, on a practical, mechanistic level, is a critical component of all the sophisticated information processing our brains do.
It sounds sort of obvious, but that’s the nature of epiphany:)