Finally someone beat Montezuma's Revenge without imitating human demonstrations! Very cool. I wonder why the algorithm then fails so hard on Pitfall? I would expect them to be similar problems.
All right, you led me into an interesting rabit-hole there, I saw this AI winter page, and after reading a bit , the guy( Filip Piekniewski) said: "...the only problem really worth solving in AI is the Moravec's paradox, which is exactly the opposite of what DeepMind or OpenAI are doing...". Then I dont know what the guy thinks about what is the exact meaning of the said paradox, but in my ignorance I went to check Wikipedia, and they gave me:"Moravec's paradox is the discovery...that, contrary to traditional assumptions, high-level reasoning requires very little computation, but low-level sensorimotor skills require enormous computational resources" Well, this sounded pretty obvious, if you accept "high-level reasoning" as meaning...well...lets see what Wikipedia says a few lines later, quoting Moravec himself: "As Moravec writes, "it is comparatively easy to make computers exhibit adult level performance on intelligence tests or playing checkers,...". So, I can see the same contradiction in both, in the Wikipedia's definition and in Moravecs statement, in my opinion and i think the common sense opinion,"intelligence tests" sounds very broad or very narrow, but anyway useless to make a point about the point, even more when "playing checkers" is in the same sentence, the common sense is that from playing checkers to the "high-level reasoning" used by Wikipedias definition (makes me laugh, sorry for the sincerity), there is an ocean of distance. The amount of memory required to do real "what-i-call" high-level reasoning can be huge only to store the heuristics algorithms execution codes, not to mention all the "concepts" of "things" and their possible ways of "interaction" and all the interactions and their possible subjects-"things" , and the probable mechanisms that we have no idea are in place, probably driven by the most primitive reaction-reasoning mechanisms, who in heavens knows...just my-an opinin after biting the bait : ) , in form of a light comment. Cheers.
Just to add a bit about the point of the "AI winter reality check" X "Stephen Hawkings hype", I think both are right and wrong at the same time: I do believe that the analisis of the human reasoning and its simulation in computers can be achieved with much less computing power than that needed by most insects sensorial systems(so in this sense agree with Moravec, but...). The computing power can accelerate learning in a system, but its a question of time and nothing more, it doesnt matter really if you take a week or a year to "set-train-load" a reasoning human level AI, the achievement would be amazing anyway. So is the mistake, in my opinion, of all the wave-of-hype's direction: neural nets is just for acceleration and generation of datasets(what you say!? yes, generation, thats what they are really for, totally contrarilly to the world's opinion : ) !!! ) and , obviously for the "lesser utility" of sensorial system autonomous programming, that is what they are using it for right now, but it have nothing to do with "Reasoning Intelligence", that would be, in ma humble opinion, lets say, a programm that could take part and give some insigthful contribution in the famous talk between Albert Einstein and Rabindranath Tagore.[1] That said, I do belive a non-neural net "real deal" AI is just around the corner, not that far, really, and here maybe I do tend a bit to the "Hawkings-hype" people, but completely differently from them, I do believe it has nothing to do with neural nets(that are just tools), and are...lets say: "non-neural networks heuristic based".
Sorry, It was not the intention to put the code tag, I tried to edit it to remove the scroll bar, but I couldnt, have to learn more about this, first timer.
By the way, the fact that "average" and "best" human performance are presented as meaningful benchmarks is one of the biggest signs that modern AI is driven by hype, rather than science.
For example, speech recognition AI is supposedly within fraction of a percent from "average human level", and yet auto-generated captions are awful. They have no punctuation, they don't distinguish between different speakers, they aren't visually grouped, and fail miserably dealing with slang. So turns out researchers are measuring only one aspect of the problem their algorithm is good at and ignoring the rest.
On the flip side, we have animal intelligence. Bees aren't nearly as smart as humans. So surely modern AI, which surpasses humans in this and that, would have no problem outperforming a bee with its 960 000 neurons, right? But in reality, there is nothing even approaching bees' versatile intelligence. Of course, modern AI researchers would just hand-wave this saying the problem is not well defined. Convenient.
> For example, speech recognition AI is supposedly within fraction of a percent from "average human level", and yet auto-generated captions are awful. They have no punctuation, they don't distinguish between different speakers, they aren't visually grouped, and fail miserably dealing with slang. So turns out researchers are measuring only one aspect of the problem their algorithm is good at and ignoring the rest.
YouTube captioning != SOTA, any more than Google Translate for years and years represented anything close to the NMT SOTA.
Well, for a certain definition of 'human performance'... I believe that's carried over from the DQN paper and is something like 'an ordinary video game player given a few hours'. When it comes to ALE you should usually treat the 'human performance' numbers as being lower bounds.
(In this case, if an agent can beat 'human performance' by only clearing 1 of 9 total levels, one is entitled to a little skepticism about how useful 'human performance' is as a benchmark for this particular game. Focus on the improvement over other DRL agents, not that.)
