I am becoming somewhat of a broken record, but sigh..

To predict the next token you must reason or have some process that approximates it.

“Given all these various factors, the most likely resolution to our conundrum is: …”

Good luck doing that with any kind of accuracy if you lack intelligence of any kind.

Language is a distraction. These things reason (badly, atm). It is totally unclear how far this goes. It could fizzle out, it could become our overlord.

It clearly does not reason. Take a famous riddle and make a paradox change. It will not create a meaningful response.

But yes, there is a lot of knowledge embedded into our global use of language and it is fascinating to see how it can be reproduced by such a model.

Does the following satisfy your requirement for "a famous riddle with a paradox change"? Because GPT-4 aces it most of the time.

"Doom Slayer needs to teleport from Phobos to Deimos. He has his pet bunny, his pet cacodemon, and a UAC scientist who tagged along. The Doom Slayer can only teleport with one of them at a time. But if he leaves the bunny and the cacodemon together alone, the bunny will eat the cacodemon. And if he leaves the cacodemon and the scientist alone, the cacodemon will eat the scientist. How should the Doom Slayer get himself and all his companions safely to Deimos?"

Furthermore, it will reason if you tell it to reason. In this case it is not necessary, but in general, telling GPT to "think it out loud before giving the answer" will result in a more rigorous application of the rules. Better yet, tell it to come up with a draft answer first, and then self-criticize by analyzing the answer for factual correctness and logical reasoning in a loop.

People will see patterns in this riddle and claim it is “just” altering those. “It’s just a bunch a patterns where you can switch the names, like templates”.

Isn’t everything like that?

“Uhh…”

I had the same discussions about chess.

“It has just memorized a bunch of high level patterns and juggles them around”.

I agree, but now I’m curious what you think chess is.

“Chess is not intelligence.”

Goalposts? Anyway, we move on to Go, the game. Same response. Programming, same, but the angle of the response is different now because programming is “clearly” intelligence incarnate.

“It programs and sometimes correctly, but it is a mirage. It will never attain True Programming.”

I’m sitting on the bench riding this one out. We’ll see.

I fed GPT-4 some really old fashioned spatial reasoning questions (inspired on SHRDLU), which it passed. Then when questioned about unstable configurations (which IIRC SHRDLU could not handle) it passed those too.

So it seems like it is definitely capable of some forms of reasoning. Possibly we both tested it in different ways, and some forms of reasoning are harder for it than others?

If reasoning is amenable to being “embedded” at all then we should perhaps reconsider its fundamental nature?

It’s easy to say something like that, but what does it mean in situations where it is producing a novel and correct answer that isn’t guessible?

What does it has to do that will qualify it as artificial intelligence ?

In my opinion, all the ingredients are there for artificial intelligence. Somebody just need to stitch everything up. It can understand text, reason about it, identity next steps, can write code to execute the steps, understand error messages and fix the code. That feels like AI.

We seem to keep moving the goal posts as to what is intelligence. I'm not sure is that ego? Or instead when we describe a thing and then see it, we say, “That is not what I meant at all; That is not it, at all.”

Be able to say "I don't know".

(You've just excluded a few people I know.)

GPT 4 tells me that on a daily basis because of the 2021 data cutoff.

So it never hallucinates APIs, theorems, biographies or places?

Well I did at one point compare the outputs for two biographies of not very well known people, 3.5 made up half of the data, 4 only said it doesn't know anyone by that name for both. I don't think I ever tried asking about any places or theorems specifically.

As for APIs, well I try to always provide adequate context with docs, otherwise it may still on occasion make up some parameter that doesn't exist or uses another library by the same name. Half the time it's really my fault by asking for something that just isn't possible in a last ditch effort to see if it can be done in some convoluted way. It sort of assumes "the customer is always right" even if it contradicts with what it knows is wrong I guess. It gets it right usually when it's at least mostly straightforward to implement something though.

It's much better at avoiding hallucination than the 3.x generation was.

The first derivative is all that matters. ML gets better over time; we don't.

God damn the down votes. I agree with your overall thesis I don’t personally know what intelligence means. However it’s just a tool even if it is AI or whatever you want to label it. It’s extremely cool and powerful too. It scares the shit out of me as well. However I think we are also taking the hype to 11/10 when it should be much lower out of 10 than that…

It reminds of me the rhetorical question: if we found intelligent alien life out there in the universe, would we even be able to recognize it as such?

The same question may apply here: if artificial intelligence emerges, would we be able to realize that it is intelligent?

I have no idea. Probably not. The philosophy is a deep rabbit hole. A fun one to ponder, and I like having that discussion. Maybe the more cynical pragmatic old man swe that I am sees a super powerful calculator kind of. It’s very very cool, and obviously I can’t hold a conversation with a calculator but my analogy is more to say my calculator can do really complex integrals and even show its steps kind of! Especially for my handy TI-89. It was my best friend for all of engineering undergrad. I see chatgpt as a steroid version of that for all of Language. Code is another language, writing is a language, painting in some ways is a language.

So you are 100% certain that no emergent properties can develop from a LLM that transcends the limitations of LLMs. I haven’t read any LLM literature, so I am honestly asking, do you know of anything close to a proof that such emergent properties cannot develop?

If course, there can be emergent properties. It will be fascinating to watch this research. But it is not going to develop another model that is even more capable.