Agreed - I think LLMs may be an important component in our eventual robot overlords, but we're not all the way there yet.

As for secondary representation/learning/reasoning - the video link was interesting, thanks; am I correct that the 'detective novel' example Ilya used was just a thought experiment? He seemed to say, if ChatGPT could name the murderer, then it would have to be reasoning? But we haven't demonstrated ChatGPT actually doing this yet have we? The token window is surely not big enough for an entire novel. So I think maybe they're jumping ahead a little bit there.

Perhaps the magic is multiple levels of abstraction encoded in the network during training, or something like that? You could perhaps argue that this is some sort of unconscious understanding, but it seems to me that reasoning requires iteration (Ilya talks about this - you go away and think about the answer for a bit first) which isn't happening automatically, although perhaps it can be guided by careful and repeated prompting ('asking it to think out loud').

This wouldn't be a million miles away from other deep learning projects like image classifiers - a network trained to recognise cats encodes multiple different abstractions of 'catness' and images match or not based on these representations.

Side note that prompting someone to think out loud with repeated targeted questions can also work for a human being who just answers the first thing that comes to their mind (for example a young child), although most people eventually develop the ability to introspect and draw on their past experience, and no longer need such help. We could maybe broadly categorise human intelligence as a combination of

- short term memory, providing context and immediate goals (e.g. 'switch on the light')

- long term memory, encoding all kinds of experience

- learning, which shapes long term memory based on short term memory (the current context; remembering the things that are happening now) and also based on long term memory (relating the current context to previous experiences)

- introspection, which allows iterative 'self-prompting' to improve the accuracy of a thought, before saying it out loud but also during learning

- language generation, which is mainly about word association based on long and short term memory

- what else?

'Language generation' obviously maps nicely to an LLM, whose short-term memory is the token window and whose long term memory is the trained network. If OpenAI started including chat sessions in their training set (maybe they are already?) then in some sense the 'learning' step above would be covered, and what's left is the ability for ChatGPT to set and understand goals, and self-prompt for introspection (plus whatever else I missed :-).