>The sum of all human artifacts ever made (or yet to be made) doesn't exhaust the description of a rock in your front yard, let alone the world in all its varied possibility.
No human or creature we know of has a "true" world model so this is irrelevant. You don't experience the "real world". You experience a tiny slice of it, a few senses that is further slimmed down and even fabricated at parts.
To the bird who can intuitively sense and use electromagnetic waves for motion and guidance, your model of the world is fundamentally incomplete.
There is a projection of the world in text. Moreover training on additional modalities is trivial for a transformer. That's all that matters.
That's the difference though. I know my world model is fundamentally incomplete. Even more foundationally, I know that there is a world, and when my world model and the world disagree, the world wins. To a neural network there is no distinction. The closest the entire dynamic comes is the very basic annotation of RLHF which itself is done by an external human who is providing the value judgment, but even that is absent once training is over.
Despite not having the bird's sense for electromagnetic waves, I have an understanding that they are there, because humans saw behavior they couldn't describe and investigated, in a back-and-forth with a world that has some capacity to disprove hypotheses.
Additional modalities are really just reducible to more kinds of text. That still doesn't exhaust the world, and unless a machine has some ability to integrate new data in real time alongside a meaningful commitment and accountability to the world as a world, it won't be able to cope with the real world in a way that would constitute genuine intelligence.
>I know my world model is fundamentally incomplete. Even more foundationally, I know that there is a world, and when my world model and the world disagree, the world wins.
Yeah this isn't really true. There's not how humans work. For a variety of reasons, Plenty stick with their incorrect model despite the world indicating otherwise. In fact, this seems to be normal enough human behaviour. Everyone does it, for something or the other. You are no exception.
Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - https://arxiv.org/abs/2305.14975
It is a very basic fact that LLMs have no concept of true or false, it only has an ability to look up what text data it has seen before. If you do not understand this you are in no position to discuss LLMs.
I really don't know what people mean when they say this. We routinely instruct computer chips to evaluate whether some condition is true and take action on that basis, even though the chip is "just" a selectively doped rock. Why would the details of an LLM's underlying architecture mean that it can't have a concept of true or false?
Okay, so then tell me how does it decide whether it is true or false that Biden is the POTUS?
It's response is not based on facts about the world as it exists, but on the text data it has been trained on. As such, it is not able to determine true or false even if the response in the above example would be correct.
Serious question, in pursuit of understanding where you're coming from: in what way do you think that your own reckoning is fundamentally different to or more "real" than what you're describing above?
I know I don't experience the world as it is, but rather through a whole bunch of different signals I get that give me some hints about what the real world might be. For example, text.
You can say the difference is academic but there is a difference.
What is the difference between a real good faker of intelligence and actual intelligence is an open question.
But I will say most AI experts agree that LLM are not artificial general intelligence. It isn't just a lack of training data, they just are not of the category that we mean by that.
GPT-4 can explain the concept when prompted and can evaluate logic problems better than most human beings can. I would say it has a deeper understanding of "true vs false" than most humans.
I think what you are trying to say is that LLMs are not conscious. Consciousness has no precise universally agreed formal definition, but we all know that LLMs are not conscious.
> GPT-4 can explain the concept when prompted and can evaluate logic problems better than most human beings can. I would say it has a deeper understanding of "true vs false" than most humans.
Sigh
GPT produces output which obeys the patterns it has been trained on for definitions of true and false. It does not understand anything. It is a token manipulation machine. It does it well enough that it convinces you, a walking ape, that it understands. It does not.
A human is an ape that is obeying patterns that it has been trained on. What is school but a bunch of apes being trained to obey patterns? Some of these apes do well enough to convince you that it understands things. Some apes fully "understand" that flat earth theory is true, or they "understand" that the Apollo moon landings were faked.
You have a subjective philosophical disagreement about what constitutes understanding. That is fine. I clearly understand it is not conscious and that programs do not understand things the way that humans do. We are fundamentally different to LLMs. That is obvious. But you are not making a technical argument here unless you can define "understand" in technical terms. This is a matter of semantics.
> It is a token manipulation machine
Deep learning and machine learning in general is more than token manipulation. They are designed for pattern recognition.
You acknowledged above that consciousness isn't what LLM is and you likely understand that the poster was referring to that...
The broad strokes you use here are exactly why discussing LLMs are hard. Sure some people dismiss them because it isn't general AI but having supporters dismiss any argument with "passes the Turning test" is equally useless.
"But you are not making a technical argument here unless you can define "understand" in technical terms. This is a matter of semantics."
I said the nature of their argument is not technical, since they are not dealing with technical definitions, but I did not dismiss their argument altogether. I clarified and restated their own argument for them in clearer terms. LLMs are not conscious, but they can still "understand" very well depending on your definition of understand. Understanding is not a synonym for consciousness. Language is evolving and you need to be more precise when discussing AI / machine learning.
One definition of understand is:
"perceive the intended meaning of (words, a language, or a speaker)."
Deep learning models recognize patterns. Mechanical perception of patterns. They understand things mechanically, unconsciously.
I stand by my point that people using synonyms for consciousness being told "LLM knows true better than humans do" is bad for discussion.
The core issue is their "knowledge" is too context sensitive.
Certainly humans are very context sensitive in our memories but we all have something akin to a "mental model" we can use to find things without that context.
In contrast LLM has knowledge defined by that context quite literally.
In either case my original point on using true and false is that LLM can hallucinate and on a fundamental design level there is little that can be done to stop it.
LLMs can outperform humans on a variety of NLP tasks that require understanding. Formally, they are designed to solve "natural language understanding" tasks as a subset of "natural language processing" tasks. The word "understanding" is used in the academic context here. It is a standard term in NLP research.
My point was to show that their thinking, reasoning and language was flawed, that it lacked nuance and rigor. I am trying to raise the standards of discussion. They need to think more deeply about what "understanding" really means. Consciousness does not even have a formal universally agreed definition.
Sloppy non-rigorous shallow arguments are bad for discussion.
> LLM can hallucinate and on a fundamental design level there is little that can be done to stop it.
That's a separate issue. They generally don't hallucinate when solving a problem within their context window. Recalling facts from their training set is another issue.
Humans sometimes have a similar problem of "hallucinating" when recalling facts from their long term memory.
Narrow to a tiny training set? What are you talking about now? That has nothing to do with deep learning.
GPT-3.5 was trained on at least 300 billion tokens. It has 96 layers in its neural network of 175 billion parameters. Each one of those 96 stacked layers has an attention mechanism that recomputes an attention score for every token in the context window, for each new token generated in sequence. GPT-4 is much bigger than that. The scale and complexity of these models is beyond comprehension. We're talking about LLMs, not SLMs.
ChatGPT can tell me about itself when prompted. It tells me that it is an LLM. It can tell me about capabilities and limitations. It can describe the algorithms that generate itself. It has deep self knowledge, but is not conscious.
> It's response is not based on facts about the world as it exists, but on the text data it has been trained on
How did you find out that Biden was elected if not through language by reading or listening to news? Do you have extra sensory perception? Psychic powers? Do you magically perceive "facts" without any sensory input or communication? Ridiculous.
By the same argument your knowledge is also not based on "facts" about the world, since you only learned about it by reading or listening. Absurd nonsense.
I did answer your question indirectly. By the reasoning in your argument, you yourself also don't know true or false. Your argument is logically flawed.
Do LLMs know true or false? It depends on how you define "know". By some definitions, they "know true or false" better than humans, as they can explain the concept and solve logic problems better than most humans can. However, by any definition that requires consciousness, they do not know because they are not conscious.
The average person spends a lot of time completely immersed in "false" entertainment. Actors are all liars, pretending to be someone they are not, doing things that didn't really happen, and yet many people are convinced it is all "true" for at least a few minutes.
People also believe crazy things like Flat Earth theory or that the Apollo moon landings were faked.
So LLMs have a conceptual understanding of true/false, strong logical problem solving to evaluate truth or falsity of logical statements, and factual understanding of what is true and false, better than many humans do. But they are not conscious therefore they are not conscious of what is true or false.
It certainly doesn't "look up" text data it has seen before. That shows a fundamental misunderstanding of how this stuff works. That's exactly why I use the example above of Alpha Zero and how it learns to play Go, since that demonstrates very clearly that it's not just looking things up.
And I have no idea what you mean by saying that it has no concept of true or false. Even the simplest computer programs have a concept of true or false, that's kind of the simplest data type, a boolean. Large language models have a much more sophisticated concept of true and false that has a lot more nuance. That's really a pretty ridiculous thing to say.
Yes, you don't understand what I said. The model has no concept of true or false. It only has embeddings. If 'asked' a question it can see if that is consistent with its embeddings and probabilities or not. This is not a representation of the real world, of facts, but simply a product of its training.
They have no inherent concept of true or false, sure. But what are you comparing them to? It would be bold to propose that humans have some inherent concept of true or false in a way that LLMs do not; for both humans and LLMs it seems to be emergent.
In all these arguments its implied that this "genuine intelligence" is something humans all have, and nothing could be farther from the truth, that is why we have flat earthers or religious people and many other people beliving for decades easily refutable lies.
There is no such thing as a world model, and you don't have one of them. This is a leftover bad psychological concept from the 70s AI researchers who never got anywhere. People and other creatures do very little modeling things, they mostly just do stuff.
Wait, you actually think we are talking about modelling as a conscious deliberate process in active working memory? Well there's your fundamental mistake. That is not what we are discussing, not even remotely.
The vast model in your brain is learned and generated unconsciously without your direct awareness.
No, I didn't say anything about doing it consciously. Motion is largely unconscious, like how you can throw things at a target without thinking about it.
But if you're just using it to mean "factual memory", calling it modeling seems like false precision.
Oh well in that case the answer is straightforward.
If you close your eyes and get lost after a few seconds, that's because that aspect of your model was not a 100% perfect exact replica of external reality that extended infinitely far in all spatial directions at all resolutions. For example, your internal spatial model is limited to some degree of accuracy and does not include the entire surface of Mars, but that doesn't mean that your model does not exist at all. Models are not perfect by definition. I thought this would be obvious.
Why would you think any model has to be a perfect exact 1:1 representation of the entire universe?
The model of reality in your head is a simplification that serves a purpose. Arbitrarily closing your fully functioning eyes is not something your model generating hardware was evolutionarily optimized for. Natural selection weeds out that kind of behaviour.
If you become blind then your model will change and optimize for other sensory inputs. Think of a blind man with a cane.
> For example, your internal spatial model is limited to some degree of accuracy and does not include the entire surface of Mars, but that doesn't mean that your model does not exist at all.
You're using "your model" as a metaphorical term here, but if you came up with any precise definition of the term here, it'd turn out to be wrong; people have tried this since the 50s and never gotten it correct. (For instance, is it actually a singular "a model" or is it different disconnected things you're using a single name for?)
- You may not have observed something in the room in the right way for the action you need to do later.
- You might have observed it in a way you don't need later, which is a waste of time and energy.
- It might change while you're not looking.
- You might just forget it. (Since people do this, this must be an adaptive behavior - "natural selection" - but it's not a good thing in a model.)
> Why would you think any model has to be a perfect exact 1:1 representation of the entire universe?
What principle can you use to decide how precise it should be? (You can't do this; there isn't one.)
> The model of reality in your head is a simplification that serves a purpose.
Not only does it serve a purpose, your observations largely don't exist until you have a purpose for them.
RL agents tend to get stuck investigating irrelevant things when they try to maintain models; humans are built to actively avoid this with attention and boredom. Robot cameras take in their entire visual field and try to interpret it; humans both consciously and unconsciously actively investigate the environment as needed alongside deciding what to do. (Your vision is mostly fake; your eyes are rapidly moving around to update it only after you unconsciously pay attention to something.)
> Natural selection weeds out that kind of behaviour.
Not that well since something like half of Americans are myopic…
> What principle can you use to decide how precise it should be?
It is not up to me or anyone else to decide. Our subjective definitions and concepts of the model are irrelevant. How the brain works is a result of our genetic structure. We don't have a choice.
All of this was in response to your comment earlier:
"There is no such thing as a world model, and you don't have one of them."
There is such a thing as a world model in humans, and we all have them otherwise we could not think about or conceptualize or navigate the world. Then you have discussed how to define or construct a useful model or the limitations of a model but that is not relevant to the original point and I'm already aware of that.
I do agree, but more importantly love this part of the argument!
Its when all the personality differences become too much to bear and suddenly people are accused of not even knowing themselves.
Been there before, what a wild ride!
> suddenly people are accused of not even knowing themselves
It's not some desperate retort. People don't know themselves very well. Look at the research into confabulation, it seems to be standard operating procedure for human brains.
No human or creature we know of has a "true" world model so this is irrelevant. You don't experience the "real world". You experience a tiny slice of it, a few senses that is further slimmed down and even fabricated at parts.
To the bird who can intuitively sense and use electromagnetic waves for motion and guidance, your model of the world is fundamentally incomplete.
There is a projection of the world in text. Moreover training on additional modalities is trivial for a transformer. That's all that matters.