> If you can find a way to make the context window on the scale of the human brain, you may be able to mostly mitigate this.
Human brains have a much smaller context window than AI do. We can't pay attention to the last 128,000 concepts that filtered past our sensory systems — our conscious considerations are for about seven things.
There's a lot of stuff that we don't yet understand well enough to reproduce with AI, but context length is the wrong criticism for these models.
> context length is the wrong criticism for these models
You're right. What I'm getting at is the overall speed, efficiency, and accuracy of the storage, retrieval, and processing capability of the human brain.
> If you can find a way to make the context window on the scale of the human brain, you may be able to mostly mitigate this.
Human brains have a much smaller context window than AI do. We can't pay attention to the last 128,000 concepts that filtered past our sensory systems — our conscious considerations are for about seven things.
There's a lot of stuff that we don't yet understand well enough to reproduce with AI, but context length is the wrong criticism for these models.