> There we go again. Claim: compression <something something> intelligence.
I don't see how this could possibly be controversial. Do you agree or disagree that, on a long enough timeline, intelligence lets you build a model of a system that can accurately and precisely reproduce all observations without simply remembering and regurgitating those observations? The model will always be smaller than simply remembering those observations, ergo, the intelligence maximally compressed the observations.
> Evidence: 34.75% on ARC AGI.
With no pretraining. Literally the only thing it's seen is 3 examples. And it does it with a very reasonable timeframe. I don't think you appreciate how impressive this is.
>> Do you agree or disagree that, on a long enough timeline, intelligence lets you build a model of a system that can accurately and precisely reproduce all observations without simply remembering and regurgitating those observations?
Why should I agree or disagree with that? Who says that? Is it just some arbitrary theory that you came up with on the spot?
>> The model will always be smaller than simply remembering those observations, ergo, the intelligence maximally compressed the observations.
"Maximally compressed the observations" does not follow from the model being smaller. You're missing an assumption of optimality. Even with that assumption, there is nothing in the work above to suggest that compression has something to do with intelligence.
>> I don't think you appreciate how impressive this is.
I do and it's not. 20% on unseen tasks. It only works on tasks with a very specific structure, where the solution is simply to reproduce the input with some variation, without having to capture any higher order concepts. With enough time and compute it should be possible to do even better than that by applying random permutations to the input. The result is insignificant and a waste of time.
> Why should I agree or disagree with that? Who says that? Is it just some arbitrary theory that you came up with on the spot?
Intelligence has some capabilities. I'm asking you if the capabilities of intelligence encompass the ability described, yes or no?
> "Maximally compressed the observations" does not follow from the model being smaller.
No other function can be smaller than the actual function that generates the observations. Correct answers indicates that "intelligence" reproduced the model in order to make a successful prediction, ergo "maximal compression".
>> No other function can be smaller than the actual function that generates the observations.
You mean if I write a shitty program that takes the string "hello", copies it 10 million times and stores the copies, and then generates a single copy any time I run it, there's no algorithm that can achieve a better compression than that because "no other function can be smaller than the actual function that generates the observations"?
I suggest, before coming up with fancy theories about compression and intelligence, that you read a bit about computer science, first. For instance, read up on information theory, and maybe about Kolmogorov complexity, the abstractions that people who swear that "compression <something something> intelligence" are basing their ideas on.
I'm very familiar with Solomonoff Induction, thanks. Are you going to answer my question or make more bad faith tangents to avoid committing to an answer?
An answer to what? To some half-baked theory you dreamed up just to sound deep and insightful about intelligence? If you think you have some insight about intelligence go do something useful with it, I don't know, try to publish a paper.
Or, you know what? I've had it up to here with all the wild-ass swivel-eyed loon theories about what intelligence is and isn't. if you really think you know what intelligence is all about why don't you write it up as a program and run it on a computer, and show us all that your theory is right? Go ahead and create intelligence. Intelligence is compression? Go compress stuff and make it intelligent or whatever it is you think works.
Put your money where your mouth is. Until then don't accuse others of bad faith.
I don't see how this could possibly be controversial. Do you agree or disagree that, on a long enough timeline, intelligence lets you build a model of a system that can accurately and precisely reproduce all observations without simply remembering and regurgitating those observations? The model will always be smaller than simply remembering those observations, ergo, the intelligence maximally compressed the observations.
> Evidence: 34.75% on ARC AGI.
With no pretraining. Literally the only thing it's seen is 3 examples. And it does it with a very reasonable timeframe. I don't think you appreciate how impressive this is.