I'm collecting data for how humans are solving ARC tasks, and so far collected 4100 interaction histories (https://github.com/neoneye/ARC-Interactive-History-Dataset). Besides ARC-AGI, there are other ARC like datasets, these can be tried in my editor (https://neoneye.github.io/arc/).

I have made some videos about ARC:

Replaying the interaction histories, and you can see people have different approaches. It's 100ms per interaction. IRL people doesn't solve task that fast. https://www.youtube.com/watch?v=vQt7UZsYooQ

When I'm manually solving an ARC task, it looks like this, and you can see I'm rather slow. https://www.youtube.com/watch?v=PRdFLRpC6dk

What is weird. The way that I implement a solver for a specific ARC task is much different than the way that I would manually solve the puzzle. Having to deal with all kinds of edge cases.

Huge thanks to the team behind the ARC Prize. Well done.

Seeing the examples while having the editor visible. That's a good idea. I haven't explored this direction, since I had my phone (with tiny screen estate) in mind.

Drafts for a such a UI are much welcome. However I'm probably too lazy to code it though.

The short story. I needed something that could render thumbnails of tasks, so I could visual debug what was going on in my solver. However I have never gotten around to make the visual inspection tool. After having the thumbnail renderer, mid january 2024, then it eventually turned into what it is now.

But the people involved in this haven't signaled that they are in that path, either in the message about the challenge (precisely the opposite) or seemingly in their careers so far

So I guess I don't share the concern but a better way to phrase your comment could be -

"how can we be sure the human-provided solutions won't turn out to be just fodder for training a RL model or something that will later be monetized, closed and proprietary? Do the challenge organizers provide any guarantees on that?"

If I can make one criticism/observation of the tests, it seems that most of them reason about perfect information in a game-theoretic sense. However, many if not most of the more challenging problems we encounter involve hidden information. Poker and negotiations are examples of problem solving in imperfect information scenarios. Smoothly navigating social situations also requires a related problem of working with hidden information.

One of the really interesting things we humans are able to do is to take the rules of a game and generate strategies. While we do have some algorithms which can "teach themselves" e.g. to play go or chess, those same self-play algorithms don't work on hidden information games. One of the really interesting capabilities of any generally-intelligent system would be synthesizing a general problem solver for those kinds of situations as well.

I swear, not enough people have kids.

Now, is it 10k examples? No, but I think it was on the order of hundreds, if not thousands.

One thing kids do is they'll ask for confirmation of their guess. You'll be reading a book you've read 50 times before and the kid will stop you, point at a dog in the book, and ask "dog?"

And there is a development phase where this happens a lot.

Also kids can get mad if they are told an object doesn't match up to the expected label, e.g. my son gets really mad if someone calls something by the wrong color.

Another thing toddlers like to do is play silly labeling games, which is different than calling something the wrong name on accident, instead this is done on purpose for fun. e.g. you point to a fish and say "isn't that a lovely llama!" at which point the kid will fall down giggling at how silly you are being.

The human brain develops really slowly[1], and a sense of linear time encoding doesn't really exist for quite awhile. (Even at 3, everything is either yesterday, today, or tomorrow) so who the hell knows how things are being processed, but what we do know is that kids gather information through a bunch of senses, that are operating at an absurd data collection rate 12-14 hours a day, with another 10-12 hours of downtime to process the information.

[1] Watch a baby discover they have a right foot. Then a few days later figure out they also have a left foot. Watch kids who are learning to stand develop a sense of "up above me" after they bonk their heads a few time on a table bottom. Kids only learn "fast" in the sense that they have nothing else to do for years on end.

I have kids so I'm presuming I'm allowed to have an opinion here.

This is ignoring the fact that babies are not just learning labels, they're learning the whole of language, motion planning, sensory processing, etc.

Once they have the basics down concept acquisition time shrinks rapidly and kids can easily learn their new favorite animal in as little as a single example.

Compare this to LLMs which can one-shot certain tasks, but only if they have essentially already memorized enough information to know about that task. It gives the illusion that these models are learning like children do, when in reality they are not even entirely capable of learning novel concepts.

Beyond just learning a new animal, humans are able to learn entirely new systems of reasoning in surprisingly few examples (though it does take quite a bit of time to process them). How many homework questions did your entire calc 1 class have? I'm guessing less than 100 and (hopefully) you successfully learned differential calculus.

Until they encounter a similar animal and get confused, at which point you understand the implicit heuristic they were relying on. (Eg. They confused a dairy cow as a zebra, which means their heuristic was a black-and-white quadrupedal)

Doesn't this seem remarkably close to how LLMs behave with one-shot or few-shot learning? I think there are a lot more similarities here than you give it credit for.

Also, I grew up in South Korea where early math education is highly prioritized (for better or for worse). I remember having to solve 2 dozen arithmetic problems every week after school with a private tutor. Yes, it was torture and I was miserable, but it did expose me to thousands more arithmetic questions than my American peers. All that misery paid off when I moved to the U.S. at the age of 12 and realized that my math level was 3-4 years above my peers. So yes, I think human intelligence accuracy also does improve with more training data.

Of course it didn't have to be this way - in a different language animals might be named based on size or abilities/behavior, etc.

So, your daughter wanting to label a cat-sized dog as a cat is just a reflection of her not having aligned her generalization of what you are talking about when you say "cat" vs "dog" with her own.

Not just that: people learn mathematics mainly by _thinking over and solving problems_, not by memorising solutions to problems. During my mathematics education I had to practice solving a lot of problems dissimilar what I had seen before. Even in the theory part, a lot of it was actually about filling in details in proofs and arguments, and reformulating challenging steps (by words or drawings). My notes on top of a mathematical textbook are much more than the text itself.

People think that knowledge lies in the texts themselves; it does not, it lies in what these texts relate to and the processes that they are part of, a lot of which are out in the real world and in our interactions. The original article is spot on that there is no AGI pathway in the current research direction. But there are huge incentives for ignoring this.

I think it's more accurate to say that they learn math by memorizing a sequence of steps that result in a correct solution, typically by following along with some examples. Hopefully they also remember why each step contributes to the answer as this aids recall and generalization.

The practice of solving problems that you describe is to ingrain/memorize those steps so you don't forget how to apply the procedure correctly. This is just standard training. Understanding the motivation of each step helps with that memorization, and also allows you to apply that step in novel problems.

> The original article is spot on that there is no AGI pathway in the current research direction.

I think you're wrong. The research on grokking shows that LLMs transition from memorization to generalized circuits for problem solving if trained enough, and parametric memory generalizes their operation to many more tasks.

They have now been able to achieve near perfect accuracy on comparison tasks, where GPT-4 is barely in the double digit success rate.

Composition tasks are still challenging, but parametric memory is a big step in the right direction for that too. Accurate comparitive and compositional reasoning sound tantalizingly close to AGI.

Simply memorizing sequences of steps is not how mathematics learning works, otherwise we would not see so much variation in outcomes. Me and Terence Tao on the same exact math training data would not yield two mathematicians of similar skill.

While it's true that memorization of properties, structure, operations and what should be applied when and where is involved, there is a much deeper component of knowing how these all relate to each other. Grasping their fundamental meaning and structure, and some people seem to be wired to be better at thinking about and picking out these subtle mathematical relations using just the description or based off of only a few examples (or be able to at all, where everyone else struggles).

> I think you're wrong. The research on grokking shows that LLMs transition from memorization to generalized circuits

It's worth noting that for composition, key to abstract reasoning, LLMs failed to generalize to out of domain examples on simple synthetic data.

From: https://arxiv.org/abs/2405.15071

> The levels of generalization also vary across reasoning types: when faced with out-of-distribution examples, transformers fail to systematically generalize for composition but succeed for comparison.

Everyone starts by memorizing how to do basic arithmetic on numbers, their multiplication tables and fractions. Only some then advance to understanding why those operations must work as they do.

> It's worth noting that for composition, key to abstract reasoning, LLMs failed to generalize to out of domain examples on simple synthetic data.

Yes, I acknowledged that when I said "Composition tasks are still challenging". Comparisons and composition are both key to abstract reasoning. Clearly parametric memory and grokking have shown a fairly dramatic improvement in comparative reasoning with only a small tweak.

There is no evidence to suggest that compositional reasoning would not also fall to yet another small tweak. Maybe it will require something more dramatic, but I wouldn't bet on it. This pattern of thinking humans are special does not have a good track record. Therefore, I find the original claim that I was responding to("there is no AGI pathway in the current research direction") completely unpersuasive.

Maybe schools teach by memorization, but my mom taught me by explaining what it means, and I highly recommend this approach (and am a proof by example that humans can learn this way).

How did you learn what the symbols for numbers mean and how addition works? Did you literally just see "1 + 3 = 4" one day and intuit the meaning of all of those symbols? Was it entirely obvious to you from the get-go that "addition" was the same as counting using your fingers which was also the same as counting apples which was also the same as these little squiggles on paper?

There's no escaping the fact that there's memorization happening at some level because that's the only way to establish a common language.

I still don't think you are. Since we agree that you memorized numbers and how they are sequential, and that counting is moving "up" in the sequence, addition as counting is still memorizing a procedure based on this, not just memorizing a name: to add any two numbers, count down on one as you count up on the other until the first number number reaches zero, and the number that counted up is the sum. I'm curious how you think you learned addition without memorizing this procedure (or one equivalent to it).

Then you memorized the procedure for multiplication: given any two numbers, count down on one and add the other to itself until the counted down number reaches one. This is still a procedure that you memorized under the label "multiplication".

This is exactly the kind of procedure that I initially described. Someone taught you a correct procedure for achieving some goal and gave you a name for it, and "learning math" consists of memorizing such correct procedures (valid moves in the game of math if you will). These moves get progressively more sophisticated as the math gets more advanced, but it's the same basic process.

They "make sense" to you, and you call it "understanding", because they are built on a deep foundation that ultimately grounds out in counting, but it's still memorizing procedures up and down the stack. You're just memorizing the "minimum" needed to reproduce everything else, and compression is understanding [1].

The "variation in outcomes" that an OP discussed is simply because many valid moves are possible in any given situation, just like in chess, and if you "understand" when a move is valid vs. not (eg. you remember it), then you have an advantage over someone who just memorized specific shortcuts, which I suspect is what you are thinking I mean by memorization.

[1] https://philpapers.org/rec/WILUAC-2

Current research in early mathematical education now focuses on teaching certain spatial skills to very young kids rather than (just) numbers. Mathematics is about understanding of relationships, and that is not a detached kind of understanding that we can make into an algorithm, but deeply invested and relational between the "subject" and the "object" of understanding. Taking the subject and all the relations with the world out of the context of learning processes is absurd, because that is in the exact centre of them.

I did memorize names of numbers, but that is not essential in any way to doing or understanding math, and I can remember a time where I understood addition but did not fully understand how names of numbers work (I remember, when I was six, playing with a friend at counting up high, and we came up with some ridiculous names for high numbers because we didn't understand decimal very well yet).

Addition is a thing you do on matchsticks, or fingers, or eggs, or whatever objects you're thinking about. It's merging two groups and then counting the resulting group. This is how I learned addition works (plus the invariant that you will get the same result no matter what kind of object you happen to work with). Counting up and down is one method that I learned, but I learned it by understanding how and why it obviously works, which means I had the ability to generate variants - instead of 2+8=3+7=... I can do 8+2=9+1=..., or I can add ten at a time, etc'.

Same goes for multiplication. I remember the very simple conversation where I was taught multiplication. "Mom, what is multiplication?" "It's addition again and again, for example 4x3 is 3+3+3". That's it, from that point on I understood (integer) multiplication, and could e.g. wonder myself at why people claim that xy=yx and convince myself that it makes sense, and explore and learn faster ways to calculate it while understanding how they fit in the world and what they mean. (An exception is long multiplication, which I was taught as a method one day and was simple enough that I could memorize it and it was many years before I was comfortable enough with math that whenever I did it it was obvious to me why what I'm doing here calculates exactly multiplication. Long division is a more complex method: it was taught to me twice by my parents, twice again in the slightly harder polynomial variant by university textbooks, and yet I still don't have it memorized because I never bothered to figure out how it works nor to practice enough that I understand it).

I never in my life had an ability to add 2+2 while not understanding what + means. I did for half an hour have the same for long division (kinda... I did understand what division means, just not how the method accomplishes it) and then forgot. All the math I remember, I was taught in the correct order.

edit: a good test for whether I understood a method or just memorized it would be, if there's a step I'm not sure I remember correctly, whether I can tell which variation has to be the correct one. For example, in long multiplication, if I remembered each line has to be indented one place more to the right or left but wasn't sure which, since I understand it, I can easily tell that it has to be the left because this accomplishes the goal of multiplying it by 10, which we need to do because we had x0 and treated it as x.

Perhaps that is how you learned math, but it is nothing like how I learned math. Memorizing steps does not help, I sucked at it. What works for me us understanding the steps and why we used them. Once I understood the process and why it worked, I was able to reason my way through it.

> The practice of solving problems that you describe is to ingrain/memorize those steps so you don't forget how to apply the procedure correctly.

Did you look at the types of problems presented by the ARC-AGO test? I don't see how memorization plays any role.

> They have now been able to achieve near perfect accuracy on comparison tasks, where GPT-4 is barely in the double digit success rate.

Then lets see how they do on the ARC test? While it is possible that generalized circuits can develop in Ls with enough training but I am pretty skeptical till we see results.

Memorization is literally how you learned arithmetic, multiplication tables and fractions. Everyone starts learning math by memorization, and only later start understanding why certain steps work. Some people don't advance to that point, and those that do become more adept at math.

I understood how to do arithmetic for numbers with multiple digits before I was taught a "procedure". Also, I am not even sure what you mean by "memorization is how you learned fractions". What is there to memorize?

What did you understand, exactly? You understood how to "count" using "numbers" that you also memorized? You intuitively understood that addition was counting up and subtraction was counting down, or did you memorize those words and what they meant in reference to counting?

> Also, I am not even sure what you mean by "memorization is how you learned fractions". What is there to memorize?

The procedure to add or subtract fractions by establishing a common denominator, for instance. The procedure for how numerators and denominators are multiplied or divided. I could go on.

I do have the single digit multiplication table memorized now, but there was a long time where that table had gaps and I would use my understanding of how numbers worked to to calculate the result rather than remembering it. That same process still occurs for double digit number.

Mathematics education, especially historically, has indeed leaned pretty heavily on memorization. That does mean thats the only way to learn math, or even a particularly good one. I personally think over reliance on memorization is part of why so many people think they hate math.

Sure, I did that plenty too, but that doesn't refute the point that memorization is core to understanding mathematics, it's just a specific kind of memorization that results maximal flexibility for minimal state retention. All you're claiming is that you memorized some core axioms/primitives and the procedures that operate on them, and then memorized how higher-level concepts are defined in terms of that core. I go into more detail of the specifics here:

https://news.ycombinator.com/item?id=40669585

I agree that this is a better way to memorize mathematics, eg. it's more parsimonious than memorizing lots of shortcuts. We call this type of memorizing "understanding" because it's arguably the most parsimonious approach, requiring the least memory, and machine learning has persuasively argued IMO that compression is understanding [1].

[1] https://philpapers.org/rec/WILUAC-2

The answer is that both humans and the model are capable of reasoning, but the model is more restricted in the reasoning that it can perform since it must conform to the dataset. This means the model is not allowed to invest tokens that do not immediately represent an answer but have to be derived on the way to the answer. Since these thinking tokens are not part of the dataset, the reasoning that the LLM can perform is constrained to the parts of the model that are not subject to the straight jacket of training loss. Therefore most of the reasoning occurs in-between the first and last layers and ends with the last layer, at which point the produced token must cross the training loss barrier. Tokens that invest into the future but are not in the dataset get rejected and thereby limit the ability of the LLM to reason.

And almost all of it is just more text, or described in more text.

You're very much right about this. And that's exactly why LLMs work as well as they do - they're trained on enough text of all kinds and topics, that they get to pick up on all kinds of patterns and relationships, big and small. The meaning of any word isn't embedded in the letters that make it, but in what other words and experiences are associated with it - and it so happens that it's exactly what language models are mapping.

I’m quite surprised at this guess and intrigued by your school’s methodology. I would have estimated >30 problems average across 20 weeks for myself.

My kids are still in pre-algebra, but they get way more drilling still, well over 1000 problems per semester once Zern, IReady, etc. are factored in. I believe it’s too much, but it does seem like the typical approach here in California.

For example after doing several hundred logarithms, I was eventually able to do logs to 2 decimal places in my head. (Sadly I cannot do that anymore!) I imagine if I had just done a dozen or so problems I would not have gained that ability.

Sure, but they learn a lot of labels.

> How many homework questions did your entire calc 1 class have? I'm guessing less than 100

At least 20 to 30 a week, for about 10 weeks of class. Some weeks were more, and I remember plenty of days where we had 20 problems assigned a day.

Indeed, I am a huge fan of "the best way to learn math is to do hundreds upon hundreds of problems", because IMHO some concepts just require massive amounts of repetition.

Now imagine how much would your kid learn if the only input he ever received was a sequence of words?

The difference is that we don't know better methods for them, but we do know of better methods for people.

I think it was; the guesstimate I've seen is GPT-4 was trained on 13e12 tokens, that over 4 years is 8.9e9/day or about 1e5/s.

Then it's a question of how many bits per token — my expectation is 100k/s is more than the number of token-equivalents we experience, even though it's much less than the bitrate even of just our ears let alone our eyes.

The analogies get a little blurry here, but perhaps we can draw a distinction between information that an infant gets from their higher-level senses (e.g. sight, smell, touch, etc) versus any lower-level biological processes (genetics, epi-genetics, developmental processes, and so on).

The main point is that there is a fundamental difference: LLMs have very little prior knowledge [1] while humans contain an immense amount of information even before they begin learning through the senses.

We need to look at the billions of years of biological evolution, millions of years of cultural evolution, and the immense amounts of environmental factors, all which shape us before birth and before any “learning” occurs.

[1] The model architecture probably counts as hard-coded prior knowledge contained before the model begins training, but it is a ridiculously small amount of information compared to the complexity of living organisms.

But after the training people are much more equipped to do single-shot recognition and cognitive tasks of imagery and situations they have not encountered before, e.g. identifying (from pictures) which animals is being shown, even if it is the second time of seeing that animal (the first being shown that this animal is a zebra).

So, basically, after initial training, I believe people are superior in single-shot tasks—and things are going to get much more interesting once LMMs (or something after that?) are able to do that well.

It might be that GPT-4o can actually do that task well! Someone should demo it, I don't have access. Except, of course, GPT-4o already knows what zebras look like, so something else than exactly that..

Yeah, but they're seeing mostly the same thing day after day!

They aren't seeing 10k stills of 10k different dogs, then 10k stills of 10k different cats. They're seeing $FOO thousand images of the family dog and the family cat.

My (now 4.5yo) toddler did reliably tell the difference between cats and dogs the first time he went with us to the local SPCA and saw cats and dogs that were not our cats and dogs.

In effect, 2 cats and 2 dogs were all he needed to reliably distinguish between cats and dogs.

I assume he was also exposed to many images, photos and videos (realistic or animated) of cats and dogs in children books and toys he handled. In our case, this was a significant source of animal recognition skills of my daughters.

No images or photos (no books).

TV, certainly, but I consider it unlikely that animals in the animation style of pepper pig helps the classifier.

Besides which, we're still talking under a dozen cats/dogs seen till that point.

Forget about cats/dogs. Here's another example: he only had to see a burger patty once to determine that it was an altogether new type of food, different from (for example) a sausage.

Anyone who has kids will have dozens of examples where the classifier worked without a false positive off a single novel item.

To continue your example, I know I've learned calculus and was lauded at the time. Now I could only give you the vagaries, nothing practical. However I know if I was pressed, I could learn it again in short order.

Yes. All that learning is feeding off one another. They're learning how reality works. Every bit of new information informs everything else. It's something that LLMs demonstrate too, so it shouldn't be a surprising observation.

> Once they have the basics down concept acquisition time shrinks rapidly

Sort of, kind of.

> and kids can easily learn their new favorite animal in as little as a single example.

Under 5 they don't. Can't speak what happens later, as my oldest kid just had their 5th birthday. But below 5, all I've seen is kids being quick to remember a name, but taking quite a bit longer to actually distinguish between a new animal and similarly looking ones they already know. It takes a while to update the classifier :).

(And no, they aren't going to one-shot recognize an animal in a zoo that they saw first time on a picture hours earlier; it's a case I've seen brought up, and I maintain that even most adults will fail spectacularly at this test.)

> Compare this to LLMs which can one-shot certain tasks, but only if they have essentially already memorized enough information to know about that task. It gives the illusion that these models are learning like children do, when in reality they are not even entirely capable of learning novel concepts.

Correct, in the sense that the models don't update their weights while you use them. But that just means you have to compare them with ability of humans to one-shot tasks on the spot, "thinking on their feet", which for most tasks makes even adults look bad compared to GPT-4.

> How many homework questions did your entire calc 1 class have? I'm guessing less than 100 and (hopefully) you successfully learned differential calculus.

I don't believe someone could learn calc in 100 exercises or less. Per concept like "addition of small numbers", or "long division", or "basic derivatives", or "trivial integrals", yes. Note that in-class exercises count too; learning doesn't happen primarily by homework (mostly because few have enough time in a day to do it).

This simply is not true as stated in the article. ARC-AGI is a one-shot task test that humans reliably do much, much better on than any AI model.

> I don't believe someone could learn calc in 100 exercises or less.

I learned the basics of integration in a foreign language I barely understood by watching a couple of diagrams get drawn out and seeing far less than 100 examples or exercises.

Second that. I think I've learned as much as my children have.

> Watch a baby discover they have a right foot. Then a few days later figure out they also have a left foot.

Watching a baby's awareness grow from pretty much nothing to a fully developed ability to understand the world around is one of the most fascinating parts of being a parent.

This reminds of the story of Adam learning names, or how some languages can express a lot more in fewer words. And it makes sense that LLMs look intelligent to us.

My kid loves repeating the names of things he learned recently. For past few weeks, after learning 'spider' and 'snake' and 'dangerous' he keeps finding spiders around, no snakes so makes up snakes from curly drawn lines and tells us they are dangerous.

I think we learn fast because of stereo (3d) vision. I have no idea how these models learn and don't know if 3d vision will make multi model LLMs better and require exponentially less examples.

I think stereo vision is not that important if you can move around and get spatial clues that way also.

Of course for a human this can either mean "I have an idea about what a dog is, but I'm not sure whether this is one" or it can mean "Hey this is a... one of those, what's the word for it again?"

Babies need few examples for complex tasks because they get constant infinitely complex examples on tasks which are used for transfer learning.

Current models take a nuclear reactors worth of power to run back prop on top of a small countries GDP worth of hardware.

They are _not_ going to generalize to AGI because we can't afford to run them.

Nice one. Perhaps we are to conclude the whole transformer architecture is amazingly overblown in storage/computation costs.

AGI or not, we need better approach to what transformers are doing.

My friends toddler, who grew up with a cat in the house, would initially call all dogs "cat". :-D

The AI models aren't seeing the same image 1B times.

So? I'm still seeing the same object. Large models aren't trained on 10k different images of a single cat.

If I was presented with 10 pictures of 2 species I'm unfamiliar with, about as different as cats and dogs, I expect I would be able to classify further images as either, reasonably accurately.

She also saw an eagle this spring out the car window and said “an eagle! …no, it’s a bird,” so I guess she’s still working on those image classifications ;)

My child experiences the world in a really pure way. They don’t care much about labels or colours or any other human inventions like that. He picks up his carrot, he doesn’t care about the name or the color . He just enjoys it through purely experiencing eating it. He can also find incredible flow state like joy from playing with river stones or looking at the moon.

I personally feel bad I have to each them to label things and but things in boxes. I think your child is frustrated at times because it’s a punish of a game. The departure from “the oceanic feeling.

Your comment would make sense to me if the end game of our brains and human experience is labelling things. It’s not. It’s useful but it’s not what living is about.

The optimization process that trained the human brain is called evolution, and it took a lot more than 10,000 examples to produce a system that can differentiate cats vs dogs.

Put differently, an LLM is pre-trained with very light priors, starting almost from scratch, whereas a human brain is pre-loaded with extremely strong priors.

Asserted without evidence. We have essentially no idea at what point living systems were capable of differentiating cats from dogs (we don't even know for sure which living systems can do this).

A human brain that doesn't get visual stimulus at the critical age between 0 and 3 years old will never be able to tell the difference between a cat and a dog because it will be forevermore blind.

I heard a similar case before I did my A-levels, so at least 22 years ago, where the person had cateracts removed and it took a while to learn to see, something about having to touch a statue (of a monkey?) before being able to recognise monkeys?

A human that has never seen a dog or a cat could probably determine which is which based on looking at the two animals and their adaptations. This would be an interesting test for AIs, but I'm not quite sure how one would formulate a eval for this.

https://en.m.wikipedia.org/wiki/Bouba/kiki_effect

well, maybe. We view things in three dimensions at high fidelity: viewing a single dog or cat actually ends up being thousands of training samples, no?

He had only ever seen cats and dogs in his life previous to that.

Then train the AI using a binaural video of a thoroughbred and see if it can distinguish a draft horse and a quarter horse as horse...

Certainly kids learn and become better at extrapolation and need fewer and fewer samples in general as they get more life experience.

If you kept training LLMs with all that data, it would be interesting to see what the results would be.

Tho I only ever did undergrad stats, maybe ML isn’t even technically a linear regression at this point. Still, hopefully my gist is clear

This isn't accurate comparison imo, because we're mapping language to a world model which was built through a ton of trial and error.

Children aren't understanding language at six months old, there seems to be a minimum amount of experience with physics and the world before language can click for them.

Chomsky's arguments about "poverty of the stimulus" rely on using non-probabistic grammars. Norvig discusses this here: https://norvig.com/chomsky.html

> In 1967, Gold's Theorem showed some theoretical limitations of logical deduction on formal mathematical languages. But this result has nothing to do with the task faced by learners of natural language. In any event, by 1969 we knew that probabilistic inference (over probabilistic context-free grammars) is not subject to those limitations (Horning showed that learning of PCFGs is possible).

If I recall correctly, human toddlers hear about 3-13 million spoken words per year, and the higher ranges are correlated with better performance in school. Which:

- Is a lot, in an absolute sense.

- But is still much less training data than LLMs require.

Adult learners moving between English and romance languages can get a pretty decent grasp of the language (C1 or C2 reading ability) with about 3 million words of reading. Which is obviously exploiting transfer learning and prior knowledge, because it's harder in a less related language.

So yeah, humans are impressive. But Chomsky doesn't really seem to have the theoretical toolkit to deal with probabilistic or statistical learning. And LLMs are closer to statistical learning than to Chomsky's formal models.

https://youtu.be/UakqL6Pj9xo?si=iDH6iSNyz1Net8j7

I vaguely remember hearing that there's even ways to expand training data like that for neural networks, i.e. by presenting the same source image slightly rotated, partially obscured etc.

I think that "whatever we do" is doing a lot of heavy lifting here. Some of those "whatevers" will be isomorphic to a frame-level analysis that pulls out structural commonalities, or close enough that it's not a clunky reductionist analogy.

ML models are starting from absolute zero, single celled organism level.

Neither do machines. Lookup few-shot learning with things like CLIP.

Humans learn through a lifetime.

Or are we talking about newborn infants?

Would an intelligent but blind human be able to solve these problems?

I'm worried that we will need more than 800 examples to solve these problems, not because the abstract reasoning is so difficult, but because the problems require spatial knowledge that we intelligent humans learn with far more than 800 training examples.

Yann LeCun argues that humans are not general intelligence and that such a thing doesn't really exist. Intelligence can only be measured in specific domains. To the extent that this test represents a domain where humans greatly outperform AI, it's a useful test. We need more tests like that, because AIs are acing all of our regular tests despite being obviously less capable than humans in many domains.

> the problems require spatial knowledge that we intelligent humans learn with far more than 800 training examples.

Pretraining on unlimited amounts of data is fair game. Generalizing from readily available data to the test tasks is exactly what humans are doing.

> Would an intelligent but blind human be able to solve these problems?

I'm confident that they would, given a translation of the colors to tactile sensation. Blind humans still understand spatial relationships.

I don't think there's any rules about what knowledge/experience you build into your solution.

Unless you have strong prior beliefs (like "computers can't be AGI") or something else that's problem specific ("these problems can be solved by these techniques which don't count as AGI"). So I guess that's my real question.

* How likely you think AGI is in general.

* How solvable you think the problem is, independently of what's solving it.

In the cases you've brought up that latter probability is very high, which means that they are extremely weak evidence that computers are AGI. So we agree!

In this case the latter probability seems to be quite low - attempts to solve it with computers have largely failed so far!

In real life, when people say "A is evidence of B" they mean strong evidence, or even overwhelming evidence. You just backpedalled by redefining evidence to mean anything and nothing, so you can salvage an obviously false claim.

Nobody in the real world says "rain is evidence of aliens" with the implicit assumption that it's just extremely weak evidence. The way English is used by people makes that sentence simply false, as is yours that anything previously not solved is evidence of AGI.

Edit: I think maybe the disagreement here is about the nature of evidence. I think there can be evidence that something is AGI even if it isn't, in fact, AGI. You seem to believe that if there's any evidence that something is AGI, it must be AGI, I think?

No.

Because there might undiscovered ways to solve these problems that no one claims is AGI.

The definition of AGI is notoriously fuzzy, but non-the-less if there was a 10 line python program (with no external dependencies or data) that could solve it then few would argue that was AGI.

So perhaps there is an algorithm that solves these puzzles 100% of the time and can be easily expressed.

So I agree that only being able to solve these problems doesn't define AGI.

1. Only humans are known to have solved problem X, and we've spent no time looking for alternative solutions.

2. Only humans are known to have solved problem X, and we've spent hundreds of thousands of hours looking for alternative solutions and failed.

Now suppose something solves the problem. I feel like in case 2 we are justified in saying there's evidence that something is a human-like AGI. In case 1 we probably aren't justified in saying that.

To me this seems evident regardless of what the problem actually is! Because if it's hard enough that thousands of human hours cannot find a simple/algorithmic solution it's probably something like an "AGI-complete" problem?

To be clear, I think we have AGI (LLMs with tool use are generalized enough) and we are currently finding edge cases that they fail at.

I guess the underlying issue with my argument is that we really have no idea how large the search space is for finding AGI, so applying something like Bayes theorem (which is basically my argument) tells you more about my priors than reality.

That said, we know that human AGI was a result of an optimisation process (natural selection), and we have rudimentary generic optimisers these days (deep neural nets), so you could argue we've narrowed the search space a lot since the days of symbolic/tree search AI.

I don't think this is obviously correct.

Three things:

1) Many actions we think of as "intelligence" are just short-cuts based on heuristics.

2) While there's probably an argument that problem solving is selected for it's not clear to me how far this goes at all. There's little evidence that smarter people end up in more powerful positions for example. Seems like there is perhaps there is a cut-off beyond which intelligence is just a side effect of the problem solving ability that is useful.

3) Perhaps humans individually aren't (very?) intelligent and it is only a society of humans that are.

(also perhaps human GI? Nothing artificial about it.)

> no idea how large the search space is for finding AGI, so applying something like Bayes theorem (which is basically my argument) tells you more about my priors than reality.

There are plenty of imaginable forms of intelligence that are often ignored during these conversations. One in common use is "an intelligent footballer" which applies to sport for someone who can read a game well. There are other, non-human examples too (Dolphins, crows, parrots etc).

And then in the world of speculative fiction there's a range of different types of intelligence. Vernor Vinge wrote about intelligences which had motivations that people couldn't comprehend (and Vinge is generally credited with the concept of the singularity). More recently Peter Watt's Blindside contemplates the separation of intelligence and sentience.

Basically I don't think your expression of Bayes' theorem had nearly enough possibilities in it.

Evolution hasn't had enough time to adapt us to our new fangled lifestyle of last few hundred years, or few thousand for that matter, and anyways in the modern world people are not generally competing on things affecting survival, but rather on cultural factors that affect number of children we have.

Humans and most (all?) intelligent animals are generalists, which is why we need a big brain and intelligence - to rapidly adapt to a wide variety of ever changing circumstances. Non-generalists such as herbivores, crocodiles don't need intelligence and therefore don't have it.

The main thing that we need to survive & thrive as generalists - and what evolution has evidentially selected for - is ability to predict so that we can plan ahead and utilize past experience. Where will the food be, where will the water be in a drought, etc. I think active reasoning (not just LLM-like prediction/recall) would also play a large role in survival, and presumably parts of our brain have evolved specifically to support that, even if the CEO probably got his job based more on height/looks and golf handicap.

But the point has previously been made else humans developed large brains long (1.5M years?) before agriculture, and for a long time the only benefit seemed to be fire and flint tools.

It's not widely understood the causal link here - there are other species that have large brains but haven't developed these skills. So it's not clear exactly what facets of intelligence are selected for.

Lol, thanks, that's quite funny. I should spend less time on the internet.

> While there's probably an argument that problem solving is selected for it's not clear to me how far this goes at all.

Yeah, I meant something much more low brow which is that _humans_, with all of our properties (including GI), are a result of natural selection. I'm not claiming GI was selected for specifically, but it certainly occurred as a side-effect either way. So we know optimisation can work.

> There are plenty of imaginable forms of intelligence that are often ignored during these conversations.

I completely agree! I wish there was more discussion on intelligence in the broad in these threads. Even if you insist on sticking to humans it's pretty clear that something like a company or a government is operating very intelligently in its own environment (business, or politics), well beyond the influence of its individual constituents.

> Basically I don't think your expression of Bayes' theorem had nearly enough possibilities in it.

Another issue with Bayes in general is that you have a fixed probability space in mind when you use it, right? I can use Bayes to optimise my beliefs against a fixed ontology, but it says nothing about how or when to update the ontology itself.

And no doubt my ontology is lacking when it comes to (A)GI...

That seems a pretty extreme position!

What's your definition of AGI ?

Not really.

Jeremy Howard has said the same thing for example.

> What's your definition of AGI ?

Things that we consider intelligent when humans do them.

Basically we had all these definitions of AGI that we have surpassed (Turing test etc). Now we are finding more edge cases where we go "ahh... it can't do this so therefore it isn't intelligent".

But the issue with that is that lots of humans can't do them either.

I think the ARC challenge is valid. But I'd also point out that there are substantial numbers of people who won't be able to solve them either (blind people for example, as well as people who aren't good at puzzles). We make excuses there ("oh we can explain it to a blind person" or for many physical problems things like "Oh Stephen Hawking couldn't solve this but that is an exception") but we don't allow the same excuses for machine intelligence.

I don't think the boundary of AGI is a hard line, but if you went back 10 years and took what we had now and showed it to them I think people would be "Oh wow you have AI!".

LLMs do have a broad range of abilities, so not narrow AI, but clearly it's not general intelligence (or at least not human level), else they would not be failing or struggling on things that to us are easy - general means universal (not confined to specific types of problem), not just multi-capability.

The lack of reasoning ability, especially since it is architecturally based, seems more than a matter of patching up corner cases that aren't handled well. This shoring up of areas of weakness by increasing model size, adding targeted synthetic data and post-training is mostly just addressing static inference, much like adding more and more rules to CYC.

To make an LLM capable of reasoning it needs to go beyond a fixed N-layers of compute and support open-ended exploration, and probably replace gradient descent with a learning mechanism that can also be used at inference time. In a recent interview John Schulman (one of the OpenAI co-founders) indicated that they hoped that RL training on reasoning would improve it, but that is still going to be architecturally limited. You can learn a repertoire of reasoning templates than can be applied in gestalt fashion, but that's not the same as being able to synthesize a solution to a novel problem on the fly.

LLMs are certainly amazing, and as you say 10-years ago we would have regarded them as AI, but of course the same was true of expert systems and other techniques - we call things we don't know how to do "AI" then relabel them once we move past them to new challenges. Just as we no longer regard expert systems as AI, I doubt in 20 years we'll regard LLMs (which in some regards are also very close to expert systems) as AI, certainly not AGI. AGI will be the technology than can replace humans in many jobs, and when we get there LLMs will in hindsight look very limited.

Testing whether an AI can play chess or solve Chollet's ARC problems, or some other set of narrow skills, doesn't prove generality. If you want to test for generality, then you either have to:

1) Have a huge and very broad test suite, covering as many diverse human-level skills as possible.

and/or,

2) Reductively understand what human intelligence is, and what combination of capabilities it provides, then test for all of those capabilities both individually and in combination.

As Chollet notes, a crucial part of any AGI test is solving novel problems that are not just templated versions (or shallow combinatins) of things the wanna-be AGI has been trained on, so for both of above tests this is key.

AGI can add 1+1 correctly, but an ability to do that is not a test for AGI.

"Absence of evidence is evidence of absence."

Presumably you would call this a simple logical fallacy for the same reason, but a little reflection would show that in many cases such a statement is true! It depends on context, in this case your estimate of how well your search covered the possible search space.

Evidence is a continuous variable - things can be weak evidence, strong evidence... There's a whole spectrum. I just take issue with statements like "X is zero evidence of Y" because often you can do a lot better than that with the information at hand.

So, just because a human can't do something, or struggles to do it, doesn't mean that the task requires a huge IQ or generality - it may just require a lot of compute/memory, such as DeepBlue playing chess.

In the case in point of these ARC puzzles, they are easy for a human, so "absense of evidence" doesn't even apply, and it's worth noting that one could also brute force solve them by trying all applicable solution techniques (as indicated by the examples and challenge description) in combinatorial fashion, or just (as Chollet notes) generate a massive training set and train an LLM on it, and solve them via recall rather than active inference, which again proves nothing about AGI.

The point of the ARC challenge is to encourage advances in active inference (i.e. reasoning/problem solving), which is what LLMs lack. It's HOW you solve them that matters if you want to show general intelligence. Even in the realm of static inference, which is what they are built for, LLMs are really closer to DeepBlue than something intelligent - they brute force extract the training set rules using gradient descent. The interesting thing is that they have any learning ability at all (in-context learning) at inference time, but it's clearly no match for a human and they are also architecturally missing all the machinery such as working memory and looping/iteration to perform any meaningful try/fail/backtrack/try-again (while learning the whole time) active inference.

It'll be interesting to see to what extent pre-trained transformers can be combined with other components (maybe some sort of DeepBlue/AlphaGo MCTS?) to get closer towards human-level problem solving ability, but IMO it's really the wrong architecture. We need to stop using gradient descent and find a learning algorithm that can be used at inference time too.

But in general I agree about active inference. Clearly there is something missing there.

Doing alpha-go style MCTS would be interesting but how would you approach training the policy and value net? It's not like we can take snapshots of people's thought processes as they read text in the same way you can perform arbitrary rollouts of your game engine.

To OP: I like your project goal. I think you should look at prior, reasoning engines that tried to build common sense. Cyc and OpenMind are examples. You also might find use for the list of AGI goals in Section 2 of this paper:

https://arxiv.org/pdf/2308.04445

When studying intros of brain function, I also noted many regions tie into the hippocampus which might do both sense-neutral storage of concepts and make inner models (or approximations) of external world. The former helps tie concepts together through various senses. The latter helps in planning when we are imagining possibilities to evaluate and iterate on them.

Seems like AGI should have these hippocampus-like traits and those in the Cyc paper. One could test if an architecture could do such things in theory or on a small scale. It shouldn’t tie into just one type of sensory input either. At least two with the ability to act on what only exists in one or what is in both.

Edit: Children also have an enormous amount of unsupervised training on visual and spatial data. They get reinforcement through play and supervised training by parents. A realistic benchmark might similarly require GB of prettaining.

A similar vintage GOFAI project that might do better on these, with a suitable visual front end, is SOAR - a general purpose problem solver.

LLM’s are unsupervised, use probabilities with unpredictable results, and don’t explain every step of their thinking. They’re the opposite.

You might argue Cyc was. It was also more complex than any expert system I had ever seen. We just called stuff like that a reasoning engine or just Cyc to avoid confusion.

The rules (some prefer to call it a world model) in an LLM are deduced, via gradient descent, from the training samples, but are still there. The transformations effected by each layer of a transformer are exactly those it has learnt - the rules it is applying.

As with CYC people seem to be hoping that some external scaffolding (better inference engine(s)) will rescue LLMs from just being a set of rules to something more general and capable, but I tend to agree with Chollet that this active inference (reasoning) is actually the hard part.

There are two countries both which lay claim to the same territory. There is a set X that contains Y and there is a set Z that contains Y. In the case that the common overlap is 3D and one in on top of the other, we can extend this to there is a set X that contains -Y and a set Z that contains Y, and just as you can only see one on top and not both depending on where you stand, we can apply the same property here and say set X and Z cannot both exist, and therefore if set X is on then -Y and if set Z then Y.

If you pay attention to the language you use youll start to realize how much of it uses spatial relationships to describe completely abstract things. For example, one can speak of disintigrating hegonomic economies. i.e turning things built on top of eachother into nothing, to where it came

We are after all, reasoning about things which happen in time and space.

And spatial != visual. Even if you were blind youd have to reason spatially, because again any set of facts are facts in space-time. What does it take to understand history? People in space, living at various distances from each other, producing goods from various locations of the earth using physical processes, and physically exchanging them. To understand battles you have to understand how armies are arranged physically, how moving supplies works, weather conditions, how weapons and their physical forms affect what they can physically do, etc.

Hell LLMs, the largest advancement we had in artificial intelligence do what exactly? Encode tokens into multi dimensional space.

Is there a number of dimensions that captures all reasoning? I don't know..

This is the wrong way to think about it IMO. Spatial relationships are just another type of logical relationship and we should expect AGI to be able to analyze relationships and generate algorithms on the fly to solve problems.

Just because humans can be biased in various ways doesn’t mean these biases are inherent to all intelligences.

Not really. By that reasoning, 5-dimensional spatial reasoning is "just another type of logical relationship" and yet humans mostly can't do that at all.

It's clear that we have incredibly specialized capabilities for dealing with two- and three-dimensional spatiality that don't have much of anything to do with general logical intelligence at all.

It's important that we try to think from the perspective of an algorithm, not a human. And it's also important that we don't jump to extremes.

It seems like you interpreted "solving problems on the fly" to mean "instantly being an expert on a completely different and novel domain". What it does mean is flexibility, resilience to novel situations, and being able to adapt over time.

How about some aliens in a SF book. When we reason about them, where are they exactly? Literally on the pages of the book?

How about a context-free grammar?

I mean what problems does physics solve not with just complex number but with even more complex vectors? Problems of...space-time.

Aliens in a SF book. What do you imagine? I see some kind of physical entity having geometric compomnents in some kind of space.

Context free grammers are represented by...trees where one side of a spatial relationship maps to one idea and the other to another. What is context? Things surrounding something, where something is.

Come up with any idea, it can be represented in space and time.

It’s similar to how chess problems are technically reasoning problems but they are not representative of general reasoning.

Blind people can have spatial reasoning just fine. Visual =/= spatial [0]. Now, one would have to adapt the colour-based tasks to something that would be more meaningful for a blind person, I guess.

[0] https://hal.science/hal-03373840/document

There may (almost certainly will be) additional knowledge encoded in the solver to cover the spacial concepts etc. The distinction with the AGI-ARC test is the disparity between human and AI performance, and that it focuses on puzzles that are easier for humans.

It would be interesting to see a finetuned LLM just try and express the rule for each puzzle as english. It could have full knowledge of what ARC-AGI is and how the tests operate, but the proof of the pudding is simply how it does on the test set.

In it they question the ease of Chollet's tests: "One limitation on ARC’s usefulness for AI research is that it might be too challenging. Many of the tasks in Chollet’s corpus are difficult even for humans, and the corpus as a whole might be sufficiently difficult for machines that it does not reveal real progress on machine acquisition of core knowledge."

ConceptARC is designed to be easier, but then also has to filter ~15% of its own test takers for "[failing] at solving two or more minimal tasks... or they provided empty or nonsensical explanations for their solutions"

After this filtering, ConceptARC finds another 10-15% failure rate amongst humans on the main corpus questions, so they're seeing maybe 25-30% unable to solve these simpler questions meant to test for "AGI".

ConceptARC's main results show CG4 scoring well below the filtered humans, which would agree with a [Mensa] test result that its IQ=85.

Chollet and Mitchell could instead stratify their human groups to estimate IQ then compare with the Mensa measures and see if e.g. Claude3@IQ=100 compares with their ARC scores for their average human

[ConceptArc]https://arxiv.org/pdf/2305.07141 [Mensa]https://www.maximumtruth.org/p/ais-ranked-by-iq-ai-passes-10...

> We found that humans were able to infer the underlying program and generate the correct test output for a novel test input example, with an average of 84% of tasks solved per participant

And I followed the second example. This was my solution:

GRG

OBO

RGR

B is the cyan like blue color. My solution looks right, but it says it’s wrong.

The grid size is part of the pattern in the same way that the colors are part of the pattern. It’s not just a color pattern, it’s a generalized mapping of input to output.

In short: you need to resize the grid because that’s what the examples do.

For two reasons:

1. The initially suggested grid size was 3x3.

2. Filling in a 3x3 grid is sufficient to show that you understood the pattern, but filling in a 1x1 (or even 2x2) grid is insufficient.

Requiring the user fill in a larger grid is a waste of time. The existence of the grid size selector would still make sense in cases where a 2x2 grid would be sufficient to show the solution, so it is not obvious at all that a 6x6 grid should be chosen.

> The grid size is part of the pattern in the same way that the colors are part of the pattern.

To understand a pattern, you have to see at least two valid inputs and corresponding outputs. For the first example, a valid example for the expected output grid size is missing.

I arrived at the "correct" conclusion eventually, but the only indicator was that the reading direction for the UI was absolutely ridiculous ( https://i.imgur.com/CuQ2z2N.png ), suggesting that the authors did not think this through properly, so the solution had to be weird as well.

This is IQ tests all over again. Actually testing how alike you think to the author of the test.

Not to mention that ignoring the size of the grid, one might disagree about the answer of one of the tests.

This makes me think of "math" problems requiring you to find the next number in a series. They give you 5 numbers, and ask for the 6th. When I can build a polynomial than can generate the first 5 and any 6th number. Any.

Sounds like the point of these exercises it to guess what the author had in mind, more than some universal intelligence test. Though of course the author thinks their own thoughts are the measure of universal intelligence. It's a tempting thing to believe.

I guess there might be a disagreement of whether the problems in ARC are a representative sample of all of the possible abstract programs which could be synthesized, but then again most LLMs are also trained on human data.

Maybe if you run into some exceptionally difficult tasks it might not be 100%, but there's no way the challenge can be called unfair because it's too difficult for humans too.

Game on for the million, if so :). If not, apologies for distracting from the good fight for OSS/noncorp devs!

E: it occurred to me on the drive home how easily we (engineers) can fall into competitiveness, even when we’ve all read the thinkpieces about why an AI Race would/will be/is incredibly dangerous. Maybe not “game on”, perhaps… “god I hope it’s impossible but best of luck anyway to both of us”?

I'd also urge you to use a different platform for communicating with the public because x.com links are now inaccessible without creating an account.

"Endow circuitry with consciousness and win a gift certificate for Denny's (may not be used in conjunction with other specials)"

We are also trialing a secondary leaderboard called ARC-AGI-Pub that imposes no limits or constraints. Not part of the prize today but could be in the future: https://arcprize.org/leaderboard

AGI will take much more than that to build, and once you have it, if all you can monetize it for is a million dollars, you must be doing something extremely wrong.

However, I do disagree that this problem represents “AGI”. It’s just a different dataset than what we’ve seen with existing ML successes, but the approaches are generally similar to what’s come before. It could be that some truly novel breakthrough which is AGI solves the problem set, but I don’t think solving the problem set is a guaranteed indicator of AGI.

Imo there's no evidence whatsoever that nailing this task will be true AGI - (e.g. able to write novel math proofs, ask insightful questions that nobody has thought of before, self-direct its own learning, read its own source code)

That's a stretch. This is a problem at which LLMs are bad. That does not imply it's a good measure of artificial general intelligence.

After working a few of the problems, I was wondering how many different transformation rules the problem generator has. Not very many, it seems. So the problem breaks down into extracting the set of transformation rules from the data, then applying them to new problems. The first part of that is hard. It's a feature extraction problem. The transformations seem to be applied rigidly, so once you have the transformation rules, and have selected the ones that work for all the input cases, application should be straightforward.

This seems to need explicit feature extraction, rather than the combined feature extraction and exploitation LLMs use. Has anyone extracted the rule set from the test cases yet?

The issue with that path is that the problems aren’t using a programmatic generator. The rule sets are anything a person could come up with. It might be as simple as “biggest object turns blue” but they can be much more complicated.

Additionally, the test set is private so it can’t be trained on or extracted from. It has rules that aren’t in the public sets.

[1] https://www.kaggle.com/competitions/abstraction-and-reasonin...

Defining intelligence as an efficiency of learning, after accounting for any explicit or implicit priors about the world, makes it much easier to understand why human intelligence is so impressive.

https://youtu.be/UakqL6Pj9xo

What about Theory of Mind which talks about the problem of multiple agents in the real world acting together? Like driving a car cannot be done right now without oodles of data or any robot - human problem that requires the robot to model human's goals and intentions.

I think the problem is definition of general intelligence: Intelligence in the context of what? How much effort(kwh, $$ etc) is the human willing to amortize over the learning cycle of a machine to teach it what it needs to do and how that relates to a personally needed outcome( like build me a sandwich or construct a house)? Hopefully this should decrease over time.

I believe the answer is that the only intelligence that really matters is Human-AI cooperative intelligence and our goals and whether a machine understands them. The problems then need to be framed as optimization of a multi attribute goal with the attribute weights adjusted as one learns from the human.

I know a few labs working on this, one is in ASU(Kambhampati, Rao et. al) and possibly Google and now maybe open ai.

Take for example a simple audiotory pattern like "clap clap clap". This has a very trival mapping as visual like so:

x x x

- - -

house house house

whereas anyone would agree the sound of three equally spaced claps would not be analogous to say:

aa b b b

-- --- -- -- ---

This ability to relate or equate two entirely different senses should clue you in that there is a deeper framework at play

I am not sure how abstract thinking for generalized pattern matching make it AGI to solve these kind of problems(not that they are not amazing abilities). If these ToM problems are reducible to these tasks posted by the OP then there would need to be some kind of theorem proving business to convert between the two sets of problems efficiently no?

Take this problem, and assume you dont know a single thing about battles/military history, etc. There are two groups of men standing few hundred feet apart. It is raining, the ground is muddy. One group of men has these wooden curved sticks with a string and iron with pointy ends. The other group has people on horses, and men with very long pointy iron sticks and theyre all covered in steel plating.

Who will win if they all fight against each other? There's really no correct answer but Id expect an intelligent agent to give some detailed reasoning for their decision and to infer details or possibilities and ask questions based again not on previous knowledge but what physically makes sense in the description that was given.

This isnt just a matter of statistics or knowing facts like "rain, mud = heavy armored units will be slower or even trapped", "horses are fast", "bows can penetrate steel", etc. If I give you the full detailed description of the battlefield, very small details can completely change your perception. For example if I said theres big giant logs in the middle of battle, you need to reason about how horses jump, and whether it's something they can clear. You can do this barely knowing horses if you understand how animals in general move. Perhaps there is even some small difference in horses that would make you think they are capable of making large jumps whereas all the animals youve seen before cannot

What Im saying is, to truly reason you need to understand spatial relations very deeply. Indeed id say spatial relations (through time) are all there is to reason about.

So you can view 100 per page instead of clicking through one-by-one: https://kts.github.io/arc-viewer/page1/

Idea for a metric: - Number of pixels that stays the same between input/output. - Histogram changes.

Is there something special about these questions that makes them resistant to memorization? Or is it more just the fact that there are 100 secret tasks?

For an AI that's more useful anyway. If the task is specified completely non-ambiguously, you wouldn't need AI. But if it can correctly guess what you want from a limited number of obvious examples that's much more useful.

1: https://www.crn.com/news/applications-os/220100498/researche...

Francois Chollet: OpenAI has set back the progress towards AGI by 5-10 years - https://news.ycombinator.com/item?id=40652818 - June 2024 (5 comments)

https://manifold.markets/JacobPfau/will-the-arcagi-grand-pri...

https://manifold.markets/Tossup/will-the-arcagi-grand-prize-...

Not sure If I have the skills to make an entry, but I'll be watching at least.

This scales for 200M users and 1 billion sessions per moth for OpenAI, which can interpret every human response as a feedback signal, implicit or explicit. Even more if you take multiple sessions of chat spreading over days, that continue the same topic and incorporate real world feedback. The scale of interaction is just staggering, the LLM can incorporate this experience to iteratively improve.

If you take a look at humans, we're very incapable alone. Think feral Einstein on a remote island - what could he achieve without the social context and language based learning? Just as a human brain is severely limited without society, LLMs also need society, diversity of agents and experiences, and sharing of those experiences in language.

It is unfair to compare a human immersed in society with a standalone model. That is why they appear limited. But even as a system of memorization+recombination they can be a powerful element of the AGI. I think AGI will be social and distributed, won't be a singleton. Its evolution is based on learning from the world, no longer just a parrot of human text. The data engine would be: World <-> People <-> LLM, a full feedback cycle, all three components evolve in time. Intelligence evolves socially.

Pay no attention to the man behind the curtain.

This type of thinking would claim that mechanical turk is AGI, or perhaps that human+pen and paper is AGI. While they are great tools, that's not how I'd characterize them.

I could say the same for us, pay no attention to the other humans who are behind the curtain.

Humans in isolation are dumb, limited, and can get nowhere with understanding the world. Intelligence is mostly nurture over nature, the collective activity of society nurtures intelligence. It's smart because it learns from many diverse experiences and has a common language for sharing discoveries.

A human, even the smartest of us, can't solve cutting edge problems on demand, we're not that smart. But we can stumble on discoveries, especially in large numbers, and can share good ideas. We're smart by stumbling onto good ideas, and we can build upon these discoveries because we have a common language. Just a massive search program based on real world outcomes, that is what looks like general intelligence at societal level.

If you take the social aspect of intelligence into consideration then LLMs are judged in an inappropriate way, as stand alone agents. Of course they are limited, and we're almost as limited alone. The real locus of intelligence is the language-world system.

For example, AlphaZero started from scratch and only had feedback from the self-play game outcomes, but that was enough to reach superhuman level. It was the feedback that carried insights and taught the model.

You can make a parallel to the scientific method: you have two stages, ideation and validation. Ideation alone is not scientific. Validation is what makes or breaks ideas. LLMs without a validation system are just like scientists without a lab.

We're not that smart, as demonstrated by the large number of ideas that don't pan out, we can churn ideas fast but we learn from their outcomes, we can't predict outcomes from the beginning and skip validation.

Here is an example of LLMs discovering useful ideas by feedback, even when they are completely outside their training distribution:

"Evolution through Large Models" https://arxiv.org/abs/2206.08896

This works because the task proposed by this paper is easy to test, so there is plenty of feedback. But the LLM still needs to apply ingenuity to optimize it, you can't brute force it by evolutionary methods alone.

>Happy to answer questions!

1. Can humans take the complete test suite? Has any human done so? Is it timed? How long does it take a human? What is the highest a human who sat down and took the ARC-AGI test scored?

2. How surprised would you be if a new model jumped to scoring 100% or nearly 100% on ARC-AGI (including the secret test tasks)? What kind of test would you write next?

Humans can try the 800 tasks here. There is no time limit. I recommend not starting with the `expert` tasks, but instead go with the `entry` level puzzles. https://neoneye.github.io/arc/?dataset=ARC

If a model jumps to 100%, that may be a clever program or maybe the program has been trained on the 100 hidden tasks. Fchollet has 100 more hidden tasks, for verifying this.