Hacker News new | past | comments | ask | show | jobs | submit login

LLMs are not very bad at reasoning. People say this but struggle to demonstrate how.

There are many types of reasoning. For some like abstract analogical reasoning, LLMs are top % performers.

https://arxiv.org/abs/2212.09196

There's many papers at this point demonstrating pretty strong reasoning. From Casual Reasoning, and explanatory reasoning to algorithmic reasoning.

https://arxiv.org/abs/2305.00050

https://arxiv.org/abs/2204.02329

https://arxiv.org/abs/2211.09066

The only kind of reasoning that's somewhat lacking is logical reasoning. And even then it's not awful, just lagging behind equivalent human performance.

So someone really needs to tell where this oft repeated "can't reason" rhetoric is coming from. Because the numbers sure as hell don't support it.




Logical reasoning is also the easiest to solve with a program if you can get it into the right format, which LLMs are good at.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: