A test of chess puzzles can reliably predict a player's ELO rating and what kinds of game elements they struggle with. My late dad did work on this in the 1980s to assess machine and human chess performance which culminated in the Bratko-Kopec Test[0], which eventually became a part of a standard suite for assessing the performance of new chess programs. He also ended up running the test on hundreds of human players to test its calibration.
He created several subsequent tests and wrote a book about it [1]. I make a version of a few of the tests for iPhone if you're so inclined [2].
> "What stops you, I think, is a combination of not really believing you’ll get it and not really caring. Is that too harsh – or is it somewhere close to the truth?"
This reminds me of the curse of working with really good senior engineers. They already know the answers, they've already solved the puzzles. It can be very easy to just defer to them all the time.
If you are a senior engineer who really understands a system, you need to be conscious of this effect if you ever want someone else to start learning your system.
In my experience, people underestimate their abilities and are so afraid to mess things up, even in a preprod environment, that they don't even try. I try to encourage people and let them know they can't mess anything up, but like the saying goes, you can lead a horse to water, but you can't make them drink. Some people get it faster than others.
There's also the pressure from above to fix things quickly, meaning some people don't have time to really explore and learn and need to be given answers...
- during long game, chess grand masters have physiology comparable to marathon runner, while he runs. Deep thinking for several hours, takes huge load on body. All the logic and critical thinking, is not going to save you, if you are not fit, and your brain does not work correctly.
- real life is not about solving puzzles. Real life is a rigged game where rules are not enforced. Instead of finding problems to solve, you need to find oportunities (and loopholes) and exploit them!
- game is rigged, and oportunities close fast. What worked a couple of years ago, probably does not work anymore.
I wouldn't be so hasty to say it is obvious (parent made no claims about calories).
Many top chess players have considered themselves athletes, and if you've ever tried to calculate under pressure at a board for a few hours, I'm sure you'll agree it is an exhausting activity. Fischer engaged in athletic training when preparing for tournaments, for example, to aid in maintaining mental focus (he wasn't unique).
It would be silly to say that 2400 Elo indicates you can run a four-minute mile, or that calculating 6-ply in a closed position burns the same calories as running a block. If the claim is 2400 Elo tend to have similar vascular flow in the brain to people who engage in aerobic exercise or something of that nature -- maybe?
I don't think that statement is meant to be taken literally. Playing chess takes ridiculous amounts of mental effort and that's still subjectively significant even though the brain isn't consuming more glucose. We aren't machines, our attention is limited and chess consumes vast amounts of it.
He says, before laying out the outline of the puzzle and giving suggestions on how to go about solving it
Everything can be about solving puzzles if you let it. Given enough time and patience you can understand anything - the only interesting question is, how to you decide what to focus on?
Navigating life is absolutely an exercise in puzzle solving, at every step you know where you’re at, you know where you want to be, and you know what resources are available to you - given all that, how do you plan your next step? If your first solution doesn’t work, you do a retro, learn your lessons, and move on to your second solution, and your third. It’s all engineering.
> It’s hard in real life, too: vanishingly few people are meta-rational enough to try really hard to falsify their own ideas. Your brain really wants to find reasons to support what you believe.
I don't think he goes with "meta" deep enough. It is great for engineering problem solving mindset But it is also a good way to end up like underpaid post doc, who needs second job just to pay rent.
And this type of advices are usually coming from someone who "made it", has its own house and is practically retired. Very impractical and harmful (to some extend) for young minds.
Practical implementation for young person is not "falsifying" and trying again again. But coming with solutions that takes minimal time, is good enough and comparable to coworkers who work on the same salary. Time you save can be invested into education, family, hustle and so on.
> - real life is not about solving puzzles. Real life is a rigged game where rules are not enforced. Instead of finding problems to solve, you need to find oportunities (and loopholes) and exploit them!
"Real life puzzles" are too open-ended and have too many levels to really be called puzzles. A puzzle has a closed set of rules that usually gives you only one level on which to solve the problem. Many interview questions could be described as puzzles. A "real life" programming task has a bunch of different levels: what's the real problem the customer wants solved, is this the problem this customer wants solved first, do we have a bigger customer with a bigger problem you should be working on instead, can the problem be solved without programming, should the problem be solved in a different system, are there other people on the team who solve problems like this in their sleep and they'll give you the answer in five minutes if you describe it on Slack? If it does seem like you need to solve the problem, what's your level of confidence that with investment of X time you can solve the problem, for different values of X, and given this information, does it still make sense to try to solve it?
What makes puzzles relaxing and reassuring is knowing that there is a solution, and that you know all the rules. Also, you know that you'll recognize the solution when you get it. Real life rarely gives you that reassurance. With a real-life problem, you don't know if there's a solution, and even when you have one, you can't know that there wasn't another solution that would have been much better, because of possibilities you failed to consider. The only way to turn a real-life problem into a "puzzle" is to strip away the open-ended real-world context and present a subset of it that can be described in a closed form.
The business ecosystem, like biological ecosystems, involves forms of collective life that have learned how to sustain themselves in competitive environments, usually by seeking moats. Those moats "rig the game". The moats tend to fail when the environment changes; e.g. due to technical innovation, social movement, external shocks. It is the central interest of any business to build moats and drawbridges.
Life is turtles all the way down and drawbridges all the way up. Anyone seeking opportunities is looking for the openings between those moats and drawbridges.
I'm curious what strikes you as pessimistic in this comment versus just being realistic about the current social structures implicit to the US (I cant speak for the rest of the world)?
Probably because GGP’s comment reflects a narrow, unfavorable, and extreme view of reality
> Real life is a rigged game where rules are not enforced
Even in the United States, there are plenty of stupid and non-stupid rules that people are forced to follow in order to “play the game”. There are also plenty of rules that only apply to certain groups. There are also plenty of people who don’t play the game at all.
Maybe if GGP weren’t so extreme and negatively one-sided with his view, then it’d come off as less pessimistic and more critical.
I would probably say "cynical" rather than "pessimistic". I would also say that it does not actually reflect reality. Reality is also not the exact opposite of this perspective. Reality simply doesn't bear reduction to either "everything is a rigged game" or "everything is fair"; it's more complicated than that.
I think you're missing the point, while you are pretty correct IMO, viewing the world in such a dichotomic way misses the fine details along the spectrum
Spot on. When you discover that people are solving a problem by considering a wider context than you did, do you broaden your thinking about the problem, or do you accuse them of cheating and complain that the rules aren't being enforced? Morality and (most) laws should be respected, but outside of that, rules shouldn't stand in the way of solving problems.
The article is fine, inspirational, interesting, and all that, but one quibble: reporting ratios is potentially misleading. If grandmasters spend 4 minutes falsifying for every minute ideating, and amateurs spend .5 / 1, that's great. But what if amateurs spend 30 minutes coming up with a move vs 1 for masters? Could be the grandmaster is faster at ideation by a larger fraction than he is faster at falsification. That also makes sense in a "just so" sense, because maybe falsification is brute force with a large depth of search, and ideation is more like a lookup table - just see where your pieces can move.
I thought maybe I could find some primary sources, but the [1] notation is just footnotes.
You’re right that GMs are much faster ideating, I point this out later in the essay. But they also spend longer on falsification, even in absolute terms.
To an extent it depends what ELO you're talking about, but I'm an amateur (~1950 rated on lichess) and I find, when I watch GM videos, that I have about the same move ideation as them, at least in the midgame. Sometimes better, depending on the GM, since everyone has different strengths. But the GMs consistently better than me by a lot, of course, especially in overall calculation and in knowing openings and endgame theory.
The examples of Dropbox and iPod being criticized on tech sites, but going on to become very successful, seems practically a part of the mythology at this point - but it seems to be _always_ those two examples.
Are there legitimately multiple good examples of "Criticized on HN pre-launch, yet became surprisingly successful"? I'm curious if the lesson to learn from Dropbox & iPod is more of "believe in a product, despite the criticism" or "sometimes, even accurate predictions are wrong"
I really enjoyed this article. I would recommend others check out "Advice That Actually Worked For Me" by the same author. This same topic is mentioned in #6.
https://nabeelqu.substack.com/p/advice
So: good (chess) players spend more time mentally countering their proposed moves before moving.
For developers or managers on HN, one outcome would be that it's best to start one's career in testing, or to respect the resumes of those who started in QA. If/since there are hundreds of ways things can break, it's a harder problem to show how it will, or prove it won't; and building a mental library of fault models helps in vetting designs and implementations.
Or, we could teach fault models directly, instead of accumulating by experience. See e.g., Robert Binder, "Testing Object-oriented Systems" (and ignore the model-driven-development gloss from later editors).
But the most important note is the aside: the author avoids chess as addictive. Should we ask ourselves: how can this be? Should that change how I think about my own work?
I think the ideal first job in tech is IT help desk, not QA.
Everything that shows up to the help desk is broken. QA people need to have a skill for breaking things or at least an awareness of how things break. They will learn this at the help desk.
> It’s not as simple as “founders are optimists, scientists are skeptics”, though.
It's not much deeper either. There's having a skeptical yet positive mindset which is somewhere in-between. I once made a complex toolkit of runtime query optimizations that was a hodge-podge of kludgy things, which worked out well in practice. Someone asked me how I ever came up with the whole thing. I said I just started out and made one new improvement after another. They said that it was so fraught with obstacles (listing them off) that they'd be daunted to even start. I said I wished I'd talked to you sooner, you just gave me a roadmap that I didn't have while I was figuring out the pieces. Note that he was a high-level chess player (unlike no-rank me), but perhaps not as much of an optimist or risk-taker.
If I had to say what a good founder mentality is, I'd say that they have experience navigating uncharted territories and finding success, whether on small or large scales. That kind of practice makes them good at sizing up risks and rewards. Related to this might be of a kinesthetic learner type--learn by interacting.
I play chess (poorly for the time spent on it) and I'm also a reasonably successful founder of a couple software companies. I find my struggle with chess is that I want to act intuitively, something that has served me well all my life in other avenues. But the board doesn't lie and if you don't think thoroughly you will get punished.
I have the capacity for it, I can think thoroughly in puzzles and perform much better there than my on board play but I just struggle so much with the discipline during regular games to falsify my moves. So much so that I've mostly given up on trying to improve despite really loving the game, it just grates on me. I know I could be better but I lack the discipline and I guess I just don't want to exercise that discipline in a game.
> I find my struggle with chess is that I want to act intuitively, something that has served me well all my life in other avenues. But the board doesn't lie and if you don't think thoroughly you will get punished.
I believe strong players do act intuitively when playing chess (especially fast chess), it's just that they've developed their intuition through lots of practice and thorough thinking in the past.
For some reason our intuition about life seems to be more developed, or perhaps the game of life is incredibly complex and most people are roughly at the same skill level.
> I find my struggle with chess is that I want to act intuitively, something that has served me well all my life in other avenues.
Same. Truth be told, I don't want to have to explicitly think at all. I want my brain to recognize the situation and steer me towards victory through entirely unconscious processes.
Where is the linked post getting the 4:1 vs 1:2 time-spent-on-falsification ratios that it's claiming? It's like the heart of the entire argument, but it's not sourced.
Apparently I try too hard to falsify the falsification. I became convinced that h4/g3 pawns could be used as a trap while I march the b-pawn. Bd5 Bxh4 b5 Bxg3 Qxf7 Qxf7 Bxf7 Kxf7 b6! and the pawn can't be stopped.
Except it doesn't work, I needed to falsify the falsification of the falsification 4 move down the line to see why :)
i am quite confused . It started saying good chess players are more careful and spent more time falsifying ideas but then he later gave startup examples which is the opposite ( not so careful with falsifying. Just jump into the water with conviction and figure things out on the way ) . Startup game is more like poker . It is very different from chess . Somehow the author drew the wrong conclusions. Very confusing
This is not quite true, because poker is fundamentally adversarial, while startups are mostly not adversarial, at least not directly.
Startups are a beauty contest where each player focuses on maximizing the things about them that appeal most to a panel of judges (customers). Similar to the scramble competition that Benenson cites here, rather than an arm-wrestling contest.
Right, but my argument is that there are many niches in the market where there is no competition, and startups should try to find that and then creates moats with IP, data, etc. There are many situations where startups don't have direct competition, because they are inventing something radically new. Often true in life sciences, for example. You're right that in those cases they are in a race against time, since someone will eventually come along, but they can go for years without a direct adversary.
I don't think startup game is more like Poker. It's a lot of experimentation and learning. True, you gotta protect your bank roll, but the type of strategy initiatives to take are something like that of a chess.
PS: I'm an early stage founder, who has finally some traction with my current B2B data infra SaaS. I've had a failed company in the past which had 4 major pivots, where we decided to return most of the funding to learn few things again.
He created several subsequent tests and wrote a book about it [1]. I make a version of a few of the tests for iPhone if you're so inclined [2].
0: https://www.sciencedirect.com/science/article/abs/pii/B97800... 1: https://amzn.to/3PVOne9 2: https://apps.apple.com/us/app/test-your-chess/id362448420