Yes you would be annoyed. Chutes and Ladders consists of activating a spinner, moving a piece along a path based on the number pointed to by the spinner, going 'up' if you land on a ladder or 'down' if you land on a chute, and you 'win' when your piece lands exactly on the last spot. You would probably get as few as 3 and as many as a half dozen 'buzzes' as you figured it out.

So you should try the test (or try it on an unsuspecting test candidate) my experience has been that no matter how 'simple' the game is, the fact that it seems completely orthogonal to the person, their ability to 'shift gears' and learn a new thing so that they can get on with the desired thing is blocked. It is especially true if you put a time limit on getting the desired thing done.

The challenge is that a number of things seem to have an assumption about your basic skills that may not be valid. An acquaintance went nuts when an app required that he cut and paste something on his phone. He had never had a reason to do cut and paste before so to complete the task he had to learn cut and paste. He was making zero progress on that goal. I did it for him, then later went back and showed him in a non time-constrained / outcome-desired setting how to do it. That worked fine of course, and now its part of his tool kit.