I find something similar happens when mowing the lawn. When I try to mow in straight lines by pushing next to the previous row, I end up with wavy shapes. But when I fixate on a distant reference point beyond the lawn (like a rock or weed sticking up above the grass) and just walk staring at it, I end up with perfectly straight lines. It's uncanny.
This is also a trick for drawing surprisingly straight lines on paper. If you start at a point and watch where you're drawing, you'll end up with a terrible line. If, however, you start at a point, focus on your target, and move the pencil towards the target, the result is surprisingly good. Also works for circles.