Personally, the main reason I moved from vim to emacs is because I don't want to have to think about my editor, I want it to be pure muscle memory. I'm okay with hitting "jjjjk" to go 3 lines down rather than trying to guess that it's exactly 3 down and typing "3j". Emacs-style navigating/editing is great for this kind of error-tolerant navigation/editing. It leaves my brain mostly free to focus on whatever I'm actually coding at the time.
I think most experienced vim users will tell you that it is pure muscle memory for them. Also, vim supports jjjjk as well as 3j. Anyway, I just realized I am debating vim vs emacs so I'll stop here.
I go with a hybrid approach: I hit the number row somewhere in the loose vicinity of the number of rows I want to move, and then hit "j" a couple times to get exactly there. If it's more than a single-digit number of rows away, I might do the first step more than once.