I use the arrow keys in both Vim and Emacs. There are enough keys to remember (most of which I forget all of the time... like paging up and down, have to look that one up every time) that I don't need another set of keys to remember. Arrow keys work just fine.