Is it really much harder to leak a private key than a passphrase? (It's obviously harder, but not sure whenever a difference is significant.)
While one can't peek from behind your shoulder, if they got a keylogger on your machine, they could steal ~/.ssh/id_* files as well (and sniff their security passphrases too).
Brute-forcing a key is pretty much impossible and people - despite all advice - still use short and insecure passwords. Certainly a machine that does not root login at all is better than a machine with key-based root login, but a machine with key-based root login is better than password based root login. The perfect is the enemy of the good here.
While one can't peek from behind your shoulder, if they got a keylogger on your machine, they could steal ~/.ssh/id_* files as well (and sniff their security passphrases too).