1) You should definitely create a passphrase for your private key (you should only be asked to unlock it once due to ssh-agent). Otherwise if someone gets hold of your private key they can login to any machine you have set up an authorized_keys entry on.
2) use ssh-copy-id to install your public key on a remote (and fix up the permissions on ~/.ssh etc. which for me is the #1 case of key based login not working).
and fix up the permissions on ~/.ssh etc. which for me is the #1 case of key based login not working
DOH! I just spent the past 15 minutes trying to figure out why it wasn't working until I stumbled upon my .ssh directory having worldly permissions. Was just about to come here and post the same thing. chmod 700 is your friend!
ssh -vv (with more or fewer vs) is also your friend. IIRC it tells you about the permission error either there or in the sshd log, and you can also find a lot more errors in the -vv output.
You should run ssh -vvv on a normal, working connection at least once to get a sense of what normal output is.
Thanks for the cool explanation. I never really dug into ssh that much, just posting a helper to connect without password ;)
So basically ssh-agent should be always running and you add your key to it (via ssh-add) and you never enter your password again, but the key still is generated with a passphrase? Correct?
That's correct. And on OS X (at least on 10.5) you can put the key into your keychain and unlock it with your system password. Just make sure you lock your keychain when you don't need it.
2) use ssh-copy-id to install your public key on a remote (and fix up the permissions on ~/.ssh etc. which for me is the #1 case of key based login not working).