Thinking carefully about loss scenarios is really important, so you're thinking about the right things. I used to have my SSH key on a Yubikey for awhile and now I have it in my MacBook's secure enclave guarded by TouchID. In both of these cases I'm not terribly worried about loss because it's almost always the case that loss of SSH access is pretty easy to recover from. In most cases other people on my team also have SSH access and can swap out my key if I lose the Yubikey. In a number of other cases the cloud server doesn't have any/much in the way of irretrievable data. It's just running various software that is usually version controlled with git. So even if no other team member had access it would be pretty easy to spin up another machine to replace the one that I can't log in to, change the domain name to point to the new machine, and terminate the old one.