You've made a great list of the downsides, perhaps you could at some point list the upsides? A yubikey you don't move around, and that you don't take extra steps to ensure the security of, seems no more secure or useful than a cert file on your hard drive.
As others have said, you have to touch the yubikey to get it to generate a one-time password (it acts like a USB keyboard).
The yubikey is one factor, the VPN also requires a second factor (memorized password). These are concatenated so you type the password without pressing the enter key, then tap the yubikey (which "types" the OTP + enter key). This process works in web forms, shells, etc. Could hardly be simpler.
If the laptop is lost/stolen, I can deactivate the token.
Unlike a cert file, a hacker can't steal the keys in a Yubikey if the machine is compromised. Same reason why many enterprises prefer using TPMs for storing machine certs.
The advantage of a Yubikey over a TPM in this case is that the Yubikey requires a physical tap before it'll sign a request, which prevents certain MITM attacks.
If I understand correctly, you must touch the Yubikey to reply to a second factor request. Simply being present in the USB port is not sufficient to utilize its credentials.
