I'm not exactly sure what your threat model is, then. The typical way this gets deployed is with U2F USB devices with non-extractable keys. "Access to all keys" is not exactly technically feasible.
If your concern is "well, there's always the admin/support backdoor", i.e. a compromised admin account or a social engineering attack on support personell could lead to attacker-controlled keys being enrolled, I'm afraid that's not really something you can solve by just throwing new technology at the problem. However, you'll definitely make it harder to even get to the point where admin accounts are compromised by rolling out U2F or webauthn.
Personally, I'm perfectly fine with a technical solution that solves phishing for everything but the most advanced social engineering campaigns.
If an attacker compromises the credential, the credential is useless, yes? The credential is still customized _per origin_, so you'd have to decap the U2F key to extract the secret. This hardware is specifically designed to make that hard to do.
