Why can't SSL be used to secure a symmetric key exchange that clients then save on their own computers or USB dongles?
Of course, privacy obsessed clients can simply run their own email client off a USB stick with their own private keys.
But anyway, if the servers simply store keys which the clients themselves unlock via their passwords, and the server stores nothing, the NSA would have to either sabotage the hardware the clients use or else do rubberhose cryptanalysis.
That's pretty much how it works, but without persisting the symmetric key for too long. Asymmetric crypto is expensive, so first the symmetric crypto key is exchanged and then only that is used for data transfer.
the only solution is if the client (open source so it cannot be compromised) did all the encryption and decryption. that way you can store your keys and the server can store your encrypted data. otherwise you trust a compromisable third party to authenticate the server as legit.
Unless you provide sources, and, really, a viable build environment -- see RMS's discussions on this, and such counterexamples as Red Hat's rather difficult-to-reverse-engineer build environment (the real value-added of CentOS and other RHEL free forks), as well as Microsoft's long-standing source licenses to academics (build environment not included AFAIR) -- it's possible to hide either failures or backdoors in your products. True crypto is well-tested crypto. That doesn't mean "verified" (can't prove a negative), but it does mean very thoroughly vetted.
Even proprietary security companies have long practiced source provided (different from "open source") code for their key crypto engines. PGP comes to mind in this regard (the company, not the protocol).
But fully open source means you've got vastly more exposure of your crypto guts to examination.
Because availability of the source without conditions is the only way you can get effective independent audit. It's not a magic wand, but you're relying much more on trust without it. This, on top of many-eyes and the pressure not to rely on secrecy of the source for security (a bad practice) improve the situation substantially. Even so, nothing will be perfect, of course.
Of course, privacy obsessed clients can simply run their own email client off a USB stick with their own private keys.
But anyway, if the servers simply store keys which the clients themselves unlock via their passwords, and the server stores nothing, the NSA would have to either sabotage the hardware the clients use or else do rubberhose cryptanalysis.