Replying to myself since I can't edit my reply now.

This is the link to the TaoBao Tengine doc describing the feature: http://tengine.taobao.org/document/http_ssl.html

My memory was faulty: they have a way to get the passphrase, not the key itself. But wrt security, both are equivalent.

That's interesting, but I don't think it raises the bar a lot. Private keys are still kept in process memory, from where they can be easily extracted by the attacker who can attach to the process.

A great innovation would be to have web server fork a special process that will only handle private keys. That other process would be running under a different username. Bonus points if a separate process can be deployed for each key. (It's possible to achieve a similar effect by running decryption in a separate proxy layer.)

My two cents is that requiring the ability to snoop on a running process is a significant extra hurdle compared to simply reading the filesystem.

Especially for virtual machines which almost everybody is running on these days. An attacker can simply read the disk from under the OS.

Although I can understand why you might think it's not significant enough. I do agree that storing private keys in a separate user's process can help with security.