Depends on the stack used. If you have persistent connections you'll incur far fewer handshakes than requests. If you use an elliptic curve scheme key exchange costs are negligible. But sure, if you do one 4096 bit RSA exchange for every request it will be costly.