You can get this as a side effect of multiplexing connections, see ControlMaster in ssh_config. Multiplexing connections also makes opening a second connection to the same host much faster, so it's usually worthwhile on its own.

(Note that it's not perfect: in particular, the first connection to a host remains open until all collections have been closed. This is being worked on, IIRC.)