There's also the "can I connect to this outside Internet service today?" risk.

As I recall, people said companies can generally VPN out, but that adds to your costs (paying the external VPN site, plus latency, another single point of failure, etc.).