Well inbound email is really not a big issue from my experience. The issue (if any) is deliver-ability of outbound email. But that can be handled in any number of ways. (You can use someone else's smtp if there are issues or you can just follow best practices to have a clean ip address they are available).

I have been doing both inbound and outbound for roughly 20 years on our own equipment. But even doing just inbound gives you better control and in a way you are able to lessen the attack surface of being a large vulnerable target.