But I'll bite, if your mail server goes down - and it will - you now have no email. You can't email your hosting support. You try to log in to your host control panel to do a reboot or raise a ticket but forgot your password. You can't get the reset email. Somehow you manage to login and raise a ticket but won't get any email notifications when the technician responds asking for your server id number...
(2) set up mail server, create account(s)
(3) reset public email account with domain registrar + hosting facility to newly set-up email addresses
(4) kill public email account
You only need one for a very short while to bootstrap your own email server, you don't need it forever.