I can't stress how important it is to own your own hardware and colocate. Also, if you are paying for a dedicated server, you can often save money by moving to colocation.
You have to compare the actually performance of the dedi or VM. Cheap dedis and VMs are usually old, cheap hardware with relatively bad performance. I'm running a 20 core, 96 GB ram, 8TB colocated server for $55/month.
Sure, but if something goes wrong with your colocated server, you're supposed to fix it yourself, aren't you? So it feels kinda important that the datacenter is close enough so you could get there quickly on a short notice. I'm imagining having to fly several hours and cross borders just to replace a failed hard drive, all while your server is down.