2 processor cores and 2G of memory is all mine needs to perform well. Perhaps my level of "fast enough" is much slower than yours, but my VM doesn't feel any slower than the host.
No, nothing at all. I have an ivy bridge i5, 8GB 1600, and two old nvidia geforce 8600 under sli. Perhaps the people at /r/techsupport could help. Full screen in my VM feels just like booting into Ubuntu - no lag at all.