Virtualenv solves 90% of the problem, and it is not like you need to revisit your setup on a daily basis.

I would be more willing to spend time on my test setup than anything else, because that affects the development experience.