As I heard, it is mostly about heat. However, there is also a distance component. It takes an electrical signal about 3 * 10^-9 seconds to cross 3cm, about the size of a CPU. That is roughly one clock tick on a 3GHz processor. Add in 'fill time' (you need some time for the voltage to build and stabilize) and moving signals across a CPU becomes a challenge.
My speed calculations used the speed of a signal in a copper wire, not sure how the minute size and perhaps even angles of CPU wiring affect that.