Well, CPU utilization is the least of Hadoop's problems (picking on it, because it is the most well-known JVM-based framework)

Hadoop core has some shockingly bad design choices (lots of disk IO), and no amount of layers on top of it is going to fix latency issues.

It has nothing to do with JVM "overhead" (which is mostly a myth, anyway).