You can think of it this way: if half the model is running on the GPU, and the GPU is infinitely fast, then the total calculation time would go down by 50%, compared with everything running on the CPU.
Wow seems like common sense turned into a law. Maybe I can get a law :-).
Anyone who has commuted on public transport probably knows this intuitively. (Using a kick scooter instead of walking cut my travel time by a good 5% which was excellent, as I still needed to be on a bus where that made no difference.)
But yeah! There's a lot of common sense that, with some mathematical formalization, yields useful and extensible laws.
Amdahl's has been robust because it opined on the 60s supercomputer coprocessor architectures, and then in more recent decades with consumer multicore chips.
Laws aren't famous because they're insightful -- they're famous because they're useful.
