The way I explain the unreasonable effectiveness of JITs and adaptive compilation (multiple compilers with the advanced ones focusing on the hot spots) is by contrasting constants and variables.

In a long enough time frame nearly everything is variable: over decades even computer architectures and language syntax change. In a short enough time frame nearly everything is constant: in a single cycle a von Neumann computer will only change a single memory location and all the others will have a constant value.

Even though an AOT compiler might have minutes or more to work on a code fragment, the code it generates has to work for a long time and on many different machines. A JIT might have milliseconds or less to work on the same code fragment but what it generates only needs to work on this exact machine and only for minutes or hours. In fact, it might even be wrong and have to be recompiled less than a second after the JIT produced it.

So the code generated by the AOT must treat as variable things that the JIT can pretend are constant (with hook to recompile if it proves not to be so).

This is slightly related to partial evaluation, which is a key idea in Graal.