In addition, it's interesting to see that TruffleRuby does better than CRuby on the longer running benchmarks, and worse on the shorter ones. The page claims that JIT warmup "is applied when necessary", but there are no further details. If I had to bet, I would guess that in the case of TruffleRuby maybe not enough warmup is being done in these experiments.
That sounds likely, but if it's not warmed up in some amount of time that a decent programmer believes to be enough, that probably means that it's warming up too slowly! With that being said, it could also mean that the faster ones could be even faster.
I totally expect base64 to perform like this though. It spends like one line in Ruby and then it's shipped off into the C extension, so really it's testing the differences between Sulong (LLVM bitcode interpreter for Graal) and a proper C compiler.
