The JVM can run pure Java code to within 2× of native code (although it has a pretty weak vectorizer, so if you're getting vectorized speedups in native code, Java won't get you that). However, calling a native function from Java is a really high overhead call. Which means that the core engine likely can run at essentially the same speed, but the graphics is going to be slower.
Both CLR and JVM can be fast, it's not that. It's slower than C++ still, but not by much. In some benchmarks a Jit:ed lang can actually be faster.
However, the real reason Java and C# apps are most often much slower than native apps is because idiomatic C# and Java is a lot more work to execute. If you want to write really high-performance Java or C# code, you have to make it allocation free on hot paths, you have to be very careful with what types are on the stack and which are on the heap, and you have to pay a lot of attention to data layout (In java that doesn't have structs especially, making lots and lots of types as SoA instead of collection of objects for example).
The bottom line is: while you can make C# and Java code run really fast, it's very hard and non-idiomatic C# and Java code you end up with. It's so hard, that it's actually no easier than writing e.g. Rust or C++ code to do the same thing! And that is exactly the point where C# and Java no longer makes sense: if you can't use the idiomatic and ergonomic way of writing it, it has lost its value.
Note that this is the default, standardized behavior. If you're willing to use Oracle (and I think by extension OpenJDK)-specific features then there are some functions with "unsafe" in the name that allows you to hand over memory directly to native code, instead of copying it over. The default behaviour is to copy all memory passed in and out of the JVM, so as to avoid native+java+GC messing with the same memory area (potentially at the same time).
