Yes and you do things in Java like create an MyIntCollection to get around the shortcomings of generics wrt. primitives, but the point is that it is just that: workarounds.
What I really want is for perf to be good while the code is still clean/readable/expressive/idiomatic. In Java/C# you must usually write idiomatic+slow and then refactor into fast kludges.
I'd say that's always the case. Performance for the vast majority of code is good enough with idiomatic or even horrible code. Places where you need to work against the language for perf reasons should be rare (and if it concerns most of the code in a project, then I'd question the choice of language for that project). Also that holds true for most languages. Even in C you'll avoid allocations in tight loops that need to be fast because it comes with an unpredictable performance cost. That's not much different than pre-allocating a buffer to work on prior to an algorithm in C#.
Agree it's always the case regardless of language, but it's about moving that boundary where you have to leave idiomatic and make something less readable in order for it to perform.
As an example, C#'s value types and generics with primitives makes that a lot better than in java.
What I really want is for perf to be good while the code is still clean/readable/expressive/idiomatic. In Java/C# you must usually write idiomatic+slow and then refactor into fast kludges.