Author of the post here, so nice to see it being discussed here. For folks interested to learn more about JFR, here are a few other post on that topic:
> the one thing which NMT does not report, despite what the name might suggest, is any memory allocated by native libraries, for instance invoked via JNI
If you're using glibc, then malloc does have information about that, and provides ways to read it, so it's a shame this isn't exposed. It would be quite helpful in the face of suspected native library memory leaks.
In the past with Java 8 JFR wasn't free for use in commercial production environments. So we built our own profilers. It's nice that with NFTC, Oracle allows you to use JFR in production from Java 11 onwards. But they can revoke NFTC anytime. So, it is still wise to develop and use open-source or private alternatives to JFR. Besides JFR still doesn't profile everything well and you still need to build infrastructure for integrated distributed profiling and tracing.
whats a real world use case for this? Been doing big enterprise java for a decade and never ran into a scenario where I would need this. Don't get me wrong, I think theres most certainly value for this, but usually doing plain ole Spring Boot or the dreaded Adobe Experience manager, things just "work". So just honestly curious what problem this would help me solve? Im guessing if you develop a platform you need performance and need to find weak points in your platform? Or is this for building out tools like Dynatrace / New Relic.
I've worked in applications where a lot of IO is required. If performance is something your application cares about then you'll probably end up using direct ByteBuffers which are off heap and you'll likely want to set a sensible value for: -XX:MaxDirectMemorySize
However if this value ever does get exceeded, you need some way of tracking down what allocations happened prior to your OutOfMemory exception.
Though the above article implies the sampling rate is once a second (I guess there's some cost to increasing that rate). Usually you won't be allocating direct memory on the reg since it's expensive to allocate and deallocate relative to heap memory so you kind of want to capture ALL allocations and deallocations. As such a sample based approach is not ideal due to possibly missing some data between samples.
> I've worked in applications where a lot of IO is required. If performance is something your application cares about then you'll probably end up using direct ByteBuffers which are off heap and you'll likely want to set a sensible value for: -XX:MaxDirectMemorySize
It's also worth noting that if your Xmx is larger than 32 GB, you can't use CompressedOOPs, which is a bad deal. Off-heap memory lets you skirt that limitation while still allocating >32GB.
Pre-allocating up front would also have been valid way of doing things.
For the specific use cases I have worked on though, I'm not sure it would have had any additional performance benefits. The specific class of systems I've worked on usually have only a few sockets which open at the start of a day and stay connected until the end of a day. Pre-allocating would make the socket opening process faster but that is not usually the part of an application life cycle that needs optimising. If there were lots of sockets opening and closing or the speed of opening and closing needed optimisation, then what you suggested would be a good idea.
I’ve encountered native memory issues in those “enterprise” apps. Lots a frameworks set an upper bound and if you load one library too much you can run out of JIT compiler cache leading to constant compilation which tanks performance.
More ways to get insights can’t hurt.
That's a good question. I can't imagine that you'd need to use this to track normal garbage collected memory. There were pretty good tools already in place for that.
I guess what you'd want to use this for, is when your application is directly allocating memory, e.g. via direct byte buffers. That's not something you'd do in an enterprise application, it's more something you'd need for high performance image processing, or maybe for some extremely high performance web server.
Using offheap memory allocation in Java is generally a bad idea.
Java is a GC'd language. If you need to manage memory manually, use a language designed for that. The lengths people go to NOT learn another language (while having to learn stuff like this and do primitive memory management in your application code - which itself may use the GC if you're not careful) amazes me.
It’s the same as using “unsafe” in rust, it shouldn’t be necessary by default, but when you do need it, you can have a well-defined part that safely encapsulates all the logic of dealing manually with memory. The end result will be a safe program with the assumed memory/performance-bottleneck of doing it naively solved.
>Using offheap memory allocation in Java is generally a bad idea.
Memory mapped files and direct buffers do work well. I'd never consider it a bad idea; technically both would be freed if there are no strong references to them (not much different than allocating massive arrays).
Think of it like that the jdk standard library has to use native memory for any IO or even a trivial zip compression. ByteBuffers were introduced in 1.4 (around 20y back) to allow operations outside the managed system, the tools are available to anyone, e.g. implementing zstd with direct buffers is not hard.
Unfortunately, you do pay a fairly large price for bytebuffers, even off-heap ones.
I saw a pretty jawdropping speedup moving from bytebuffers to the new foreign memory stuff with in the marginalia search index code. Most operations run about 50-100% faster, even discounting the rigmarole needed to deal with mmapping >2 GB files. Haven't dug too deep into why this is so I'm not entirely sure why, may have something to do with being able to declare non-shared memory ranges in the arena allocator, saves a bunch of synchronization maybe?
I can't wait for this stuff to leave experimental. It's such a quality of life boon for dealing with off-heap memory, having explicit lifecycle control is.
> Unfortunately, you do pay a fairly large price for bytebuffers, even off-heap ones.
I'd disagree. Heap based bytebuffers - I don't consider them interesting, they are just byte arrays. The direct ones suffer from unmapping (munmap on linux) - it's a slow process that has to flush the TLB. So the only sane way to use them allocate once/reuse. Anything else I'd consider a programmer error.
mmap on large files indeed does suck (a bit) w/ ByteBuffers as it'd require an array of them.
Personally I have been using direct buffer since 1.4, so I guess I am also quite used to them as well.
Right, I'm not talking about mapping or allocation costs or anything surrounding the access, just the impact of addressing off heap bytebuffers them seems quite a bit slower than what you get with the new foreign memory API... for whatever reason.
You mean using directBuffers in java - that could be, if the JIT fails to remove the bound checks.
It should never be an issue in native code, of course. But yes - it's possible that it happens with Java code. In that case you'd need print assembly and looking at the generated code.
Some libraries have 'switched' to straight unsafe use (which has no bound checks, of course), so there is that.
While I fully agree as a rule you shouldn't have to do manual memory management in Java, the functionality is there and useful in exceptional situations.
I've seen integrations where there's no '100% pure Java' implementation available, so you have to choose between picking a battle-tested native library or reimplementing from scratch in Java (which sometimes isn't even possible with closed source systems)
It is called taking your cake and eating it too, in terms of productivity, IDE tooling and library ecosystem, for the same reasons ML frameworks use Python with bindings to C++ libraries, instead of being 100% written in C++.
* https://www.morling.dev/blog/finding-java-thread-leaks-with-...: Discusses how to find thread likes with JFR and JFR Analytics, a project I've created for querying recordings with SQL
* https://www.morling.dev/blog/towards-continuous-performance-...: Discusses how to use JFR for continuous performance testing, by means of asserting "proxy metrics" such as allocation rates and IO
* https://www.morling.dev/blog/rest-api-monitoring-with-custom...: Discusses how to create your own application-specific JFR events