Hacker News new | past | comments | ask | show | jobs | submit login
Oracle doing a survey on sun.misc.Unsafe usage (oracle.com)
51 points by pjmlp on Jan 27, 2014 | hide | past | favorite | 23 comments



Usage of the Java package sun.misc.Unsafe. For some reason, I was thinking NNTP groups ...


After Java 8 gets released, the long due modularization will take place.

They plan to drop all private APIs people have been using, while providing an upgrade path with public APIs.

Hence the survey.


I do hope they don't forget DirectBuffer, bug #4724038. This incantation is required to unmap a memory-mapped FileChannel, which you need to do so that you can delete it, because the OS of course locks it:

((sun.nio.ch.DirectBuffer) buf).cleaner().clean();

Apparently they didn't want to make it public because there is no portable way to make it thread safe. Which is just great if you have to work with temporary mmapped files...


I'm sure a Maven mirror can give you enough JARs to do your own survey. I don't believe a poll like this, unless really publicized, will be accurate. Also, Unsafe (and maybe another one or two of the unpublic APIs) should have an opt-in shim/polyfill that you can put in newer versions of the JDK/JRE after these APIs are taken out.


I doubt it. Any publicly available JAR shouldn't use a private namespace call. However, private repositories in an Enterprise that has "standardized" on the Oracle JVM is much more likely to use private methods. Hence the need for a survey.


Not really.

It will show which APIs are being used, but not WHY they are being used.


An interesting post on c++ like memory management in java:

http://vanillajava.blogspot.ca/2014/01/sunmiscunsafe-and-off...


This is also a good post, along the same lines: http://mechanical-sympathy.blogspot.com/2012/10/compact-off-...


The most interesting quote:

"Lately, I was comparing standard data structures between Java and .Net. In some cases I observed a 6-10X performance advantage to .Net for things like maps and dictionaries when .Net used native structure support."

The ".Net" can be much faster than Java because Java has to have the things on the heap which can be on the stack.


> The ".Net" can be much faster than Java because Java has to have the things on the heap which can be on the stack.

This is a famous fallacy (except in some very specific circumstances). The reason for the performance difference is probably due to packed structures and better CPU cache behavior.

Stack/heap makes very little difference in most circumstances for two reasons: first, Java heap allocation is as fast as stack allocation – it is a thread-local pointer bump. Deallocation for short-lived objects is free. You do incur an indirect cost of triggering a young-generation GC which causes a pause linearly related to the size of new-but-not-so-young objects. These pauses add up to very little (under 1% of total time). The second reason is that stack memory is usually a very small percentage of the total memory for interesting programs. Programs manipulate data. Non trivial programs usually manipulate lots of it, otherwise we wouldn't need so many gigabytes of RAM. A thread's stack is rarely over a few megabytes in size, so to have a significant portion of your data on stack you need tens of thousands of threads which the OS can't handle anyway.

To sum up: 1) Java's handling of short-lived memory is extremely fast; 2) all interesting data lives on the heap anyway.

Stack allocation can make a difference in reducing the number of young gen GCs, but it's not a huge difference in most circumstances. Packing your heap data structures better is more important.


> pause linearly related to the size of new-but-not-so-young objects

Stop and copy still has to traverse all objects that contain pointers, even ones in the nursery (otherwise you'd incorrectly free objects in the nursery that are only referenced by other objects in the nursery). So the amount of scanning you do is proportional to all objects in the nursery (also the size of your stack/root set), and only the copy is linearly related to objects to be tenured.

> The second reason is that stack memory is usually a very small percentage of the total memory for interesting programs.

A small percentage of the long-lived memory, sure. But programs spend a lot of their time working on ephemeral data, and this is precisely why the generational hypothesis holds. There's no faster nursery than the stack: allocation fits nicely into the function prolog and deallocation requires no traversal of interior pointers.


I hear this sort of claim over and over again. Yet whenever I've had to optimize Java code that does heavy math (particularly work with vectors, matrices, complex numbers, etc) the number one optimization is to more or less maintain my own small object stack, or manually put aside and reuse objects within individual functions. We're talking 10x speed improvements by making sure that garbage created per-iteration is near zero as opposed to just making a mess and letting GC sort it out.

These are objects that have basically no-op constructors, and never stay live for more than a method call, which makes me think that the observed speed boosts must have something to do with GC. Am I missing something?


Yes, vectors is one of those cases where stack allocation has a significant advantage. This will hopefully be resolved (along with cache related issues) when value types are introduced, hopefully in Java 9.


Sadly I'm not holding my breath. For the past ten years we've been upvoting requests for value types, being told that they were not necessary because the JVM was brilliant already, then after much argument and ample proof being told that stack allocation in the JVM was a better solution, then having it delayed, finding out that the implementation didn't really work that well, etc. Sun has been pretty shit about prioritizing this issue, perhaps Oracle will be better, but it's a far shittier company than even Sun was so I'm skeptical.

It's also possible I'm jusy raging because this is a problem that has been well known for well over a decade but has been played down by the folks in charge of the JVM every time it comes up, even though it remains a real problem with the platform.


The difference is that this time you hear Doug Lea and Brian Goetz talking about this as if it's definitely going into Java 9. This time it's not coming from users but form Oracle. Also theres a JEP and a prototype by John Rose. In fact, it is mention by Oracle as one of the first features of the core platforms planned for 9. This has never been the case before.


By having most frequently used data grouped on, as you note, "small" stack instead spread all over heap you do have much better cache use. Moreover, even if the allocation is fast, if there are a lot of them, they just add up up vs. the case where there is nothing to be done when everything precalculated during the compiling phase.

If you haven't witnessed these effects I suspect it's only because you haven't had the reference to compare with.


I believe that when people complain about "lack of stack allocation" in Java they sometimes mean "lack of value/non-reference types," which in other languages (at least C#) are frequently but not always stored on the stack.


Oracle might want to offer a survey tool (rather than a SurveyMonkey) which analyzes a codebase (or even optionally tallies dynamic usage).

The reports could in a standard format, and reviewed by respondents before submission - letting the respondent add comments or elide sensitive info.

This could catch a lot more usage – including deep-hidden in third-party libraries or old code.


They have a tool for you to use. It is called jdeps.

http://download.java.net/jdk8/docs/technotes/tools/windows/j...

So you will have Java 8 timeframe to make your code behave nicely.


I found a good article explaining all it's power. The Multiple Inheritance trick is cool.

http://mishadoff.github.io/blog/java-magic-part-4-sun-dot-mi...



LOL. I actually laughed IRL when I read the article title. Java is a language that's supposed to have encapsulation. Now they are wondering if developers are breaking encapsulation to the point that they need a survey? What a joke.

Java's like the language that's supposed to have encapsulation and types, but actually has neither in practice. (everyone uses reflection / casting)


I didn't down vote you, but I would like to take a moment to respond to your statements.

The survey is due to some people such as library makers using this feature to make really advanced, complex, rule-breaking things in Java. It wasn't intended to get out, but it is. So Oracle, and by extension OpenJDK, want to know what would happen if they got rid of it. Presumably they would replace it with a more open API.

Now as to encapsulation and types not being used, I've seldom seen that be the case pretty much ever. Normal developers will use casting, yes, but probably not reflection directly. They will use libraries that will use reflection, sure. For example, few on my team use reflection directly (I know, we've talked, they've been confused because they've never done it). I'm the team lead. I use it a lot. We have data parsers. That data needs to get mapped into a canonical model. My code, the mapper factory, uses reflection and naming standards and code generation to automatically pick the proper mapper based on the inbound parsed object. So my code really does have an API of "interface Converter:public Object mapper(Object)". Behind the scenes, using the reflection API, the first step is to determine the parsed object's package. Replace a keyword in that package to find another package for the mapper and the reflectively load the mapper that implements that interface if it doesn't exist. Thus the SingletonFactoryFactory of Java!

But following that scant 50-100 lines of code is about 15k of plan old, getString->setString, auto-generated mapper code. Boring, encapsulated, typed. And all of that allows me to know if the mapping documents the BAs created are good. If they compile, they're good. If not, got to talk to the BAs.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: