There did used to be a way of bridging from applet into javascript. It was introduced in netscape, and IE added support. It allowed you to have business logic in java (including custom jars, such as for middleware) sending instruction to the DOM layer. But firefox didn't reimplement it, and it was dropped from IE too. I worked on a business system that worked like this for a while. I think they rewrote it later to use webstart and swing.
It is certainly possible to communicate between Java applets and JavaScript, and in a widely portable way. One of my current projects does it all the time, for much the reason you suggest: it means I can have the model in a well-structured Java system, but branch out to JavaScript for the parts of the UI where it is helpful.
To call from JavaScript to Java, you can just use the public members of the applet object.
To call from Java to JavaScript, you can use JSObject.
Aside from working around a few minor gremlins in particular browsers, most of the basic things you'd want in this context "just work". In practice, when I'm transferring big chunks of highly structured data from the Java model to views using JavaScript, I use an intermediate layer in the applet that looks up all the relevant data Java-side and builds a JSON representation that is trivial to work with in JS, which avoids any trickiness with things like marshalling arrays and means rendering specialist types in Java world into suitable strings or objects in JS is all done in one place according to consistent rules. This all runs fine on IE, Firefox, Chrome, etc.
The biggest practical problem I've found is that because relatively few people are doing this for major projects, there is also relatively little documentation around on the web (including the official Java pages and the developer pages for the various browsers). That means if you do run into one of the awkward cases, it's not always a five-second search to find the answer. Still, so far I've been doing this for over a year, using the same techniques in a couple of different contexts, and most of the problems with "incompatibility" have quickly been resolved by updating to the latest versions of things.
