At least for the optimization work I've done I've always had "real-world" test cases, and "drive the current bottleneck really hard" synthetic tests (I profile the real-world case, identify some bottlenecks, make tests that target those bottlenecks then optimize for that, then rinse and repeat), so I can understand why the various JS benchmark suites contain some very synthetic tests...
Except, unfortunately, they encourage people to keep optimizing that one issue even when it's long past being the bottleneck. Equally, what's a bottleneck in one implementation might not be in another.
Having the entire real-world case means you have to optimize for the actual case, and not just implement what was at one time the bottleneck in one implementation as quick as possible.
IIRC if you use Emscripten to compile Bullet instead of using Mandreel (the closed-source compiler they used), it's faster in FF than Chrome. Probably explains the choice of a closed source tool there.
Do you happen to know which of the Empscripten or Mandreel compiled versions is fastest?
(After searching a bit, it seems that the emscripten version is the faster one[1], at least in February, but the testing methodology in that post isn't clear.
The main problem is that Mandreel is closed source. I don't have access to it, so I can't just compile two C++ benchmarks and compare the results.
However, I suspect the Emscripten version is faster, for a few reasons:
* Indirect measurements on Box2D in that link indicate Emscripten is faster.
* Looking at the generated code shows several things Emscripten does better, for example it uses the native JS stack for function calls (Mandreel pushes all the arguments onto the emulated C stack which is additional work), the Emscripten relooper implementation is more optimized, etc.
* Emscripten uses closure compiler to minify and further optimize the generated code while Mandreel does not. This mainly affects download size, but not only - closure can do optimizations at the JS level that can't be done before, like for example inlining handwritten glue code in JS into compiled JS code.
* Mandreel used to use an older version of LLVM and Clang than Emscripten, which always uses the latest, and there have been big 'free' speedups in Emscripten due to improvements in the LLVM optimizer. (But, perhaps Mandreel has updated, without access to it I don't know.)
* Emscripten has gotten a lot of issues filed about performance issues, pull requests, etc. as a benefit of being open source, which led to plenty of improvements. I can't be sure but I am guessing there are fewer people that bought a Mandreel license and contributed feedback.
Note though that the question of which is fastest is a little problematic - one compiler's output might be faster in one browser but slower in another, so there isn't an objective sense of "fastest" here. But with that said, there are aspects of the code that should be faster in all browsers, like as mentioned above the more optimized way that Emscripten does function calls.
I think it helps somewhat that, for different reasons, this is not really about the money for either one of them: Firefox is a nonprofit organization, and Chrome is not Google's main product.
In some ways it is about money, Google pays Mozilla about $300 million a year for search engine placement and click-through ads. That accounts for the vast majority of all of Mozilla's revenue (in 2010, Google paid $115 million and that was 85% of Mozilla's revenue).
It might also help a little that Mozilla depends mostly on Google for revenue :-) ...but google has stated before that basically "anyone in favor of improving the web is on our side"
I think it makes it all the more admirable, though arguably Chrome benefits Google pretty well. They build better apps. You spend more time on their property viewing their ads. They build more HTML5 functionality that makes their ads even more engrossing. They get your homepage, etc.
