A lot of floating point operations are done in GMM likelihood calculations. Dynamic decoding with Viterbi beam search and language model queries are other operations. We pool some memory but we do not try hard for this. Allocation is cheap in Java. Cache aware design saved a lot of performance for us.