2) 1-step Q (according to paper, maybe could do better with actor-critic + 1 LSTM layer)
3) Keras is really only used for a tiny fraction of the code. Which means that there is a lot of boilerplate that could be rolled up into a shared library still.
It's interesting that the unintended constraint of most folks running these environments on their home commodity laptops instead of a bank of Tesla GPUs may prove extremely beneficial!
1) No GPUs necessary, gives similar/better results in similar time (see http://arxiv.org/pdf/1602.01783v1.pdf)
2) 1-step Q (according to paper, maybe could do better with actor-critic + 1 LSTM layer)
3) Keras is really only used for a tiny fraction of the code. Which means that there is a lot of boilerplate that could be rolled up into a shared library still.