I made similar chess engine using FastText: https://github.com/thomasahle/fastchess

It is surprising to me that you can predict optimal/strong engine moves with 27% accuracy using a completely trivial linear model, that is by a single matrix multiplication.

I wonder how well it would compete with this GPT-2 engine.