Thanks for sharing this. I've been interested in Haskell for a while and I hadn't really started writing any actual programs until very recently[1]. I also wanted to learn about neural networks, and reading your source code was quite pleasant. It looks really interesting, congratulations.
off-topic question: is it possible to speed up a neuron network software using a gpu or specialized hardware running openCL ? what about using supercomputers ?
Actually, that's exactly how they speed up neural networks, especially deep neural networks. Andrew Ng showed that they could run the Google Brain on COTS GPUs for about $21,000 [1].
You can experiment with this yourself using a package like Theano (Python) [2] or Caffe (C++) [3].
[1] I wrote a toy IRC bot for a channel I have with some friends. It's quite primitive. https://github.com/jdiez17/HaskellHawk