Neural networks! Bah! If I wanted a black box design that I don't understand, I would make one! I want rules and symbolic processing that offers repeatable results and expected outcomes!
...and maybe there's still a place for that.
But for someone who has been possibly foolish and ignoring neural network-oriented AI, where's the best place to start learning?
Go back to the basics of recurrent neural networks, deep learning texts, and so forth? Or is there a shortcut into the hip and popular transformer-based technology at this point?
Logically, predictions are extremely simple: data (encoded in some way) goes in, answer (encoded in some other way) comes out. As far as training goes, the modern and “useful” models are so big that you cannot train them yourself anyway.
This simply means there are 2 very separate ways to approach them:
-If you want to understand the internals, I absolutely suggest to go the traditional way: start with linear algebra/ vector spaces, then understand how MLPs work, then CNNs, (by now I would skip RNNs), and finally transformers. Other important topics include: latent spaces/embeddings/autoencoders/etc
This is almost academic knowledge though.
-On the other hand, if you want to play with them, really all you need is: first learn python (if you don’t know it already), and one DeepLearning library (probably Pytorch). Then go to Huggingface, and download some models. You pretty quickly get a feeling of what are the common formats for models and data, and you can start putting them together..
Even many of the popular AI papers today are creative ways of plugging the output of some model into the input (or training objective) of some other..
This is my 2 cents anyway!