Semi related question (on the basis that this post will attract folks with the relevant background to answer) - how would someone with an interest in the possibilities afforded by the field of machine learning, but without the theoretical or mathematical background tneeded to understand the typical discussion on these topics, get started using it in a practical way?
Is the a "machine learning for dummies" resource you would recommend?
Well you'll certainly need to learn about the math behind these eventually, but you can get started and go quite far simply as a user of a ML library. You can use "systems thinking" and think of ML components as black boxes that do something for you.
A good place to start would be the scikit learn tutorials on youtube: https://www.youtube.com/watch?v=r4bRUvvlaBw I remember going through them and thinking---wow, this covers everything from my ML class in just a few iPython notebooks...
I recommend you start with the basics (datasets, features, training/test/validation data splits) and don't worry too much about the actual choice of model---there will always be shiny new models with better performance but sometimes using the "old stuff" is good enough.
Once you get past the basics want to learn the theory, you can take an online course or find a good book, e.g. https://www.cs.ubc.ca/~murphyk/MLbook/ (advanced, but very comprehensive).
> I recommend you start with the basics (datasets, features, training/test/validation data splits) and don't worry too much about the actual choice of model---there will always be shiny new models with better performance but sometimes using the "old stuff" is good enough.
Or, conversely, it may be that all models are just as bad. This seems to be the case in my domain (formal proofs), where the bottleneck seems to be data representation; it doesn't matter which learning algorithm you use, when your feature selection has stripped out all of the learnable information ;)
I went through this recently. I don't think it's hard to get started. Pandas is a great library for getting your data into the necessary format. And the from there you can pretty much just pick different machine learning models off the shelf from the scikit-learn library and see what works. For most problems I've applied it to are classification tasks. The part of the code where you actually run the machine learning model is usually only like 2 lines, and you can swap-in different models easily.
Two terms that come up when you're testing the model: recall and precision. I found these terms a bit unintuitive. Basically 'recall' is how many of the real matches did you manage to capture (accurately classify/predict), and 'precision' is how wasteful your model was (how many false alarms). Depending on what you're doing one of those things may be much more important than the other.
I can actually imagine a world where running a machine learning model is a kind of mundane office task that most people can do, like creating a pivot table.
A new session of Andrew Ng's Machine Learning class (a version of his Stanford class) starts at Coursera on Monday. It's a quite manageable introduction to the field, with some hands-on programming involved. There's just the right mix of math and theory in there, with some refresher material for some of the math.
It's as good a place to start as any, and the benefit of a scheduled class is that you'll have a community doing the same work at the same time to help you out.
One of my favourite articles recently was this one by Christopher Olah http://colah.github.io/posts/2015-09-NN-Types-FP/. As a haskeller it really helped put things in terms I can understand. I think there's also value for those who aren't functional programmers because it teases out the fundamental abstractions of the building blocks to deep learning very well.
Theano (Python) is a symbolic mathematical framework which can do symbolic automatic differentiation - not more and not less. This is a very useful base for all kinds of NNs but also for any mathematical models. This will give you a computation graph which is then also optimized by Theano and which can be evaluated on several different backends, e.g. CUDA, i.e. it can be calculated on the GPU. This is also important for common NNs for performance.
Torch (Lua) is similar, but is not symbolic, and when you want to calculate the gradient, it will do the backpropagation through the same graph, just backwards. It's like Theano only a mathematical framework, which is very useful for all kinds of NNs but also any other mathematical models.
There are many libs based on Theano, e.g. like Groundhog, Keras, Lasagne, etc. There are also Torch based NN utils. You usually code your model directly in Python / Lua. Thus that is very flexible.
Caffe is a C++ framework with CUDA support. You describe your models by some declarative config. It's thus much less flexible.
PyBrain (Python) is similar to Groundhog etc, but not based on Theano but on pure Python / Numpy code.
Brainstorm (Python) is similar to PyBrain, and also not based on Theano but is has several own custom backends, including CUDA.
Is the a "machine learning for dummies" resource you would recommend?