I find that at a practical level, linear algebra and probability/stochastic processes are the most valuable and heavily used topics in machine learning. For Linear algebra, i'd recommend Gilbert Strang's book+his MIT OCW lectures. Check out Papoulis's text for probability. It is very dense, and packs in lots of insight per page.
Thank you, fortunately I was able to find an economy edition of both books (Linear Algebra and its applications by Strang though) after reading your post, so I have lots of work to do now. Also I don't know how I missed probability in my original post. That has been one of my weak areas and definitely needs fixing.
