According to an older ocw version of this course [1], you need "General ease with algorithms, elementary number theory and discrete probability". You can learn these from MIT's own ocw's excellent "Mathematics for Computer Science" [2] and "Introduction to Algorithms" [3].
For a gentler intro to crypto, I am planning to do Cristof Paar's course first, given that he takes his time to carefully explain all the math; there is a youtube list with all the lectures [4] and a course webpage [5]. In case any further encouragement is needed, according to a blog posting "Matasano Crypto Challenges, Set 5" [6]: "This set was surprisingly easy, actually. The book Understanding Cryptography by Paar & Pelzl is an excellent intro to the basic maths needed for crypto — namely, the group theory and number theory necessary for RSA and Diffie-Hellman."
Now that I am at it, here are some HN links you may find helpful if planning to learn crypto, in no particular order:
Rolling Your Own Crypto (DON'T!) [7]
Telegram’s Cryptanalysis Contest [8]
The most dangerous code in the world [9]
BearSSL – Smaller SSL/TLS [10]
NaCl: DJB’s new crypto library [11]
JavaScript Cryptography Considered Harmful [12]
Cryptographic Right Answers [13]
Ask HN: Current Crypto Best Practices [14]
Mathematical Cryptology [15]
Ask HN: What are the best books for learning information security? [16]
Of course, there are also the courses Bohne's CS255, Rivest's 6.857 and many others, and the books Cryptography Engineering, Introduction to Modern Cryptography etc ... Harvard's [pdf] https://people.seas.harvard.edu/~salil/cs127/fall13/conclusi... is a good guide.
