> and it was only in the last decade that the vanishing gradients problem was tamed.

One of the big pieces was Schmidhuber's lab's highway nets, done ~30 years ago, but just didn't land until a more limited version was rediscovered.