I'm not an expert but it seems these perturbations are fiddling with the fundamental approach used with a NN. Mainly, that a NN works in layers. So these perturbations must be messing up the lowest layers and then the higher layers end up generating the wrong features and ultimately the model misclassifys. See http://i.stack.imgur.com/jpYdN.png