Don't confuse power and legitimacy.
This is one of the major reasons why India isn't ruled by the British now.
British had power, Gandhi and the peaceful revolutionists had legitimacy... guns can only project so much power.
You might say that morally it should come from consent, but descriptively, legitimacy comes from having a monopoly on force and not being totally, utterly hated.
The moment they loose consent they become the oppressors.