A society in which most people were walking around stoned, drunk or on hallucinogens is not consistent with what most people would consider a civic or moral society and the negative aspects wouldn't need 'explaining' as it would be self evident.
Show up drunk or stoned to your friends kid's birthday, to a wedding or a funeral, to work, to meetings with clients, to the DMV, to your parents Christmas meet-and-greet, to your kids school play, to the PTA meeting, and then ponder that and you'll have your answer.
If you live in US, it is quite likely that you're interacting with people who are under influence of some drug (usually cannabis) on a daily basis, without realizing that.