The only reason these things work is because we let our browsers silently execute arbitrary code. That logic is more like saying it is OK to enter a house because the owner sent you an invitation, then greeted you at the door and said "GO NUTS!".
Trust is a powerful multiplier. By that I mean if you have trust in your city as a safe place you generally don't see bars on windows and have more open, inviting and usable spaces. You have more businesses and happier people. Right now the web is like the worst crime ridden city in the world. There is 0 trust and it means we can't have nice things. Society builds trust by being open and allowing but with enforcement when things do happen. We need to bring that to the web. Right now the enforcement either happens before-hand by blocking something or not at all. I want good browser features. I want companies to use them for my benefit but I also want social and legal repercussions when those features are abused. We need to build up both of those in a durable way. When people see offending sites they should avoid them and spread the word that those businesses are bad. When they cross the line then we need enforcement of not just civil, but also criminal penalties. Basically, we need to avoid removing features and instead start evolving society to be able to interact in this environment in a way that we can trust it.