That's how most law works. I have to give up my right to murder someone in order to enjoy a society where it's illegal for everyone.
If you believe privacy not inspectable by law enforcement is wrong the prerequisite is saying that you're willing to have the the law apply to you as well.
I believe that privacy not inspectable by law enforcement is a fundamental right. I'm willing to accept that aids some crimes but also willing to change my mind if the latter becomes too much of a problem. It doesn't seem to be the case at all ATM.
Yes, that is my position. E2EE back-doors might not affect my communications or yours, but have serious and undesirable repercussions for some journalists and whistleblowers. The thing is, regular people aren't going to tolerate a sustained parade of news stories in which E2EE helps the world's worst people to evade justice.