I agree it's important to think about; and that morally you can't just take benefits - harms = net good. It's reasonable at least to be more morally concerned about avoiding harm than about doing good. That said, I've never heard a plausible story about how someone like Matt is causing harm in his career (unlike he might be doing if he worked for tobacco companies, for example).