I think a good start would be getting rid of the feminism-as-status-symbol problem. That guy who dismissed you because you're not a woman and therefore privileged was expressing a belief that's most likely just signalling his status as "enlightened person". If we can stop that, we can start having a movement towards equality that doesn't hurt both genders more than it helps them.