The hostility isn't to sexual dimorphism per se. If only because of hormones and reproduction, there is clearly sexual dimorphism. Anybody can see that.

The hostility is to the tendency to attribute differences between men and women to sheer biology without a careful consideration of other factors.