The top answers are essentially well-thought-out probability arguments describing the Lizardman Constant [1]. Essentially, random humans who don't care very much about your survey are likely to respond in arbitrary or perverse ways, making them a very noisy data source. Small signals (e.g., 10% or less of a sample) are likely to be meaningless. Its nice to know that there's a solid argument that this doesn't compromise the validity of large signals, at least.

[1] Which for some reason doesn't have a Wikipedia page, even though it's a phrase that seems to turn up a lot in certain circles. I suppose the best reference is http://slatestarcodex.com/2013/04/12/noisy-poll-results-and-....