Hacker News new | past | comments | ask | show | jobs | submit login

I wonder how many of these 200 people were from India, which appears to be supplying a good chunk of Mech Turk userbase.

I also ran a couple of polls with similar intent on MT and then stopped doing that because results were skewed dramatically. For one there's no filtering by a geographical region and you bet your 10c-per-answer that you will get a ton of replies from the poorest corners of the world. Secondly, responders are afraid of you rejecting their answer and thus affecting their MT rating, so they tend to tell you want you want to hear and not what they actually might be thinking. Hence your 73% approval rate.

This sampling is not random, very far from it. You are effectively sampling MT community with its own dynamics, the community that is likely NOT to be representative of your own target user base.




I've been using CrowdFlower as an interface to MTurk, and it let me exclude specific countries. Your point about the secondary motivation is unfortunately valid, though.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: