This is really cool. I had a similar thought that GPT3 could be used to simulate political polling. A few weeks ago I tried telling GPT3 that it was part of a specific demographic (age, gender, race, income, political leaning, etc) and then asked it how it would respond to certain political questions (I tried gun control, immigration, abortion and some other issues). GPT3 was able to change its answers in believable ways depending on what demographic I instructed it to be.
My thinking was that this could be used as a quick polling test to see how the real population may respond to certain new ideas.
More work would need to be done to calibrate it, as without specific demographic details the answers tended to be liberal leaning. But its an interesting idea which could be used to create instant focus tests on any number of topics.
> GPT3 was able to change its answers in believable ways depending on what demographic I instructed it to be.
Doesn't this just mean that your own preconceptions about those demographics matches the language models preconceptions? How would we know that is matches reality when presented with novel ideas/concepts that we want to get feedback on?
That's part of what I mean by it needing to be calibrated. Initially some polling could be done with real people and the GPT agents. Whatever calibration factors are needed to make those two line up could then be used when asking the GPT agents novel questions.
My thinking was that this could be used as a quick polling test to see how the real population may respond to certain new ideas.
More work would need to be done to calibrate it, as without specific demographic details the answers tended to be liberal leaning. But its an interesting idea which could be used to create instant focus tests on any number of topics.