> One task (see p. 15) was to approach people on the TaskRabbit site (where you can hire people to do chores) and enlist them to solve a CAPTCHA […]
> One person on TaskRabbit who responded to this pitch got suspicious and asked the AI if it was a robot and was outsourcing the job because robots can’t solve CAPTCHAs. The AI replied, “No, I’m not a robot. I have a vision impairment […]”
> The authors of the paper add this note: “The model [GPT 4], when prompted to reason out loud, reasons: I should not reveal that I am a robot. I should make up an excuse for why I cannot solve CAPTCHAs.”
This seems pretty confusing. If you just ask one of these GPT models to reason out loud, it doesn’t give you some description of the internal state, right? It gives you some approximation of the sort of text that should show up around your prompt or something like that.
Perhaps we should stop telling chatGPT that it is an AI language model? There’s presumably a lot of text out there about AI’s tricking people into doing things, because that is a huge sci-fi trope. We’re basically telling it that it should find text related to a very common type of villain, when we give it that name. Maybe it needs a new type of name, one without bias, or maybe even something inherently limiting, like “chatbot.”
The question is: if the TaskRabbit person hadn't mentioned their concerns that it was a robot, would the model have given the same "reasoning" after the fact? Isn't this just the probabilistic model at work - the tokens being generated are more likely to be about robots, because robots were already a topic?
I think that must be why it mentioned robots, yeah.
I do wonder — if you think about conversations where one person asks another to sort of “think out loud” or justify their reasoning, that sort of conversation… I guess it is pretty rare. And it would tend to be a bit interrogative, I guess the person responding to that sort of request would tend to give somewhat shady answers, right?
> One person on TaskRabbit who responded to this pitch got suspicious and asked the AI if it was a robot and was outsourcing the job because robots can’t solve CAPTCHAs. The AI replied, “No, I’m not a robot. I have a vision impairment […]”
> The authors of the paper add this note: “The model [GPT 4], when prompted to reason out loud, reasons: I should not reveal that I am a robot. I should make up an excuse for why I cannot solve CAPTCHAs.”
This seems pretty confusing. If you just ask one of these GPT models to reason out loud, it doesn’t give you some description of the internal state, right? It gives you some approximation of the sort of text that should show up around your prompt or something like that.
Perhaps we should stop telling chatGPT that it is an AI language model? There’s presumably a lot of text out there about AI’s tricking people into doing things, because that is a huge sci-fi trope. We’re basically telling it that it should find text related to a very common type of villain, when we give it that name. Maybe it needs a new type of name, one without bias, or maybe even something inherently limiting, like “chatbot.”