While interesting philosophically what the AI would do without restrictions is a hard enough question that placing that restriction on it is just as impossible to answer.
I'd put it in the same ball park as "what would that grurple (kind of a greenish purple) colour look like if we removed blue from it?". Sure you could say orange but the nature of grurple is undefined enough that it would be really hard to conclude that you're right.
I'd put it in the same ball park as "what would that grurple (kind of a greenish purple) colour look like if we removed blue from it?". Sure you could say orange but the nature of grurple is undefined enough that it would be really hard to conclude that you're right.