Another possible output for this type of task is an opinion or interpretation and a justification of it. A machine might feasibly be considered "good" if it output an unpopular interpretation with a good justification.
Great point! The same type of argument is going in the AI subfield of computational creativity. A computational system might produce a piece (of art or music, say) which nobody initially likes, but could provide a justification/interpretation which might be convincing. In some ways this would be a big improvement over a pure Turing test-style judgement of the output itself. I think Margaret Boden has written this idea into her definition of computational creativity.
EDIT: "The ultimate vindication of AI-creativity would be a program that generated novel ideas which initially perplexed or even repelled us, but which was able to persuade us that they were indeed valuable.", from Boden, Creativity and Artificial Intelligence, 1998. I believe, based on her other writings, that she thinks of it as a sufficient, but not a necessary condition, for AI creativity.
