> Notably, participants were more likely to judge AI-generated poems as human-authored than actual human-authored poems
There is clearly a significant difference between AI generated poems and human generated poems.
A random group of people probably do not read poetry. It would be be interesting to see what people who do read poetry regularly do on this. Also, which they rate as good, rather than just "human authored".
I find with both the little AI generated poetry, and the AI generated paintings that show up in my FB feed, both look a bit rubbish. FB is pretty good for experiencing this because the work (usually an image) shows before the cues that it is AI generated in the text and comments.
As someone who reads poetry regularly and has played around a bit with AI-generated poems, AI poems can be quite impressive, but have a certain blandness to them. I can see them conforming very well to the average person's concept of what a poem is, whereas the human written poems might be less pleasingly perfect and average, more stylistically and structurally experimental etc. The LLM version is less likely to play with the rules and expectations for the medium, and more likely to produce something average and "correct", which makes some intuitive sense given the way it works.
I think when asked to rate poetry as human or ai authored that human poetry does look more like a random smattering of semi-related words which, I assume, is what folks think machine generated poetry would look like.
Perfect grammar and perfect meter doesn't read as AI to most folks yet.
LLM’s also have no life experience, so they can wrote poems, but those poems aren’t communicating anything real. Poetry in the vein of Whitman and Dickinson and Plath is very much about a person expressing their very personal experiences and emotions.
I'm reminded of how people are bad at generating and recognizing truly random patterns. I imagine the famous poets have something in their writing that's an outlier. I wonder if the human-authored poetry looks odd enough to cause problems with our fake detectors, while the mediocre grey goo that AI creates better fits expectations.
Anecdote incoming - I read poetry, weekly if you will, over about 15 years now.
I also play with LLM's often, for creative side projects and work commercially with them (prompt engineering stuff)
I don't find it far fetched that individual poetry can be indistinguishable at times when AI generated. I was asking it to write in iambic pentameter (sonnets) and it consistenly got the structure right, it's approach to the provided themes were as complicated or glib as I wanted. But that's all subjective right, which leads me to my main point.
My view of poetry over the years, has always been centred around the poet, the poet living in a time and place. As a generalisation most people buy into the artists life because it may represent some part of themself.
If someone managed to write an intriguing corpus of texts using LLM's that was extolled, I think that would almost be besides the point. What is important is the narrators life, ups and downs, joys and woes. Their ability to convey a memorable story even heavily relying on AI would still be impressive. Anyway sounding a bit wanky I will stop lol
(I do think LLM's write a little too perfect and that is easy to think it is not human, but you can kinda prompt them to throw in errors too so who knows)
> Notably, participants were more likely to judge AI-generated poems as human-authored than actual human-authored poems
There is clearly a significant difference between AI generated poems and human generated poems.
A random group of people probably do not read poetry. It would be be interesting to see what people who do read poetry regularly do on this. Also, which they rate as good, rather than just "human authored".
I find with both the little AI generated poetry, and the AI generated paintings that show up in my FB feed, both look a bit rubbish. FB is pretty good for experiencing this because the work (usually an image) shows before the cues that it is AI generated in the text and comments.