I'm answering both you and the previous one in the chain. I used huggingface's OpenAI detector[0], the pizza example is detected as fake with 99.98% probability, whereas the Battle of Agincourt text is also reported as fake with over 99% probability. I don't have any example at hand of how much "real" text on these topics to try and see what that model detects, but up to this point[1] this text is reported as 99% real. You can play and see how much % it assigns to different examples.[2]
[1] I had to cut somewhere to copy and paste into the detector, so [1] is the cutting point
[2] My whole response was still detected as 99.98% real, and copy-pasting the Agincourt bit at the end still reported as 99.98% real. However, moving it to the start (having my answer as a last paragraph) made it detect as 97.71% real. Deleting only "huggingface's" balanced it as 54% real. You can play around with the demo to test different variations of any text you analyze with it.
Very interesting, I didn't know that detector. But making small local changes, it's easy to make the probability of "fake" go down, while still benefitting from the automatically-written essay. As long as students have access to the same tools as teachers, it will be easy for them to get away with automatically generated essays.
You play with ChatGPT a bit and you do start to see certain patterns in the results. So I guess if you train on the outputs it gets relatively straightforward to recognize other examples that are just cut and pasted from the ChatGPT--even if, in isolation, it seems like something a person might write.
The "weird" (or "funny" or whatever you want to call it) thing is that the detector I linked was originally made for GPT-2, but still works reasonably well for GPT-3 (and derived) output.
[0] https://huggingface.co/openai-detector
[1] I had to cut somewhere to copy and paste into the detector, so [1] is the cutting point
[2] My whole response was still detected as 99.98% real, and copy-pasting the Agincourt bit at the end still reported as 99.98% real. However, moving it to the start (having my answer as a last paragraph) made it detect as 97.71% real. Deleting only "huggingface's" balanced it as 54% real. You can play around with the demo to test different variations of any text you analyze with it.