I only did 12 of them before exiting but got 100% of them. I watched them on my phone without turning it landscape to simulate how I'd normally watch a brief news clip like this.
They're still not particularly convincing, especially the audio ones sound very fake.
But I think these sorts of tests are flawed. The fact that it's a test primes you to look for these details in a way you might not if these clips were shared on Twitter/Reddit/etc. The deepfakes are still pretty obviously fake, so I don't think I'd fall for one "in the wild". But I also see this style of test used for things like "can you hear the difference between these 2 audio compression formats" or "does this virtual guitar amp sound just like the real thing", and that's just not how you consume real media. For example I'd probably never know Meshuggah switched to entirely digital guitar amps years ago if it weren't for them saying it in interviews.
When deepfakes one day do get as close to the real thing as digital guitar amps have gotten to real amps, I doubt I'd notice a random r/worldnews post is faked, even if I could still reliably pass an A/B test.
This is the real test. While we (HackerNews) readers might be able to realize they are fake, that other 90% of the world won't.
The reverse side of it is if an overwhelming amount of deep fakes get called out, people will stop trusting the real ones (we have already started stepping into that reality)
Perhaps you detected them because you knew there would be 50% fakes? If you'd see one of these fakes in a CNN article, would you still reliably detect it? I think the role of the prior expectation should not be underestimated.
I think it's one of those studies where they say they are testing for X but they are actually testing for Y. Not sure what Y could be. But I don't even live in the US or consume it's media, and could still figure out all of the examples I saw/listened.
They're still not particularly convincing, especially the audio ones sound very fake.
But I think these sorts of tests are flawed. The fact that it's a test primes you to look for these details in a way you might not if these clips were shared on Twitter/Reddit/etc. The deepfakes are still pretty obviously fake, so I don't think I'd fall for one "in the wild". But I also see this style of test used for things like "can you hear the difference between these 2 audio compression formats" or "does this virtual guitar amp sound just like the real thing", and that's just not how you consume real media. For example I'd probably never know Meshuggah switched to entirely digital guitar amps years ago if it weren't for them saying it in interviews.
When deepfakes one day do get as close to the real thing as digital guitar amps have gotten to real amps, I doubt I'd notice a random r/worldnews post is faked, even if I could still reliably pass an A/B test.