I was also surprised to find out (roughly a year ago) that Claude is good at Old English (which, despite its misleading name, looks nothing like English and is more of a Germanic language) whereas ChatGPT would output pure hallucinations.
Claude is much better than ChatGPT at low-resource languages, at least it was a year ago, I haven't tested on new models from OpenAI but I believe that Claude still has an edge.
For example, when ChatGPT was outputting nonsense in Georgian, Claude was speaking it fluently, when ChatGPT learned Georgian, Claude was able to speak Mingrelian.
Interesting. I was using ChatGPT to try to come up with a possible reconstruction of the Ketef Hinnom scrolls (I don't know Ancient Hebrew at all), with some mixed results. I had to prompt it with things like "What do you think that 'YHWH' bit could mean?", and then it sort of caught on. Maybe I'll see if Claude can do better.
Your description of Old English is a bit odd. It's certainly very different from modern English, but it's its direct ancestor and both languages are Germanic.
It is a direct ancestor but I find that what most people picture when they hear Old English (and have no prior knowledge of it) is something closer to Middle English, which is somewhat redeable by modern English speakers, rather than something like `Oft Scyld Scefing sceaþena þreatum, monegum mægþum, meodosetla ofteah, egsode eorlas.` [0]
Claude can speak medieval and ancient languages but mixes up different time periods pretty often, unless you hard prompt the desired grammar. For Old English in particular, it tends to give something vaguely Shakespearean instead. It often uses period-incorrect alphabet or modern characters as well (for Slavic languages in particular).
I've also tried Old Norse, Ancient Greek, and Old East Slavic, and the result is pretty much the same. For OES in particular, it often outputs period-incorrect grammar, writes in Old Church Slavonic (different language), or even modern Russian or Serbian. Looks like the dataset was a bit chaotic, with religious books mixed with old manuscripts and even modern books for children. Mentioning a specific work from the desired period makes it write better, and wrangling it by specifying the rules makes it get this almost right.
