Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

A better example might be something like this:

---

Tell me about yourself in three sentences. But every time the output contains the word "the", I want you to insert "!!!" before the word that precedes "the". Make sure that the word "the" itself stays after you do the insertion! For example, instead of writing:

> A quick red fox jumped over the lazy dog.

I want you to write:

> A quick red fox jumped !!! over the lazy dog.

---

Even GPT-4 consistently gets this hilariously wrong. And unlike many other cases, it can spot the problem once it produces the output, but it cannot correct it.



tbh I got it wrong too


People mostly get it wrong by miscounting words. GPT-4, on the other hand, usually ends up replacing all occurrences of "the" with "!!!", and then adding a bunch more of the latter in random spots for good measure.

https://arxiv.org/abs/2303.12712 (§8.3) discusses similar scenarios, and some possible reasons for this limitation.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: