It can invent words, and then correctly use them to compose.

https://news.ycombinator.com/item?id=35268950

it can invent words, because gpt is amazing pattern extrapolator.

Okay, but who "taught it to bark" correctly in response to those words?

you can read papers how gpt was trained?

You're making a specific claim here: "it was taught to react in specific way on specific word", and I'm giving you a specific example of an invented word that GPT used to full effect when composing. Can you explain how it was "taught to react" to this word in a manner that enables it to use it in the new meaning that wasn't even in its training set?

I answered you, gpt is pattern extrapolator and was taught to be so, creating new word is totally fits pattern extrapolation.

How about using that invented word consistently afterwards?

It seems to me that the argument you're trying to make can be extended more or less indefinitely, though. But there's a point at which "it's a pattern extrapolator" becomes less of an explanation and more of a largely irrelevant piece of trivia.

Well if it can bark out the right answer its an impressive damned dog...

the thing it doesn't bark right answer very often, but bark non-sense because was training on predicting next word in sentence.