GPT-4 is an amazing achievement, however, it is just a language model. LLM (large language models) are well documented in literature and GPT-4 is just a much larger version (more parameters) of these LLM models. Training of LLM models is also well documented. GPT-4 just has been trained on a very large subset of the Internet.
Of course there are proprietary models, that will be improved versions of the academic LLM models, however, there are no big secrets or mysteries.
The individual components are well documented, but which specific arrangements produce the best results is still very much an active research area.
As far as training, the differences between GPT-3 and GPT-3.5 (the latter being a smaller model!) demonstrate just how much fine tuning and reinforcement learning is important to the quality of the model. Merely throwing more content from the Internet at it doesn't automatically improve things.
Exactly, this has been my guess as well. They must have trained the model specifically to write poems, haikus and other things. So that the output looks much more polished than it really is.
Of course there are proprietary models, that will be improved versions of the academic LLM models, however, there are no big secrets or mysteries.