That's because we're currently largely not using them correctly, i.e. hooked up to RAG instead of hoping that they've memorized enough of the training data verbatim, which is arguably a waste of neurons in a foundational model.

Imaging being graded on your ability to quote exact line numbers of particular parts of your codebase as a senior software engineer without being able to look at it!

LLMs are not, in isolation, a search product.