Being useful and a stochastic parrot are not mutually exclusive. And I in fact think the opposite. it's Necessary to remind people what it really is, especially in this phase of "Enthusiasm" because I see to many people attributing some meaning or some hidden Insight and especially some innate infallibility to AI nowdays, maybe confused by the name AI.
Right, but most arguments, including the one here, go something like "AI is a Stochastic Parrot so it's a lie and now I think it's bad and we shouldn't do it."
Which is a pretty dumb position imo. Not that I personally think these newer LLMs are a stochastic parrot, or at least not to the degree proponents of the Stochastic Parrot argument would have you believe.
The argument of Stochastic Parrot is not that we shouldn't do it. It's just "Do not attribute any meaning to it".
The one in this article is in the same vane that author thought the AI was learning to program when in reality it was just repeating the most statistically probable combination of the code it had seen. That is, "correctness" is not part of "considerations" that the model does. If the majority of the code that it has scanned contains a particular form of logic bug, it will suggest the same logic bug. The trap is in the fact that the AI will write perfect syntax because that is it's bread and butter and people seeing this perfect syntax attribute also perfect logic to it.
As long as people are aware of this kind of problem, LLMs are a very useful tool that will save a lot of time. But if applied blindly "Because AI knows best" it will create more problems down the road.
