Yes, I use LLMs every day. Primarily for coding (a mix of Claude and OAI). I was trying to implement a simple CSS optimization step to my JS framework's build system last night and both kept hallucinating to the point (literally inventing non-existent APIs and config patterns) where I gave up and just did it by hand w/ Google and browsing docs.
The problem with your "close to the metal" assertion is that this has been parroted about every iteration of LLMs thus far. They've certainly gotten better (impressively so), but again, it doesn't matter. By their very nature (whether today or ten years from now), they're a big risk at the business level which is ultimately where the rubber has to hit the road.
Yeah I don't think we're going to come closer to a real AGI until we manage to make a model that can actually understand and think. An LLM sounds smart but it just picks the most likely response from the echoes of a billion human voices. I'm sure we'll get there but not with this tech. I'm pretty sure even be OpenAI said this with their 5 steps to AGI, LLMs were only step 1. And probably the part that will do the talking in the final AI but not the thinking.
At the moment people are so wooed by the confidence of current LLMs that they forget that there's all sorts of types of AI models. I think the key is going to be to have them work together, each doing the part they're good at.
> An LLM sounds smart but it just picks the most likely response from the echoes of a billion human voices.
This is where reasoning models come in. Train models on many logical statements then give them enough time to produce a chain of thoughts that’s indistinguishable from “understanding” and “thinking”.
I’m not sure why this leap is so hard for some people to make.
I personally don't think that will go very far. It's just a way of extracting a little bit more out of a technology that's the wrong one for the purpose.
The problem with your "close to the metal" assertion is that this has been parroted about every iteration of LLMs thus far. They've certainly gotten better (impressively so), but again, it doesn't matter. By their very nature (whether today or ten years from now), they're a big risk at the business level which is ultimately where the rubber has to hit the road.