based on the code I've seen come out of these tools, I'm denying they enable any productivity gains. The code is either hilariously bad, filled with subtle bugs, or just something I could have googled then copy and pasted an OSS solution into my code.
Is anyone's productivity really higher if we have to spend all our time correctness checking the output?
We should test it and your ego at the same time. We might find some interesting results.
Find someone at the same level of competency as you, set a goal, that person gets access to any LLM they want, you have to do it with no assistance other than reading stack or googling. And if you think your ego can handle it, find someone who you know is below you in competency (but not junior) and set the same goal and see if they surpass your own productivity or progress.
We’d have to set up some controls but you see where I’m getting at. It would be an interesting experiment either way.
It’s all in how you use the tools. If you’re zero shotting and hoping for the best then yeah, not great. But that’s where your domain expertise is supposed to come in.
Is anyone's productivity really higher if we have to spend all our time correctness checking the output?