> gpt4 still makes dumb mistakes every few sentences, hallucinates stuff when it runs into cryptic words it's not familiar with, and sometimes omits important passages
This, but in all domains where you are an expert - it becomes apparent that GPT-4 makes stupid mistakes. It makes mistakes frequently. Can't seriously see it replacing humans soon, it is far from having four 9's of reliability.
Probably needs 99.99% accuracy to work alone unsupervised by humans, because at each decision step it incurs the error rate again and again, so error rate is growing exponentially in sequence length. Coupled with the "forward only" thinking pattern of LLMs that doesn't allow backtracking and planning this error rate kills autonomy.
Recently, GPT-4-Vision was found to have poor OCR accuracy. Smart but stupid, same story.
This, but in all domains where you are an expert - it becomes apparent that GPT-4 makes stupid mistakes. It makes mistakes frequently. Can't seriously see it replacing humans soon, it is far from having four 9's of reliability.
Probably needs 99.99% accuracy to work alone unsupervised by humans, because at each decision step it incurs the error rate again and again, so error rate is growing exponentially in sequence length. Coupled with the "forward only" thinking pattern of LLMs that doesn't allow backtracking and planning this error rate kills autonomy.
Recently, GPT-4-Vision was found to have poor OCR accuracy. Smart but stupid, same story.