Your comment/question seems to assume that somehow accurate classification of performance is possible, which is demonstrably false assumption.
First, if somehow system-internal watchdog was able to detect erroneous outputs for training, it would also be able to stop those outputs propagating to control system live, leading to zero errors. Such a watchdog requires self driving to be quantified analytically, leaving implementation entirely testable and therefore control system not passing the testsuite (i.e. exhibiting erroneous behavior, i.e. still having "edge cases") not deployable in public. Many words to say that that in practice with driving being somewhat loosely defined you need humans in the training loop.
Second, vehicle operators are not only unqualified to accurately monitor behavior of these autonomous systems, in part due to not being formally trained on safety systems in general and the system they are monitoring in particular, but on top of that incentivized to underreport erroneous output.
My prediction has been that we are going to see increased number of accidents with ADAS deployments, not less, before the number can start dwindling.
Your comment/question seems to assume that somehow accurate classification of performance is possible, which is demonstrably false assumption.
First, if somehow system-internal watchdog was able to detect erroneous outputs for training, it would also be able to stop those outputs propagating to control system live, leading to zero errors. Such a watchdog requires self driving to be quantified analytically, leaving implementation entirely testable and therefore control system not passing the testsuite (i.e. exhibiting erroneous behavior, i.e. still having "edge cases") not deployable in public. Many words to say that that in practice with driving being somewhat loosely defined you need humans in the training loop.
Second, vehicle operators are not only unqualified to accurately monitor behavior of these autonomous systems, in part due to not being formally trained on safety systems in general and the system they are monitoring in particular, but on top of that incentivized to underreport erroneous output.
My prediction has been that we are going to see increased number of accidents with ADAS deployments, not less, before the number can start dwindling.