Even if the success rate of the human eye was in the 99.5%+ range, why not have an extra sanity check from an AI model?

You're confusing False Negatives with True Negatives. For Non-Benign (Positive) vs. Benign (Negative) classification:

* True Positive Rate (TPR): non-benign classified as non-benign.

* False Positive Rate (FPR): benign misclassified as non-benign.

* True Negative Rate (TNR): benign classified as benign.

* False Negative Rate (FNR): non-benign misclassified as benign.

> It's quite easy to correctly classify 100% of benign cases as benign.

You can engineer a 100% TNR if you just classify everything as the "benign" negative class. The FNR is going to be 100% too, but that doesn't matter -- you correctly classified 100% of benign cases as benign.

> why do people die from having lesions misdiagnosed as benign?

Because the FNR is not 0%. FNR is important. You probably want a decent TPR in there as well. And FPR can be very important too, depending on how life-changing/painful/invasive the treatment for a positive case is!

    def is_benign(mole: Image) -> bool:
        return True