> Can you point me to a similarly principled approach to decision making based on utility with frequentist methods?
Sure, as I said anything involving ROC curves, where we pick an operating point by trading off the cost of false positives vs. false negatives and a design rate of incoming true positives and negatives.
Can you give a mathematical recipe with assumptions and deductions? ROC curves don't cut it, it's just twiddling of a parameter of a classifier. Is it optimal in any sense? How do you know it's a good classifier for making decisions, when it's a classifier based on "given the hypothesis, how likely is the data" and not the other way around? Is it generalizable to other situations?
Yes, given a design rate of true positives or negatives, and a cost for false positives and false negatives, you can pick the optimal operating point. It will be optimal in the sense of minimizing average cost when the incoming rate equals the rate you designed for. You'll get the exact same answer as a "Bayesian" who uses conditional probability to calculate the same thing and whose prior equals the design rate. I gave a worked-out example in my original post ("If you want to decide whether to take action...").
Sure, it is generalizable -- we use ROC curves for radar, medical imaging, almost any diagnostic test...
Sure, the problem is not which point on the ROC curve you pick, the problem is which classifier you use to obtain it in the first place. I can pick a random classifier with a tunable parameter and draw its ROC curve and then pick the "optimal" point, but if the classifier sucks then that's no good. Why would a frequentist classifier based on a hypothesis test be good? A hypothesis test is the answer to the wrong question for the purposes of making a decision.
As I showed above, you can indeed get the same result from Bayesian decision making if you use a weird prior and utility function, which shows that frequentist decision making based on hypothesis tests is a subset (of measure 0) of Bayesian decision making. Again, that just means that you encoded a most likely wrong prior and utility in the choice of method without any justification.
Sure, as I said anything involving ROC curves, where we pick an operating point by trading off the cost of false positives vs. false negatives and a design rate of incoming true positives and negatives.