Poster comment here. I spent time in the banking/finance world where "Model Risk Management" is a huge thing. I think some of that can be used to think about the risks/controls. It focuses on all the things that can go wrong - a) model doesn't produce an expected result and b) model is used inappropriately.
So, they do things like, performing a model validation which includes rigorous qualitative and quantitative tests, setting limitations on use, testing the sensitivity to inputs and boundary conditions, among many others.
In the traditional sense, a machine learning model is a new beast. In most cases, these models don't have an agreed upon "methodology" (i.e. neural networks) and therefore don't have an "expected" result that can be tested (especially for unsupervised learning models). So, curious to know what others are thinking?
Superintelligence by Bostrom is a decent treatment of some ideas related to this.
Beyond that, there have been a number of stories in the news over the past few years that touch on various risks related to ML.
Things like: automated trading systems causing economic problems. Facial recognition systems not recognizing the faces of black employees. "Predictive policing" creating a "Minority Report" type environment. Etc.
I doubt there's any comprehensive list of "risks of machine learning" though, since there's not really a corresponding comprehensive catalog of "capabilities of machine learning".
So, they do things like, performing a model validation which includes rigorous qualitative and quantitative tests, setting limitations on use, testing the sensitivity to inputs and boundary conditions, among many others.
In the traditional sense, a machine learning model is a new beast. In most cases, these models don't have an agreed upon "methodology" (i.e. neural networks) and therefore don't have an "expected" result that can be tested (especially for unsupervised learning models). So, curious to know what others are thinking?