Bingo. I've made this comment several times on Hacker News in the past as well and in my opinion it's the number one reason I've seen ML projects fail to have impact at companies it's deployed: the operators (typically higher level Math/CS types) simply don't understand the domains well enough and so frequently end up making absurd recommendations/suggestions (often to the detriment of other business areas).

The successful application of ML requires a deep understanding of the domain it's being applied in.