In software you can track the program and detect what instruction is not doing what it is suppose to do. In machine learning there is no program to track, it is not a set of instructions with a purpose, the whole thing either works or not. In order to discover what could be failing, you need to have a deep knowledge about a lot of stuff (maths, statistics, CS) to figure out what is wrong. And sometimes the answer is that the problem doesn't have a solution.
It is very much possible to track your calculations and quite literally debug your models. But like in computer science, it is hard to find the data scientist who can actually do that and not just copy-paste tutorial code from some blog post and then wonder what isn't working...
