> But on the other hand complex models such as Neural Networks are being used for very wide datasets with incredible amounts of parameters, so that understanding a single parameter's contribution is not useful in the real world.
In the near future AI and statisticians will need to cooperate by:
1. Finding means for extracting rules/general principles from neural networks.
2. Creating new fields of statistics that can handle multi-dimensional data sets, such as by representing them as small-dimensional datasets that interact (?), which converges to the same multidimensional model when the modeler sutures these interactions (?) together according to a topological structure.
We know that we can do this because somehow human beings reason about complex systems successfully.
There’s a whole field around 1 called explainable ai. Interestingly, one of the SOTA techniques, SHOP values, comes from economics (game theory).
Regarding the article, I think it was a good read, but as a data scientist on a research at Amazon, there’s a reason our interviews have shifted from stats-heavy to more cs-heavy: the ability to actually implement and maintain analytic products is just more useful. (Note, this isn’t across the board - we still hire PhD-level candidates to do research).
In the near future AI and statisticians will need to cooperate by:
1. Finding means for extracting rules/general principles from neural networks.
2. Creating new fields of statistics that can handle multi-dimensional data sets, such as by representing them as small-dimensional datasets that interact (?), which converges to the same multidimensional model when the modeler sutures these interactions (?) together according to a topological structure.
We know that we can do this because somehow human beings reason about complex systems successfully.