I think this is right - my experience as JS and similar places is that really understanding your data-generating process and the nitty-gritty assumptions of your data/simple model makes a huge difference

E.g knowing what's causing missing values in your data and what implications various fixes on that might have on bias in your linear regressor is probably way way more valuable than fitting some shiny non-linear toy