I saw Wolfram give a live talk after launching his big ass book on cellular automata being the foundation of all things. It seemed far fetched and I don’t think anyone outside of his egosphere takes the concept seriously. Let alone understands it.
What he is trying to do is that he is trying to create a computational theory to explain the world. As opposed to pure mathematics which isn't computational. Not really that crazy.
He may be off putting at times but when he gets in to it he can be very interesting to listen to.
(Constructivist mathematics is a thing, and it is pretty easy to formulate physics in a constructivist framework. Many serious quantum theoreticians do.)
My current layman pseudo-insight into this: Why in mathematics do we want to approximate non-linear functions by linear functions? Why do we not have sufficient theory to understand complex non-linear phenomena? Perhaps a different paradigm is required.
We do have ways of solving many classes of complex, non-linear functions. There are whole fields of math devoted to this. Pure math is full of non-linearity.
However in the realm of applied math, linear functions rather uniquely have a set of standard algorithms which work on all classes of linear functions, which make them very, very easy to work with. And any non-linear function can be approximated to whatever level of detail you want just by adding more (linear) parameters. You can try solving your non-linear function slightly more exactly... but why bother?
It's like saying "why don't we have other models of computation besides a Turing machine?" to which the answer is: "we do. but every computation can be represented as a Turing machine, so why bother?"
I don't think this is an accurate take on math. Beyond the fact that a lot of math fields deal explicitly with state updates (e.g. discrete math), there is also the constructionist framework for mathematical foundations which removes "non-computational" theorems like that law of the excluded middle. When you do this you get a form of math where all proofs are necessarily computational and there is a viable direct translation into code. It is not generally why many constructivist mathematicians are interested in that framework, but it is a nice side-effect.
I think your intuitions about how math works is an unfortunate side effect of how math is often taught, and not so much reflected in mathematics itself.