TLDR mathematical representations of physical things are not the things themselves – they are insufficiently abstract, and may introduce nonsense which needs to be “discovered” and fixed later.
It occurs to me (and is often done in various ways) that one should be able to write all physical laws as an abstract function that defines the “physical system” in question, that when passed through a (possibly non linear) abstract functional (function of a function, or “operator”) gives the result equal to zero.
Then later we can argue about what basis is best to use for a particular application to represent the function and functional, knowing that choosing bases and origins may introduce fake degrees of freedom, where we then have to say the answer is such and such times some generator of a bunch of group invariance nonsense.
For example in quantum physics we might begin by saying a single particle has the wave function Ψ. As soon as we say its Ψ(x,y,z,t) we are already on a loser because we fixed four bases and four origins which have to be “unfixed” later. And for a start that means it’s not Lorenz invariant.
But even before then we are still assuming non-physical things by saying Ψ is a complex number. Because actually it should be normalized over the interval of interest in order to give correct probability values – so it’s in a projective space; and also there should be no absolute phase; or if gauge invariance applies, then we should’t be fixing local phase either by assuming the 12 o’clock phase position of one location is the same as every other – especially in the context of space time curvature.
The above discussion show that the usual assumptions about Ψ introduce at least two and possibly an infinite number of spurious mathematical degrees of freedom in the representation of reality.
General relativity while a wonder of beauty, is also terrible, in that it only fixes the second derivative of the metric, and the Ricci tensor is a reduction of the Riemann curvature tensor, so any solution that represents a particular space time is just one of an infinite family of equivalent solutions which also satisfy the same equations and describe the same physics, even if you stick to one coordinate system.
If the math hadn’t introduced non-physical degrees of freedom then Higgs wouldn’t have had to discover/introduce the Higgs field and boson because it would have already been present in the solutions.
I could jokingly claim that the history of physics is a history of people not realizing they are assuming extra degrees of freedom in their equations, and making great discoveries about physics later, which are actually in fact discoveries about math.