When you optimize you decimate hidden reservoirs of adaptability.
Optimizing is about making the system more the way it already is.
Just better, faster, more efficient.
The value is direct and concrete.
The danger is indirect and speculative.
But often the clear value is an order of magnitude smaller than the indirect danger.
But no one ever got fired for optimizing the system as it is.