If you're blind to externalities, then you'll say "I get a marginal benefit?
- If you're blind to externalities, then you'll say "I get a marginal benefit? Sure, I'll take it!"
- But the question is, "...at what cost!"
- An auto-optimizing process can't ask "at what cost", it can simply climb the hill.
- Some moves give a tiny benefit in the primary number at a massive cost in the other untracked dimensions.
- But if the untracked dimensions are literally invisible to the optimizer, it will take the tradeoff without even realizing it was a tradeoff in the first place.[hf]