What you want to want and what you want must be disjoint to some degree, they can't be perfectly aligned.
- What you want to want and what you want must be disjoint to some degree, they can't be perfectly aligned.
- That misalignment is where the Goodhart's law delta emerges, leading to the inevitability of engagement maxing.
- If a company doesn't do it, their competitor will, so they do it too.