It's not that Goodhart's Law just so happens to find shortcuts.
- It's not that Goodhart's Law just so happens to find shortcuts.
- It's that the gradient the swarm descends fundamentally are shortcuts.
- The ideal vector is what all of the members of the collective, if they didn't know which member of the swarm they were, would pick.
- The veil of ignorance, where the individual views the system from the collective's perspective.
- The local incentive vector is what will give the individual the biggest bang for the buck.
- But the ideal vector for the collective will always be different from the individual's ideal vector.
- Sometimes a little, sometimes a lot.
- The difference between them is what makes it a shortcut.