Why is Goodhart's law such a fundamental, unstoppable force?

· Bits and Bobs 6/9/25
  • Why is Goodhart's law such a fundamental, unstoppable force?
    • Goodhart's law arises because the metric must be a proxy.
      • A map is not useful if it's 1:1 with the territory.
      • Its leverage comes from how much of a useful subset it can be.
    • Because it's a proxy, there are ways to "cheat" that improve the proxy but not the reality.
    • If the members of the swarm are not fully committed to the good of the collective (if there's any principal agent problem) then they will be optimized to cheat because it is a cheaper way to improve the proxy.
    • You can get the swarm to not Goodhart's law if all of the members feel an infinite connection to the collective over their individual incentives.
    • This can happen if they view themselves as fundamentally a subcomponent of the swarm, and only secondarily as an individual.
      • This can happen if the group all believes in the same infinity together.
    • That can also happen for example if they are perfect clones of one another.
      • But perfect clones won't have variance, so the system overall will be less resilient.

More on this topic

From other episodes