Swarms are adaptable but have Goodhart's Law.
- Swarms are adaptable but have Goodhart's Law.
- The antidote is trust in the collective and long-term goals.
- When individuals trust each other to behave as a collective they believe in, they will take actions that don't follow Goodhart's Law and don't destroy the collective.
- Instead of only optimizing for their local incentive gradient, they also balance what's best for the broader collective.
- That gives you the best of both worlds.
- Similar to Francis Frei's observation that diverse teams that trust one another are the way to get reliably great results.