There's a class of shortcuts that are good for the individual but bad for the collective.

  • There's a class of shortcuts that are good for the individual but bad for the collective.
    • Those are what will be taken, according to Goodhart's Law.
    • That's the gradient that the swarm will follow.
    • An individual who doesn't care about the collective (or feels that it doesn't make sense for them to do an action that's not good for them) will take an action that gives them a marginal benefit even if it's a catastrophic cost to the collective.
    • The more that the user trusts the collective (believes in it as a long-term thing they want to exist) the higher their threshold will be for what personal benefit they'd have to get to take an action that could harm the collective.
    • But a short-term swarm with no allegiance to the collective will just be a caustic acid.
      • Efficiently taking the individual shortcuts as quickly as possible.

More on this topic

From other episodes