The Stripminer's Bargain is what has hollowed out society.
- The Stripminer's Bargain is what has hollowed out society.
- The bargain: a real, immediate gain on what you measure, in exchange for a cost you can't see.
- It sounds reasonable.
- Every individual step is an improvement.
- It's a deal with the devil, except there's no devil.
- Just an optimizer that doesn't care what it gives away.
- Three ingredients make it ruinous:
- 1) Optimizers will take an epsilon gain on the dimension they're graded on, even at catastrophic cost to the dimensions they aren't. The unmeasured is free to spend.
- Russell's Catastrophe.
- 2) Direct effects are fundamentally easier to see and measure than indirect ones. So the gains are always counted, and the costs never are.
- This creates an asymmetry.
- The hotter you run it, the more it hollows things out.
- 3) We optimize hotter and hotter, across every dimension of society.
- It runs until everything that held it up has been strip-mined away, and all that's left is a hollow shell.