For evals to give you a gradient of improvement, the eval has to not be saturated.

2025-09-22 · Bits and Bobs 9/22/25

For evals to give you a gradient of improvement, the eval has to not be saturated.
- If it's saturated then the gradient gives you Goodhart's Law.
- It pulls you towards optimizing something that does not actually improve what you care about.

More on this topic

From other episodes

The Principal Agent Problem is a specific subclass of the broader Goodhart's Law phenomenon.2026-03-09
Every swarm has an emergent Goodhart's Law.2026-03-09
There's a class of shortcuts that are good for the individual but bad for the collective.2025-11-17
Optimizers will choose gains on the target metric at catastrophic cost to unmeasured externalities.2025-11-10
James Evans gave a fascinating talk I attended this week.2025-11-10
Hyper financialism is just Goodhart's Law.2025-11-04
Excellent piece from Ben Mathes on Goodhart's Law and "Lowest Common Consensus".2025-11-04
It's not that Goodhart's Law just so happens to find shortcuts.2025-10-27