A nice distillation of the core observation I was getting at with the Smuggled Oracle:

"A self-improving agent can only get as good as the test that scores it."

"The test does not merely measure progress, it defines it, so the efficacy of the test becomes a ceiling the agent cannot climb past."