17 chunks · 16 episodes
...is infinite. The absurdity in the analysis is entirely due to the infinity. The smuggled infinity strikes again! The frontier labs (and their investors) have all implicitly accepted Altman's Wager. If you don't accept the wager then the frontier l...
...-imminent argument has a problem: The Smuggled Oracle. Building on Ben Mathes's Smuggled Infinity frame. Everyone assumes that because we've gotten to recursive self improvement for coding that general ASI takeoff is also close. But coding is a ve...
A smuggled infinity argument can lead to accidental security nihilism. "Mechanisistic containment boundaries for powerful models can never be perfect, so we are doomed a...
...ral. If you introduce one, it infects everything. The same logic as Ben Mathes' Smuggled Infinity.
Smuggled infinity is a kind of accidentally magical thinking. It's magical without you realizing it, which means you're in the realm of fantasy while you think you're ...
"Magic" is another example of the smuggled infinity. If your system assumes a magical oracle, then no matter how many layers it's pushed down it's still magic, and thus non-viable. Prompt injection app...
...utions hide behind an implicit "once the LLM is perfectly good". "Perfect" is a smuggled infinity.[bc] Once you introduce an infinity into an argument, everything downstream is absurd, because anything other than zero multiplied by infinity is inf...
... to do and more because they have a poor understanding of security. It's also a smuggled infinity. "If only models were perfect at not being tricked this would be safe." Remember, the threat coevolves; as these tools are more used, the incentive f...
People who believe "the model is the product" are potentially falling prey to a smuggled infinity. "Simply make the model smarter and give it more tools and it can do everything." A maximalist take. "it will make better decisions than humans at so...
"Perfect" is a smuggled infinity.[ng][nh] A smuggled infinity narrative is useful to get coordination on big projects. Even if the vision is impossible because it has a smuggled infi...
...s you have the perfect accounting of misery and thriving points it's easy" is a smuggled infinity. Everything past the "perfect" is absurd, because perfection is impossible in complex environments.
...ively equivalent to "This would be easy if communication was perfect" This is a smuggled infinity. Communication is fundamentally imperfect and lossy and squishy and frustrating. This is the curse of knowhow.
...nificantly beyond "good enough" requires infinite time and resources. This is a smuggled infinity. Everything is fundamentally only done to a "good enough" bar, because perfection is impossible. At some point you have to stop investing time and en...
...orld isn't perfect so it's hopeless". Both are erroneous reasoning because of a smuggled infinity of perfection being achievable. Once you embrace that perfection isn't possible, you can lean into making as much good happen as you can.
...Cory Doctorow. A seemingly reasonable assumption that is secretly impossible. A smuggled infinity. Everything downstream of that assumption is also impossible. In fractally generated, natural systems, you hit a ceiling when trying to formally mode...
... that's easy to fall into "We decided X, therefore X is right." This is another smuggled infinity: "[We are infallible, because we're the main characters.] We decided X, therefore X is right." But if X turns out to not work in practice, then be op...
An unchallenged belief in one's own perfectionism is a smuggled infinity. The intuition of a smuggled infinity: "If you slip an infinity into any argument you'll end up in an absurd place" It's easy to accidentally slip in...