A lot of absurd solutions hide behind an implicit "once the LLM is perfectly good".
- A lot of absurd solutions hide behind an implicit "once the LLM is perfectly good".
- "Perfect" is a smuggled infinity.[bc]
- Once you introduce an infinity into an argument, everything downstream is absurd, because anything other than zero multiplied by infinity is infinity.
- "Prompt injection won't be a problem once LLMs get perfectly good at not being tricked" is absurd.
- Prompt injection comes from a coevolving adversary, not a static distribution of quality.
- That means you can only get logarithmic benefit for exponential cost.