Anthropic announced Claude for Chrome this week.
- Anthropic announced Claude for Chrome this week.
- Their blog post announcing it mentioned it will be available to a small set of users because they haven't yet made it safe enough.
- They shared their stat of attack success rate: 11.1%.
- It's multiple orders of magnitude too high to be safe for mass market use.
- The majority of the blog post was about prompt injection, which basically guaranteed that all of the press coverage was mostly about the danger.
- Notably, articles I've read about other AI browsers also mentioned prompt injection this week, due to Anthropic's blog post.
- This category is structurally impossible to make safe for the mass market today.
- Even Brave, who pointed out flaws in Perplexity's Comet, is likely mostly vulnerable to the same class of attacks, even if not so egregiously as Comet is.
- Vivaldi's response to AI browsing is "we won't do it for moral reasons" which looks kind of weak… people might think, "maybe they just couldn't get it working well enough?"
- Here's a random theory: maybe Anthropic is trying to put a stake in the heart of the so-hot-right-now AI browser category?[ba]
- Imagine if you thought that it was structurally impossible to make this feature safe, but since everyone was getting into the fray you looked weak if you didn't.
- A way to do that would be to do a demo that shows yours works pretty well… but that you consider unsafe in its current form, and then set a yardstick that everyone else will fail, too.
- Anthropic sharing its "catastrophic" attack success rate number begs the question… what is everyone else's attack success rate?
- Almost certainly they are much worse than Anthropic's.
- That could put a low ceiling on the whole category.