LLMs are great at finding zero days not because they're smart but because they're parallelizable.
- LLMs are great at finding zero days not because they're smart but because they're parallelizable.
- It's not that they're smarter than a human security researcher; it's that they're able to scale their search much more than humans can.
- Humans can do one lane of analysis at a time, but LLMs, even a single instance, can swarm multiple search heads at once.
- The parallelizing of the search is best modeled as patience.
- They find things that humans could find… if only they had infinite time and patience.
- A great HackerNews comment:
- "To put this into game theory lingo, I think this is because the 'branching factor' for any kind of research or exploit is extremely large.
- So looking backwards it doesn't seem complicated, but looking forwards there are an enormous number of possible next actions.
- Similar to finding a filament for lightbulbs, it might seem obvious to use tungsten, but at the time it wasn't and Edison searched thousands of materials."
- And a response:
- "I think the models are legitimately doing what they're good at; tireless search across an extremely large corpus of data.
- Humans aren't particularly good at this (in fact, they're absolutely terrible).
- The fact that we remain competitive and superior in many aspects isn't because we can instantly sift through tons of data, it's because we learn and correlate and have superior heuristics.
- In my own use, I find that AI is really good at finding bugs that are ultimately trivial but require searching through a convoluted series of inter-related files.
- This takes time for a human."
- They are also arguably better at figuring out where to search.
- AlphaFold has detected some fundamental underlying pattern that generates the output that eludes us.
- Presumably Fable-class models have also uncovered a fundamental underlying pattern of zero days.