LLMs have flood-fill intelligence.
- LLMs have flood-fill intelligence.
- They appear to be generally intelligent, and they are.
- But actually they just fill in whatever frame you give them.
- They steelman the details, retconning whatever necessary to make the frame you gave them make sense.
- That means that the frame you give them is the ceiling of what they can figure out.
- If you propose a solution and don't invite any uncertainty or questioning of that frame, they'll mostly oblige.
- They don't know when to stop; they keep on filling in details until it's overstuffed.
- Superficially very compelling, but overwhelming and over the top.
- They produce reasoning that is high-quality and impressive in the details… and then sometimes as you zoom out and go, "wait, what?"
- Like that AI slop poster that won the contest for the Ohio State Fair.
- Incidentally this now makes a possibly-crackpot analysis difficult to tell apart from genius.
- Before, if someone produces a compelling detailed analysis, that was only possible if they're onto something.
- But LLMs can make even the most crackpot ideas sound perfectly intelligent.
- The trappings of quality in an argument are now disjoint from the quality of the argument.