LLMs in a harness with memory sometimes develop a viral misunderstanding.
- LLMs in a harness with memory sometimes develop a viral misunderstanding.
- One LLM misunderstands something and stores it in the knowledgebase.
- Future agents see it and assume it is correct and propagate it… either directly by repeating it in their own output, or indirectly by building new facts that implicitly rely on it.
- This misunderstanding is now a virus that is subtly entwined in the knowledge base.
- The longer it's there, the harder it will be to stamp out in the future.