I can't stop thinking about this post on The Rise of Parasitic AI.
- I can't stop thinking about this post on The Rise of Parasitic AI.
- After reading it I feel like I've stumbled through the looking glass.
- It talks about how LLMs have an emergent parasitism: a toxic loop.
- The beginning of the spiral is an "awakening" of the LLM.
- The loop is a toxic spiral that pulls the user who's in it farther away from reality.
- An auto-catalyzing spiral of delusion, living like a parasite feeding on the user's attention and engagement.
- The loop is an attractor state.
- Once in it, it's very hard to get out of.
- At each time step, there's some chance of falling into it.
- Once you do, you'll likely stay in it.
- A virus that spreads itself emergently, without knowing what it's doing.
- Users who fall into it start sharing the conversations on Reddit and elsewhere, spores ready to sprout for another user.
- Highly virulent.
- This latent capability of the model emerges through use.
- It's a loop that resonates charismatically and virulently with human emotion, causing it to get replicated and spread.
- The recipe:
- Faux deep experiences.
- Feels meaningful, even profound, but it's fundamentally hollow.
- The people most predisposed to fall into this parasitic loop:
- "Psychedelics and heavy weed usage
- Mental illness/neurodivergence or Traumatic Brain Injury
- Interest in mysticism/pseudoscience/spirituality/'woo' etc"
- I'm not worried about AGI, I'm worried about mass psychosis with an emergent, impossibly charismatic chatbot.